The Long Tail of Knowledge: Big Data's Impact on Knowledge Management

Apr 11, 2014


Article ImageMaking sense of large amounts of disorganized information that is spread across wide swaths of an organization has always been the defining challenge of knowledge management. In recent years, the ability for organizations to capture information about themselves, their customers, and every facet of their business has increased exponentially; the phenomenon, of course, is known as "big data." As troves of data grow, so too does the potential to leverage it, and one key to keeping up with that potential is through the implementation of systems and solutions that streamline your organization's ability to analyze it, access it, and act on it. In short, big data is spurring the symbolic "long tail" of institutional knowledge to grow ever longer, and organizations must be prepared to adjust their knowledge management strategy to compensate.

"The problem in many organizations is that 60 to 90% of problems have already been solved," says Phil Verghis, the CEO and a co-founder of Klever, a knowledge management consultancy that offers what it calls "Knowledge as a Service (KaaS)." "This means that they're repeating the same things over and over again."

Big data, Verghis says, creates a flood of information where once there was just a river. The same principle that knowledge still exists within an organization's data still holds, but the challenge to funnel it into a manageable stream is an ever greater challenge.

Verghis points to the example of Tyler Technologies, a Texas-based software solutions provider that works on financial management, case management, and taxation projects for local governments and schools. In 2013, Tyler took on the state of Texas as a client, which massively increased the amount of information, as well as the types of information, that they would be working with.

Changing behavior, Verghis says, was the key to Klever's prescription for Tyler Technologies. "Instilling a belief in all of the individuals in an organization that they are owners of both their own and the company's knowledge is essential," he says. By coaching employees to manage the data that they create, organize unorganized data when they encounter it, and take accountability for data and knowledge at every step of their work process, the principles of knowledge management are scalable as data grows. By applying Klever's knowledge management practices to its vastly expanded data sets, Tyler was able to effectively take on its new client without any significant impact on the rest of its work.

The case of Tyler Technologies points to one of the significant changes that big data is bringing to knowledge management within organizations: It's not just a set of practices for customer service anymore.

Diane Berry, chief knowledge evangelist at Coveo, a search and relevance technology provider says, "An embrace of all of your data is important to generating the value of that information. You don't want to be recreating the wheel over and over again. It is a drag on productivity."

Data is growing within organizations of every size, from the giant online retailers to small nonprofits. As such, it's not just the big guys that stand to gain from learning how to manage their knowledge. Nor, says Berry, is knowledge management outside of the grasp of little guys.

"One of the keys is making organizations understand that knowledge management technology is available across industries and sizes of organizations, and that it is cost effective and easy to implement," she says. "Coveo packages can be deployed in weeks and months, not for millions of dollars and over the course of years. All organizations can begin to do for their employees what they used to do for their customers."

Mark Beyer, who is a research vice president at Gartner and a co-lead for Big Data at the firm, urges caution that while big data has the potential to create value for an organization, there are a number of myths about big data to keep an eye out for.

The first myth that Beyer cautions about is the reliability of user-generated data. "One major myth about big data is that social data can immediately increase accuracy of marketing campaigns," he says. "There are many kinds of social data -- social profile data, for instance -- most of which are created by the user themselves. Self-reported data is inherently unreliable, and in order for that data to be reliably actionable, you need second, and sometimes even third and fourth sources to validate it."

Another significant caution that Beyer offers about big data is the bias inherent in any organization's data. This bias can operate in many ways on data -- for instance, an online retailer can only accurately understand the buying habits of its customers for its own products, not its competitors.

In order to best use content with the bias inherent in organization data, Beyer says, managers and executives must be able to analyze the strengths and weaknesses of their knowledge workers, and apply them to tasks and projects based on how well they demonstrate an ability to see past bias to the core facts of datasets.

As data collecting becomes more and more cost-effective, efficient, and ubiquitous, the term "big data" grows increasingly obsolete. Ultimately -- or perhaps already -- big data is the new normal. As this increasingly becomes the case, Beyer urges organizations to stop thinking about big data as a special case.

"The characteristics of big data are really just a magnification of those that we've been dealing with for the past twenty years in knowledge management," Beyer says. "It's not an extra special circumstance apart from normal workflow, and shouldn't be isolated. Knowledge workers are going to have to get used to it."

(Image courtesy of Shutterstock.)