Get Smart with Business Intelligence Software

Page 2 of 3

      Bookmark and Share

Mining Unstructured Data
With the prevailing wisdom that 85% of the data in an enterprise exists in sources other than databases, an awful lot of valuable data can be left behind by tools that search in only structured databases, and if you don't look at this vast collection of data, depending on your needs, chances are you'll be missing something. To get at unstructured data, you need a different toolkit. Vesset from IDC thinks the unstructured side of the market is growing as companies develop tools to get at this data.

He says, "It's a hot emerging market and it's faster growing than structured data analysis. Some of the BI vendors are getting into it such as SAS and Insightful. Both have come out with text-mining solutions, the ability to mine unstructured text and then integrate that with your analysis. I think those two areas (structured and unstructured) will continue to merge." Although Vesset says it's early in the merging process and structured and unstructured for the moment are treated separately, there are industries such as oil and gas, manufacturing, and pharmaceuticals where they need to look at unstructured data as part of the nature of their business process.

Jeffrey Coombs, senior vice president of sales and marketing at Insightful—makers of both structured and unstructured data tools—agrees with Vesset's analysis. His company has found that industries such as pharmaceuticals benefit from having both types of tools. He says, "Within pharmaceuticals is a very large requirement that was previously unmet regarding looking at unstructured data." This is the ability to not only research a drug, but to also check the literature to make sure nobody else has already looked at it."

Another way to see these kinds of relationships across unstructured data is using software such as Inxight SmartDiscovery to build a taxonomy that provides a visual map of data relationships. David Spenhoff, vice president of marketing at Inxight, says, "What we enable people to do is to analyze text-based data." They then present the data in a visual map that shows the relationships between different key entities as defined by the end-user. The information could come from inside the firewall such as documents, PDFs, PowerPoint presentations, or email; or from outside the firewall such as Web pages or newsfeeds.

Climbing the Firewall
Of course, a whole world of information lives beyond the firewall and, if companies want to keep track of their competitors, follow industry news and stay on top of information outside the company, they need to be able to get at that information, too. Much of this information could be on Web sites or it could be in online Web-based databases like Dun and Bradstreet. The type of software that allows users to collect, aggregate, and report on information on the Web is only just beginning to emerge. IBM's WebFountain has received a lot of attention in the press and a British company, Anacubis, is beta-testing a new product called Anacubis Desktop, which collects information from online databases and presents it in a way that you can aggregate and report on.

Of course, a whole world of information lives beyond the firewall and, if companies want to keep track of their competitors, follow industry news and stay on top of information outside the company, they need to be able to get at that information, too. Much of this information could be on Web sites or it could be in online Web-based databases like Dun and Bradstreet. The type of software that allows users to collect, aggregate, and report on information on the Web is only just beginning to emerge. IBM's WebFountain has received a lot of attention in the press and a British company, Anacubis, is beta-testing a new product called Anacubis Desktop, which collects information from online databases and presents it in a way that you can aggregate and report on.

IBM describes WebFountain as a tool to gain access to information that is not otherwise readily available such as people's perception of a brand or product. IBM's Jhingran uses a product launch as an example. He says, "Think about a marketing manager. You've just launched a product and you want to know what people are saying about it or what the competition is doing to counteract that. Does it have a positive buzz or a negative buzz." WebFountain looks for this data in unusual places such as chat rooms, advertising sites, competitor's sites, or newsfeeds such as Factiva and tries to build a picture for the marketing manager as to what is happening with the new product.

Anacubis takes a different tack. They have partnered with several Web-based subscription companies including D&B and LexisNexis, and have developed a desktop product to help end-users makes sense of the information they find in these databases. The companies still need to subscribe to the fee-based information services, but they have a tool to help them see patterns and trends that might not otherwise be obvious. Rebecca Pointer, marketing manager at Anacubis describes it as follows. "What Anacubis Desktop does is support all parts of the information flow, so gathering information from multiple sources, consolidating and organizing that data, and then allowing users to perform some real analysis of the data they are looking at."

As information resources continue to develop, it becomes imperative for the enterprise to get a grip on data to help employees understand the business, products, relationships, and competitive landscape. There are a growing number of software packages and services to help do that. Perhaps these tools will eventually merge to allow you to capture structured and unstructured data both inside and outside the firewall from a single tool, but until then, depending on your needs, you may need an information gathering toolkit to be certain you gather, analyze, and understand all the available data, and finally, communicate what you have learned to the rest of the organization.

Page 2 of 3