Searching for Commitment

The world loves automobiles, and nobody loves them more than Americans do. What other country writes pop tunes about its cars, from "My Merry Oldsmobile" in 1905 to the Beach Boys' "Little Deuce Coupe," not to mention the travails of daddy taking one's T-Bird away. At times, that love of cars extends to even their engines. Today, the only automobile engine that boasts good vibrations (of a different sort) may be the Toyota Prius—a techno-marvel sporting Bluetooth wireless networking. In the 1970s, Oldsmobile owners fawned over their Rocket engines. Those customers were hopping mad when GM secretly switched the Rocket with a Chevy power plant.

Substitute Web sites and CMS for cars, and search systems for automobile engines, and you get some interesting parallels. I'm not humming a tune about Google, but that company rocks, and its engine is very popular. On the content side, Adobe can claim universal acceptance of Acrobat and its built-in search. Most large firms have made long-term commitments to a single enterprise CMS from the likes of Documentum or FileNet, or to a single database vendor like Oracle. Each such commitment is also an indirect commitment to that vendor's search system.

Over the past 18 months there has been an engine-switching phenomenon similar to what GM did with the Olds. Oracle (which invested development resources and PR in its own search system) recently purchased TripleHop, an excellent enterprise search vendor. In Acrobat 6, Adobe switched out Verity with Lextek's Onix for indexing and searching PDF files. Recently, Documentum replaced Verity with FAST. Not only were the Verity engine switches made without consulting customers, but IT support representatives themselves didn't seem to mind. It's as though search engines—like many auto engines—have become a commodity. I was very surprised at the lack of outrage in the trade press and online discussion groups. Clicking a search button is presumably like stepping on the gas; press down and you go.

I polled a recent Documentum discussion group to gauge reaction to this engine switch, and I was surprised that most respondents essentially said it was no big deal. To paraphrase the group response, "Can't you use any search system with Documentum?" Yes, you can, if you want to purchase a license for a new one and possibly pay to integrate it with Documentum. EMC's Documentum (like most content management systems) bundles an OEM version of a specific search system, so searching is essentially free. Bundled search systems usually have also developed deep integrations with the host content management system, allowing you, for example, to select all metadata and content to perform precise searches. When you use that "free" engine you create an infrastructure that includes training, IT support, and users who probably have invested lots of time and mental energy developing and saving search queries that leverage the search system. Often customers cannot quickly upgrade to the newest version of anything since they must carefully test the implications of an upgrade. If you are in the pharmaceutical industry where you must validate and prove that your systems do what they are supposed to do, you probably spent lots of time and money revalidating your search system due to index and query incompatibilities. Acrobat 6's search system can read earlier Verity-based indexes, but the reverse is not true. If you or your PDF customers didn't have the luxury of upgrading all users and indexes to Acrobat version 6, some searches failed.

It gets worse. If you have planned ahead and implemented a sophisticated, federated strategy for searching, engine-switching causes even more problems. Systems like EMC's Enterprise Content Integration Services map search queries to a high-level model so you can translate enterprise searches into the syntaxes of different repository search systems. Switch the search engine and you need to overhaul your map to make it work with the new engine. Even then, your "search results may vary" depending on the richness and similarity of the new engine's search syntax to that of the older system.

There are two key issues here: First, will search systems become commodities, so switching them doesn't matter? I believe they have not and maybe never will completely. Search engines vary so widely in the content they search, their techniques for performing searches, and what they do with results, that significant search standards (and commoditization) will not emerge soon, if ever. Secondly, how can you nudge your content vendors towards appropriate standards? One source of relevant standards is the W3C, with emerging standards to capture and express meaning. Among these standards is the Resource Description Framework (RDF). A quick search on the W3C's site reveals which vendors have participated and are therefore likely committed to standards.

If nothing else, vendors: remember what happened to the once popular Oldsmobile. Brand loyalty wasn't enough to save it.