Complex Search @ Your Web Service

Page 1 of 3

      Bookmark and Share

BEST PRACTICES SERIES

As enterprise users face growing repositories of valuable data, it becomes increasingly important to be able to search across these data storehouses (and the Web) to find the best available answers to a query, then to effectively apply that data. While enterprise search technology has been capable of searching multiple repositories for some time, it required a great deal of programming and computing overhead and didn't necessarily allow users to manipulate the results. XML and other Web Services have changed all that, making it possible to search multiple repositories and across various Web sources and then use the data in various ways, while using fewer resources.

This means that the enterprise user can not only produce more pinpointed search results but also use that information in interesting ways as programming logic is applied to those results. For owners of content, Web Services provide a way to use the metadata associated with content to slice and dice the content in new and meaningful ways for customers and internal users alike, to combine the data with other Web Services or internal applications, or to incorporate data from searches into the fabric of the enterprise where it is needed most. What's more, search engine companies have opened up their search components (and other services) as Web Services and invited programmers to come up with creative ways of producing custom sets of results.

Searching for Meaning in Web Services
Web Services have been around for a number of years and take advantage of the ubiquity of the World Wide Web, metadata tagging, and coding standards like XML to deliver a unique solution inside a Web browser. Tim Matthews, co-founder and VP of marketing at Ipedo, an enterprise information integration company, says that when it comes to search as a Web Service, it's all about what users can actually do with the results. "In general, I would say if you are looking to do something programmatic with the results, search, as a Web Service, gives you a great advantage because it normalizes the search results in a form you can manipulate." This means that a search solution can examine the tags (metadata) associated with a particular item and simply render the results in a page, or more importantly, use the tags to combine with other sources to produce more meaningful results.

Prior to the Web Services model, companies typically used a database to locate information, but this approach had inherent limitations, according to Max Schireson, VP of customer solutions at Mark Logic, which offers an XML content server product. "Traditionally, if you wanted to go beyond text search, you would have to take content and shred it in a relational database. It required that you know a lot about your content to be highly consistent, and you lost your flexibility going forward."

Another way this might have been done, according to Frank Gilbane, CEO of Bluebill Advisors, a content management consultant and analysis firm, is to build a federated search, a program that checks several different repositories and builds queries across these different systems, then pulls them together in a single list of results.

With Web Services, instead of using a program to retrieve the data, the data is tagged with metadata and compiled. Then, using this information, you can build programming logic to manipulate the results. Bradley Allen, founder and CTO at enterprise search vendor Siderean, says it doesn't affect his company's application whether the content comes from a structured database with relational tables, a content management system exposing XML feeds, or even information out on the Web. "We bring that information into a central metadata repository set up to extract information from those sources, categorize them, and put them into the context of a metadata-level description, and then index those in a way to provide low latency navigation services." He says this information can then be presented as a page of results or the user can slice and dice it or take queries and turn them into RSS feeds that they can plug into other applications such as an RSS reader. Schireson says this type of data flexibility is not possible in a traditional search model without significant coding and performance overhead.

Page 1 of 3