The Sorry State of Search Satisfaction

Working as a consultant, mainly on short-term assignments, I rarely know such details as exactly whom I will be working for more than a couple of months ahead. As a result, I look forward to certain fixed points in the year with special anticipation. March is the IntraTeam intranet conference in Copenhagen, May is the Enterprise Search Summit in New York, and November brings the J. Boye conference in Aarhus, Denmark. I also look forward to James Robertson's Intranet Innovation Awards and Jane McConnell's "Global Intranet Trends" report. The end of the year is punctuated by the Online Information conference in London.

For everyone, the past year has been dominated by depressing statistics. I thought I was inured to these negative numbers until I read the section in the "Global Intranet Trends" report on finding information. You might like to turn to page 40 in your copy (you do have one, don't you?) and follow along with me. First, only 14% of respondents were very satisfied with their search application. Certainly 47% were moderately satisfied, but this shouldn't be an acceptable benchmark. Only 6% were very satisfied with the content that was returned from a search. As McConnell notes, despite improvements made over the past few years, about 40% of respondents are "not really satisfied" or "not satisfied at all" with their search application.

Why are the results so poor? The answer lies on page 41: Only 12% of respondents have a search strategy for implementation and evolution, and in only 6% of cases are business owners involved in defining requirements. Users are consulted about search requirements by only 10% of the respondents, and just 13% conduct usability tests. Those figures are bad enough, but it gets worse. Seventy percent of organizations have less than one full-time employee working on search support, 30% do not look at search logs at all, and a further 35% do so only when resources allow.

Moreover, poor search results seem to be blamed on technology, as a third of all respondents had implemented or were in the process of implementing a new search application in the past 12 months. This is good news for search vendors but not for employees in affected organizations. The reason I say this is that 20% of those surveyed felt that incomplete information was holding the intranet back, a figure very similar to that in the recent "IBM Global CIO" survey.

The sample size of the survey is too large to ignore, and the results are supported by several presentations I've recently heard. It seems that SharePoint installations suffer from the lack of functionality of the 2007 search application, partially because Microsoft's channel partners do not always fully understand search implementation and maintenance. This situation is likely to get worse before it gets better-the leap in search functionality in SharePoint 2010 is going to need a lot of careful consideration as well as other major (and potentially more important) changes to other components of the new product.

Last year, Stephen Arnold and I wrote Successful Enterprise Search Management (Galatea), and, to be honest, we were surprised by the lack of book sales. If it was because everyone knew how to manage search, we might have understood. But the reality is that the level of understanding-in particular, search team resourcing-is totally inadequate. I heard a paper read at the J. Boye conference in Aarhus that revealed that one of the most prestigious pharmaceutical companies in the world is satisfied with about three-tenths of an employee to support its global intranet search application (SharePoint) because making the case for anything else was clearly not an option.

For McConnell's purposes, search is just one component of the intranet picture; there are limits to the depth that her survey can go in looking in more detail at the current situation. Before trying to fix the problems, we need to know what the problems really are. Miles Kehoe (of New Idea Engineering) and I are in the final stages of developing a more-comprehensive survey of the search market from a user perspective. If you would be interested in taking part, please contact me. However, I fear that, like the economy, there is no quick fix. I must stop-I need to go and cheer myself up. New York in May gives me something to look forward to ... and perhaps at May's Search Summit, Kehoe and I will have some useful information to move us along the road to search recovery.