How Big is the Smile?


I enjoy giving presentations at conferences. It enables me to promote my company, saves me the cost of the delegate fee, takes me to some very pleasant locations, and enables me to gather ideas on intranet deployment that I can then use in my consulting work. Then, a few weeks later, an envelope arrives with the results of the delegate satisfaction survey! Sometimes this has a significant impact on my ego, and on other occasions I find myself scrabbling to account for the fact that what I regarded as a very boring and lack-lustre presentation received a better mark than I expected. We all like to be able to benchmark our performance, whether in business or on the golf course. Companies wait anxiously for industry surveys, such as those in Fortune magazine, which rate companies on the extent to which they are good employers.

In the intranet world, there seems to be quite an interest in benchmarking intranets. Equally popular is the subject of determining the ROI of an intranet in order to justify the deployment. In the early days of intranets, as long ago as 1999(!), ROI studies were all the rage, with one study suggesting that an ROI of 1500% could be obtained from an intranet. I have to admit that I am very skeptical of ROI calculations and the theory that intranet applications have to be justified on an ROI basis. Let me explain why.

How Good is Your Crystal Ball?
Using ROI as a metric to justify either deploying or enhancing an intranet is fundamentally flawed, because it requires a business to forecast the future and have a very precise measure of the value of staff. Forecasting the future was difficult enough under the relatively steady conditions in the pre-Internet era. Now it is virtually impossible. Additionally, an intranet changes not only the minutiae of daily business, such as booking a meeting room, it transforms the way in which staff throughout the business work with each other, with customers, and with suppliers. Other benefits will include a reduction in staff turnover, significantly reduced induction periods for new joiners, and the creation of innovative and competitive products and services. The real costs and benefits of an intranet are all staff-related-the hardware and software are getting less expensive all the time. Good staff are getting more costly to find and keep, but these costs are rarely found in the corporate accounts and annual reports.

Instead, ROI calculations tend to focus just on productivity gains or on cost reductions, because they can be measured in numeric terms. One of the favorites is the way in which expense claims and vacation forms are handled. In this case, the intranet may have decreased these costs by 50%. Excellent news if you are in an organization that does nothing else but process expense forms. What really happens (assuming an inefficient design) is costs are merely transferred. Now it takes a senior consultant twice as long to complete his or her form on a poorly designed forms package, with no built-in authority checks as it had in the old days on an Excel spreadsheet.

The problems are even more acute for organizations in the public and nonprofit sector. Here, there are often no revenues that can be set against investment, and yet for this same reason the requirements to justify even a minimal investment in an intranet are more evident. The problems of justifying intranet investment in nonprofit organizations was the topic of a very lively breakfast forum table at Intranets 2001. The general feeling was that if your management wanted you to produce an ROI justification for an intranet, it was advisable to find another position sooner rather than later!

One of the best current commentaries on ROI justifications has been published by Plumtree ( "A Framework for Assessing Return on Investment for a Corporate Portal Deployment" can be downloaded from the Web site. The detail in this document is admirable, but I especially like a comment that appears right at the end of the document:

"Regardless of how thoroughly researched and elaborate your calculations for a corporate portal may be, the art of business is making objective decisions without perfect knowledge in an uncertain business environment. This requires the assessment of qualitative factors beyond the ken of accountants."

I could not have said it any better. Any management team that is seeking to justify an intranet, or a corporate portal, based to a significant extent on making ROI calculations, is not living in the knowledge millennium.

Benchmarking, But Against What?
Ironically, once an intranet has been set up without any ROI quantification, there is then even more interest on the part of managers to quantify the success of the intranet by benchmarking it against other intranets. As I remarked in an earlier column, intranet managers are very lonely people. Very rarely are they able to do more than gaze briefly at any live intranet, and often have to make do with unreadable screen shots carefully selected by conference presenters to conceal disaster areas.

Frankly, any one-off benchmarking exercise is always going to be of very limited value. A business is always looking to optimize its competitive position by developing new products and services, and the intranet will need to move to support these initiatives.

Asking the Right Questions
I am not suggesting to blindly install an intranet just as an act of faith. There needs to be a clear business justification, but in terms of output, not in terms of the number of hits on the pages or the number of documents added to the system per month. One company I worked with recently was overjoyed that as a result of an intranet redesign exercise, the number of hits on the server had gone up by 300%. The reality was that it had made the top-level structure more complex, and virtually all the additional hits were a result of staff having to work through the extra layer of navigation and some additional indexes. Once users had become smarter at locating information, the hit levels decreased substantially, but user satisfaction with the intranet increased markedly!

The solution lies in asking employees, customers, and suppliers questions that provide information on the health and wealth of the business, and on the impact that the intranet has had on these relationships.

The impact on customers should be especially marked. An intranet-enabled company creates the impression that the entire resources of the company are available to any employee the customer is in contact with, and that the information provided to the customer is reliable and relevant. Many companies are investing in Web survey applications that enable very focused and easily interpreted questions to be asked of customers and suppliers, as well as employees. These applications are now being built into corporate portal applications, such as the OpinionWare application from divine Inc.

As with all survey data, it is more important to look for patterns and exceptions in use rather than one-off measures. An increase in customer satisfaction of 25% in Division A is good news, but why hasn't any other division managed more than 5%? This is especially important in multinational companies, where the metrics for intranet success have been devised at HQ without any conscious research into the extent to which these metrics are either measurable or relevant in other countries.

Information-sharing and intranet-use issues also need to be on the table at employee evaluations, which also means that you will need to ensure that not only are intranet responsibilities set out in the job specification, but also the extent to which the satisfactory performance of that role depends on the intranet.

Measuring intranet success is not easy, and needs careful attention to both the strategy and the detail. Above all else, use the information to enhance the intranet to meet emerging business and information requirements. Put away the calculator and get out the ruler to measure the smiles on staff and customers as they benefit from your intranet investment. In the final analysis, that is probably the best measure of all.