Looking for a cost-effective way to get new customers for your content? Look no further than public search engines like Google, Yahoo!, and MSN. Americans conducted 6.4 billion searches online in June 2006, according to comScore Networks, and this doesn't include huge markets outside the United States, such as China, where search growth is explosive as well.
Even more compelling is comScore's finding that 86% of searches were in one of the three major search engines. A searcher can traverse consumer, business, and health content frontiers with ease and, whether at work or at home, the search behavior and the software browser remain the same. Any given searcher might be looking for government information, company information, product information, business statistics, or their local weather, or researching their favorite esoteric topic, all from the comfort of a familiar browser and favorite search engine.
The simple fact is that customers are increasingly finding companies and products by starting with one of the major search engines. By and large, they don't remember specific URLs; but even if they do, it's easier to key a phrase into the Google search field and bring up relevant results than it is to key a URL into an address bar. Additionally, web users find that the URLs of sites they frequent may not be intuitively obvious, so it's simpler to start with a search at a search engine.
So how can your organization become more visible on public search engines? The press focuses on the money spent for "search marketing," which involves bidding for keyword placement and creating ads. Search marketing is another term for paid advertisements on search results pages, which is in many ways similar to creating ads in the print world. There is an entire cottage industry devoted to this paid placement marketing, not to mention traditional advertising and marketing agencies.
However, reaching customers more effectively doesn't have to involve paid advertising. Instead, improving "findability" in "natural" or "organic" results is a tactic free of advertising expense. Content producers have an inherent advantage because they have more words to index than other types of sites, such as retailers; but any type of organization can maximize its content for findability.
It's worth it to try. The search industry rule is that 70% of click-throughs are from "organic" listings. According to Jody Nimitz of Enquiro, a search marketing company, "Recent data suggests that with Google this may even be higher, at about 85% organic and 15% sponsored." Click-through rates on individual pay-per-click ads are usually well under 5%, whereas users will generally click at least one of the top 10 results on the search results page.
Reaching customers by improving the relevance of search results can be straightforward—if you develop a basic understanding of how search engines present results from your web pages to a searcher. Let's explore the factors that impact the results shown on the results pages after the potential customer enters a query into the search box.
Search Engine Indexing
Visibility in search engines isn't automatic, even though your company and products may be well branded in the offline world. There can also be a major amount of content simply not included in the search engine indexes. According to GovExec.com's daily briefing, as much as 40% of content on agency sites is invisible to Google's web crawlers. This is public service information, mandated to be openly available.
Just because pages exist on your website doesn't mean they can be read by web crawlers and added to the indexes. Existence of the "invisible web" has been well documented. Anything that requires interaction with users has the effect of stopping indexing spiders. However, web designers and programmers can also inadvertently create indexing barriers by using Java script and images that don't have ASCII navigation links. AJAX is the latest software innovation to create indexing problems for website businesses.
Search engines are basically interfaces to enormous index files that point to individual pages on your site. This means that product pages need to be found by indexing crawlers. Crawlers blithely ignore all the HTML coding that makes the visual web pages attractive, since coding for color and font is not relevant to a search query. Only the ASCII text on a web page matters, though increasingly other types of text in PDFs and Office documents are also indexable, and thus searchable. The crawlers also busily rank the value of text words on web pages. Much like a newspaper, the title and first paragraphs are more important, with bold words weighted more heavily, and repeated words an indication of the importance of the word to the subject.
Algorithms for indexing vary by the search engine, which can lead to major variations in the number of pages indexed. I worked with one site that had 1.2 million pages indexed in Google, but only 3% of those pages were indexed by Yahoo! and 1% by MSN. As a result, 95% of web traffic came from Google. However, Google represents only 45% of searches. The site was essentially invisible to the other 55% of the search market.
The first test for findability is checking the number of pages actually indexed in each search engine. To test this, just put "site:[your URL]" into the Google search box to display the number of pages from your URL in the indexes. This number probably won't be the actual number of web pages for a variety of reasons, but the order of magnitude should be similar. If there are thousands of web pages on your website, but fewer than 50 pages indexed in Google, you have an indexing problem. Also check Yahoo!, MSN, and Ask.com—there may be big differences between these search engines that need to be addressed.