At Hitwise our clients are always discovering new trends from the vast amount of data we offer. From those findings we frequently get asked by clients to let them post their findings to our research blog. So I am happy to present our first guest blog post below from Dustin Woodard, a veteran SEO and long-time Hitwise client. You can see Dustin speaking at search conferences or read his writings on his SEO blog.
Considering the large number of references to the “long tail of search” over the past couple years, you’d expect a plethora of research about it. Truth is, there has been very little published on this topic. After great dissatisfaction with the existing research, which I felt vastly understated the true size of the long tail, I decided to do my own research armed with the powerful set of data Hitwise provides.
Estimating the Long Tail of Search
In September 2008, I opened the All Categories search report to discover Hitwise has measured more than 14 million different search terms in the U.S. alone for the previous 3-month period (note – Hitwise provides data going back three years). I exported the top 10,000 search terms to analyze the data. It came as no surprise that the top terms are generally navigational or behavioral terms with Myspace topping the list. However, what was surprising was the resulting chart below of the top 10,000 search terms:
Top 10,000 Search Terms by Percentage of All Search Traffic

Source: 1 Hitwise
Despite how it may appear, my Y-axis is scaled correctly and, though difficult to see, there is actually data displayed along the entire keyword plot. Keep in mind, this is just the top 10,000 search terms of more than 14 million. Zooming in even further to the top 100 search terms, I get this chart:
Top 100 Search Terms by Percentage of All Search Traffic

Source: 2 Hitwise
Now we are seeing something similar to past long tail search reports. However, this is just 100 search terms out of the more than 14 million. Even looking at the top 100, it appears that the tail starts around term #18, “bank of america.”
Assuming the tail doesn’t begin until term 18, the head and body together only account for 3.25% of all search traffic! In fact, the top terms don’t account for much traffic:
· Top 100 terms: 5.7% of the all search traffic
· Top 500 terms: 8.9% of the all search traffic
· Top 1,000 terms: 10.6% of the all search traffic
· Top 10,000 terms: 18.5% of the all search traffic
This means if you had a monopoly over the top 1,000 search terms across all search engines (which is impossible), you’d still be missing out on 89.4% of all search traffic. There’s so much traffic in the tail it is hard to even comprehend. To illustrate, if search were represented by a tiny lizard with a one-inch head, the tail of that lizard would stretch for 221 miles.
The truth is my research is still greatly understating the true size of the tail because:
· The Hitwise sample contains 10 million U.S. Internet users and a complete data set would uncover much larger portions of the long tail.
· The data set I used filtered out adult searches.
· I only looked at 3-months worth of data (which were some of the slower months for search engines).
In summary, the long tail aspect of the search is true, but the data tells us that there may really be no head or body. When it comes to search, virtually all traffic is long tail and the word “long” doesn’t do the length of the tail justice.