Since I wrote about benchmarking two weeks ago, I’ve received many follow up emails. Many of you said “Amen! Thanks for dispelling the myths around web analytics benchmarking.” However, some of you pushed back, saying that “conversion benchmarks are useful if you simply weigh them appropriately.” And truth be told, I do not disagree with the second point. Benchmarks can be useful if you weigh them appropriately – of course, when compared to site-centric data, I give benchmarks very little weight.

To illustrate why benchmarks are often misleading at best, take a look at the Fireclick Index (http://index.fireclick.com/). I’ve focused on this index because it’s publicly available and everyone can readily access it.

In looking at the index, you’ll first notice the Global Conversion rate is down 17%. Is that conversion for all websites, i.e. lead generation, media, etc? Or is that just retail? Based on the presence of shopping cart abandonment further down the page, I’d guess this index is only focused on retailers.

So let’s assume I’m a retailer and there’s some relevance here to my business. Global conversion is down 17%. What does that tell me? That a competitor screwed up their checkout process? That a competitor’s site is worse than mine? That the industry in general just wrapped up a major tradeshow? That someone’s homepage promotion failed? Should I invest millions into a land grab campaign to take advantage of this brief window of market opportunity? Let’s say my conversion went up 15%, but I didn’t do anything differently on my site. What does this tell me? What decision am I going to make? This data, while interesting, doesn’t support any strategic decision I’d want to make with confidence.

Moving down the list, I see conversion for keywords is 0.90%. Interesting, except I bid for keywords based on ROAS, not on conversion. If my keywords were converting at 10%, does that mean I should buy more because I’m higher than the index? Should I bid higher? No, of course not. Because any search marketer worth their salt knows a 10% conversion rate does not tell you how much profit these keywords are generating.

I’m still looking for something actionable – turning now to email. Interesting to see the 1.5% conversion. Are these acquisition campaigns? Are these third-party lists? HTML email, or just text? Are these auto-bounce backs? Do they include promotions with pricing discounts, or just brand emails? Do they also include newsletters? Well, without these details, it’s pretty useless comparing my email to this index.

Average Connection Speed dropped 1%. Hmm, what do I do about this? I can compare this to my site connection speed. My connection speed decreased 10%. Well, since I’m 9 percentage points below the average, should I start buying more server hardware? Increase the pipe? Outsource to Akamai? Again, this benchmark data simply doesn’t deliver action intelligence.

So that’s my take on benchmarks. Despite claims to the contrary, I have yet to see a persuasive argument for taking strategic action from benchmark data.

0 comments