Measuring Visitor Engagement: Brave New World or the Emperor’s New Clothes?
Hello, everyone. Here’s a nice meaty post for you to sink your teeth into. And, it will be followed by several more, just as meaty, because I’ve decided to focus on one of the most controversial topics in the industry right now: audience measurement.
The credibility of audience measurement data from firms like comScore, Hitwise and Nielsen has been plagued for years, for a variety of reasons, including fundamental limitations with their data collection methodologies, unpredictable revisions to those methodologies and results from different firms that contradict each other.
Recently, some firms have been claiming that they have hit on the best and most effective metric for online advertising: time spent on site.
And I believe they’re dead wrong.
I believe the best way to measure online advertising is by the very metric we most often use to sell it: by impression. And, while there are a few potential problems in using this as a metric, the problems are not insurmountable.
An “impression” is counted any time an ad is “served” to an end-user, and it is the metric advertisers most often use when they purchase ad space (the commonly used “CPM” means cost per thousand impressions). Yet the metrics that measurement data firms have used in the past have little to do with impressions. Rather, measurement firms and the online advertising world have experimented with a number of very different measurement options.
Beginning in the late 1990s, we focused on unique visitors – largely because this was believed to be unique people. As I’ve discussed in the past, unique visitors are anything but unique people and this metric is fundamentally flawed. As we grew more sophisticated, we began to measure how visitors interacted with content on the site. Page views emerged as the de facto standard for measuring content consumption, and for a long time, that metric reigned supreme.
But now Web 2.0 technologies are completely disrupting the status quo and the page view is no longer as accurate as it once was. With Flash, AJAX and streaming media technologies becoming more prevalent, the page view increasingly fails to capture the complete user experience. That’s because these technologies deliver a richer, more dynamic experience that does not require a new page view with each new piece of content.
The Gap.com website provides a great example. If you navigate to the Jeans category, you’ll quickly notice a Flash-based module in the center of the page that shows different jeans on actual people. It’s pretty slick and you’ll notice as you click on the different types of jeans, a new person will walk out with those jeans on. But you’ll notice something else. Each time you click, the full page does not change. It remains the same, even though the user is able to interact dynamically with fresh content.
Thus, whereas in the past a visitor would have gone to a second — or third, or fifth — page in order to get the information about each subsequent style of jean, she can now remain on a single page. So in this example, is the single “page view” that the visitor saw really the best measurement of that visit, when she actually browsed through several products? Definitely not.
With this in mind, it makes sense that audience measurement firms are exploring new ways to quantify visitor engagement on the web.
What makes less sense is that they have chosen time-spent-on-site as the best measure of audience engagement. The move has attracted a good deal of press, some of which has gone so far as to suggest this is a ground-breaking step.
In coming blog posts, I will go into detail about page views, time spent on site and impressions. I’ll explain why time spent on site is not, contrary to what much of the trade press claims, the best available measurement. I will show that, while those metrics are interesting proxies of ad inventory, they are secondary metrics when it comes to the true nuts and bolts of ad buying, and they do little or nothing to help advertisers and media publishers connect with their target audience.
And I’ll prove that measurement by impression — the same metric that the majority of web publishers use to sell online ads — is not only desirable, but is eminently attainable.
To me, this decision to measure time spent on site is a case of the Emperor’s new clothes — the Emperor, and nearly everybody else, claims his clothes are the best ever, when in reality, he’s wearing nothing more than air.