Metrics Manifesto: Raising the Standard for Metrics
In my last blog post, I discussed how people are still confusing metrics with other data types such as dimensions and reports. It’s important to be able to discern between what is and isn’t a metric, but that’s not enough. There are some other things we can do to improve how we use metrics in web analytics.
A big problem with some metrics is that they are not specific, clear/intuitive, or actionable. We say “garbage in, garbage out” when it comes to data collection and reporting. Similarly, fuzzy metrics lead to weak adoption, poor decision-making, and frequently inaction. If your metrics are nebulous or unclear, you’re doing a disservice to yourself and your company. I’d like to propose some ways in which you can improve your metric usage and avoid metric abuse.
Be specific and avoid vague metrics
One of my least favorite metrics is traffic. When you hear someone say they want to increase traffic to their site, what do they really mean? Traffic could mean page views, visits, daily unique visitors, monthly unique visitors, etc. We should have eradicated referring to traffic as a metric a long time ago, but it still regularly comes up in articles, white papers, and conversations to this day.
Another vague metric example is engagement. As a metric, what is engagement? There has been much debate in the web analytics community about engagement (Neil Mason provides a good summary), and whether it is actually a metric or not. I believe the problem with referring to traffic and engagement as metrics is that they aren’t tied to a single, standardized metric — they are really an area of analysis where several metrics or approaches could apply. You could develop a custom engagement index or track a combination of different metrics to measure customer/visitor engagement, but in and of itself “engagement” isn’t a metric.
I cringe each time I hear the word “metric” associated with new (and ambiguous) social media terminology such as buzz or influence. Are we doomed to repeat the same mistakes with the latest social media metric “du jour”? A guaranteed way to make a metric less actionable is to make it vague. We need to ensure our metrics are specific.
Properly define new metrics
All the cool kids are doing it. Lots of new, exciting-sounding metrics are being introduced all the time, especially in the social media space — e.g., Buzz Velocity, Brand Amplification, Influencer Impact, etc. (these ones might sound familiar, but they’re 100% fabricated). Unfortunately, many of these buzzword metrics end up being generally meaningless to most people and companies. Why? Too much effort is spent on hyping the metrics and reporting, and not enough time is spent on adequately defining the new metrics and explaining how they can be useful.
As marketers, we’re trained to differentiate something and build excitement for it. In the case of metrics, it shouldn’t be about differentiation and hype — but instead standardization, clarity, and utility. We don’t need more jargon that is pretentious, convoluted, or vague — especially not in the realm of social media. It doesn’t move us forward, it sets us back. Rather than creating more buzzword metrics, I’d prefer more descriptive names and increased emphasis on better defining and documenting new metrics for end users. In many cases, a fancy name hides the fact that the metric is just a re-packaging of a commonly-used metric or actually a report and not a metric at all.
Be careful with acronyms
We have lots of popular abbreviated metrics in online marketing and web analytics: AOV/AOS, CPC, CPA, CTR, ROAS, etc. Having worked in the high tech industry for more than a decade, I know how susceptible high tech firms are to using acronyms. Don’t follow our industry’s bad example! You need to be careful when using acronyms, especially when it comes to web metrics. Overzealous and premature usage of acronyms can impede metric comprehension.
At a recent conference I attended, the presenters used the metric “PVV” on a few different slides. Surprisingly, there was a moment (maybe 20–30 seconds) where I struggled to identify a metric I had used hundreds of times before — Page Views per Visit. When it dawned on me what they were talking about, it made me wonder what people outside of the web analytics team would think of this abbreviated metric. Would they automatically know what this acronym stood for or would they pretend they knew but not truly understand what was being discussed? Abbreviations are only acceptable if they are widely used and understood.
Focus on meaningful, actionable metrics
Just because you can track everything in SiteCatalyst or Insight doesn’t mean you should. It’s great when people get excited about web measurement and analytics; however, this exuberance can quickly become counterproductive if your company decides to measure everything under the sun. One consultant on my team recently labeled this phenomenon as going “Cuckoo for Cocoa Puffs”, and it often leads a tall stack of inconsequential, underutilized metrics and reports.
I’m not the first to say less is more when it comes to metrics, but it’s definitely worth repeating. Ultimately, you want metrics that are meaningful to your business (i.e., tied to your key business goals such as increasing revenue or reducing call center volume) and actionable (i.e., your company can take specific actions to move the metric up or down such as increasing ad spend for a particular campaign or optimizing a specific landing page).
The most meaningful and relevant metrics to your business should be your KPIs, which are a special subset of metrics used to measure performance against key business goals. I get concerned when I see the “KPI” label being applied too loosely or carelessly to ad hoc metrics. You don’t want to dilute the true power of real KPIs by crowding them out with false ones. Some supporting metrics can still be meaningful even though they aren’t KPIs because they provide useful insights into how your KPIs are being impacted.
In terms of actionability, you need to be careful with overly complicated formulas or what Matt Belkin referred to as metric mashups or uber metrics. These complex, calculated metrics are prone to being less actionable because an analyst may need to dissect the entire formula to discover what aspect(s) drove a large increase or decrease before any action can be taken. In most cases, you need the organization, not just a statistics PhD on the analytics team, to understand and grasp the metric for action to happen. Just remember less is more — in terms of both quantity and complexity — and be judicious with your unique set of KPIs and metrics.
One way to follow the “less is more” principle is to watch out for “nice-to-know” metrics. As consultants, we can typically ferret out these metrics when we ask what actions a particular client is prepared to take if the metrics come in really high or low. If they aren’t prepared to take any action then we’ve uncovered a potential “nice-to-know” metric. It can be frustrating for a web analytics manager to learn that one of the marketing team’s “must-have” metrics, which ended being the most time-consuming and difficult element to implement, results in only “hmmm…that’s interesting” and nothing else. As Pablo Picasso stated, “Action is the foundational key to all success.”
As I said my previous post — enough is enough. We need to stop the metric abuse today! If we’re going to change the face of marketing, we need to ensure our analytics foundations are sound. I’m making the pledge to not tolerate metric abuse any longer, and solemnly swear to:
- Only refer to actual metrics as metrics
- Be specific and never use vague metrics
- Never give fancy, meaningless names to metrics
- Avoid renaming well-known metrics and go with standards if they are in place whenever possible
- Ensure new metrics are well-defined and documented for anyone that will be consuming the reports/analysis
- Only use acronyms that are well-established and understood
- Make sure metrics are meaningful and actionable
- Be careful with complex, calculated metrics that may be less actionable
- Avoid “nice-to-know” metrics that waste time and mostly go unused
- Strive to correct people that misuse the terms: metric and KPI
Where do you see metric abuse? What points would you add to my metrics manifesto? The web analytics industry has come a long way, and I feel as though perpetuating simple oversights in areas such as how we use metrics is going to hold us back. Let’s stamp out metric abuse and move towards an even brighter, data-driven future.