I’ll say it up front: bear with me here. An extended analogy is on its way.
If you happen to be a baseball fan—or maybe even if you’re not—take a look at this article, published last week by ESPN.com columnist Bill Simmons. In it, he finally embraces the new statistical measurements that stat geeks have been heralding for years. Like Simmons, my father and I grew up on batting average, home runs, runs batted in, and other, accessible, easily measurable, but inherently flawed (and sometimes extremely misleading) statistics. Life was grand. You knew which players to adore—those with high batting averages or RBI totals—and which to hate. So what happened?
Well, Moneyball happened. Fans and management started to realize that high batting averages or tons of home runs on the back of a player’s baseball card didn’t necessarily translate into the team’s end goal: winning. Ultimately, new ways to analyze the game emerged. Some embraced it; others are still coming to terms with these ideas. But everyone pretty much agrees that fielding a team based on statistics like RBI totals or Earned Run Average (for a pitcher) alone doesn’t make sense.
J.D. Drew is something of an enigma—whether he’s average or stellar depends on the metric you choose
Perhaps the perfect example of this shift in mentality: David Jonathan Drew is an outfielder for my beloved Boston Red Sox. Baseball fans know him as J.D. Drew (which is strange, since his initials are obviously D.J.D.; turns out the J.D. actually refers to his middle name and his last name,—Jonathan Drew=J.D.—and not to his first name and his middle name as is common). Anyway, last night I was watching Drew nonchalantly strike out against the Yankees’ C.C. Sabathia in the second inning when I remembered this quote from Simmons’ piece:
I longed for the old days when you could say things like, “I hate watching J.D. Drew—when is that contract going to end?” and there wasn’t some dude lurking behind me with Drew’s stellar OPS, VORP and WAR numbers saying, “Well, actually … ”
And it occurred to me that this shift in thinking is exactly like the one that needs to take place within online marketing organizations who are using SiteCatalyst, Discover, and other advanced web analytics tools to optimize their campaigns and user experience to drive conversion. Page Views might make sense as a metric in some limited contexts, but if you’re judging the success of your campaigns by how many page views they generate, that’s the wrong metric most of the time. Is the Visits metric useful in understanding general traffic trends? Absolutely—there is a reason we offer it. But do you want to optimize based on Visits? Probably not.
I’m not the first person to say this, so I don’t claim credit for a wholly unique outlook on the matter. That said, it’s critically important and warrants mentioning here and probably on every blog where online marketers—from newbies to gurus—look for info. So I’m saying it: You need to optimize around something that actually works and that conveys the real value of your efforts. (NOTE: This is a big reason why we presented on participation at Summit.)
The whole premise is exactly like J.D. Drew. According to the raw numbers, he’s a just-slightly-above-average ballplayer. Over his 11+ seasons in the majors, he’s a .283 hitter (that’s batting average), who averages 26 home runs and 85 RBI per full season of baseball. If those are your metrics—again, these are the metrics that are relatively easy to calculate and to understand for even casual fans (in the analogy, these are your HiPPOs)—Drew is definitely NOT worth the five-year, $70 million contract that he signed prior to the 2007 season. If he were a campaign optimized on Page Views, you’d throw him away like a rotten egg.
But, as Red Sox general manager Theo Epstein knew when he signed Drew, and as Bill Simmons learned, those are the wrong metrics. They don’t take into account the many varied situations and factors that contribute to success in baseball, and instead place all of their eggs in a few important-but-limited baskets. If you happen to hit a lot of home runs, then you’ll excel if that this the metric people are using. But that isn’t the best way to gauge a player’s value, just as increased traffic might suggest a successful campaign, but only if the goal of the campaign was to generate traffic. ROI (or at least ROAS) is probably a better measure of true value.
Red Sox GM Theo Epstein is changing the way Boston fans think about players. Who is changing the way your organization thinks about online marketing?
I won’t try to explain the “new” metrics of baseball—I don’t understand all of the math myself, frankly—but suffice it to say that when you look at Drew’s OPS+, he is the 24th best active player in baseball. His WPA (which Simmons doesn’t cover, but which attempts to quantify how much more likely your team is to win games with the given player in your lineup) puts him 22nd. And most impressively, his career WAR of 44.2 (i.e., his presence has theoretically netted his teams 44.2 more wins than a perfectly average alternative player) puts him 229th (out of nearly 17,000 players) in the entire history of baseball, ahead of a large number of current and future members of the Hall of Fame. So, yeah, he’s pretty good. And if Drew were an online campaign, you would invest more money in him. If he were a web page, you would devote more time and effort to drive people to him. If he were a segment of users, you would target him more aggressively—even if his Page Views and Visits weren’t as high as those of, say, Carlos Beltran.
What’s my point? I almost broke out in spontaneous applause during Josh James’ keynote address at Summit last month when he talked about online marketers leading the “next digital decade” with metrics that emphatically justify their activities. Don’t let yourself get sucked into believing that certain methods are working better than others because the “old metrics” tell a different story than proven, powerful metrics that your web analytics solution makes available to you with a bit of forethought and a solid implementation. There’s so much more out there available to you.
As Simmons put it:
You can’t write about baseball in 2010 (or play serious fantasy or gamble or have an educated conversation) without embracing sabermetrics. Fight it, and you’re just being stubborn.
You can’t be in online marketing in 2010 without embracing metrics that tell the real story—the valuable story. Fight it, and you’re just being stubborn.
If you’re interested in the metrics revolution taking place within the world of American professional sports, take a look at the following extra-credit reading:
And yes, I did just get to spend my afternoon at the office writing about baseball. I love my job.
As always, I welcome any questions, concerns, comments, etc. that you might have about any of these posts (or about anything else related to the Omniture Online Marketing Suite. Please feel free to comment on this or any other blog post, or to contact me via Twitter (@OmnitureCare) and I’ll do my best to get you the information that you need.
zulf
hi Ben
like your article, thought any chance you can provide further info on what ” OPS, VORP and WAR ” stands for.
thanks
http://blogs.omniture.com/author/bgaines Ben Gaines
Most of them are explained in the ESPN.com article that I mentioned at the beginning of the post, but they stand for:
OPS = on-base percentage plus slugging percentage
VORP = value over replacement player
WAR = wins over replacement
I hope that helps!
Tony John
So true Ben. It is the same in my HR consulting field when it comes to hiring. There are much, much better ways to measure how good an employee will be compared to the standard job interview. But people like hiring people that they like, that they feel in “their gut” will be good. Somehow peoples’ distrust of newer metrics and ways to measure performance inflate peoples’ love of the old.
This blog was so good, I think I just might send you $40…
http://blogs.omniture.com/author/bgaines Ben Gaines
Ha! Your $40 payback from an old friend would be much appreciated, Tony. Interesting note about HR. I had never considered the application of the sabermetric movement in other fields, but it makes sense; there are just so much more data these days available to people in nearly all fields. Ironically, Theo’s approach hasn’t been working so well for our Sox over the first few weeks of the 2010 season.
Embracing the new metrics: Web analytics, baseball, and you
I’ll say it up front: bear with me here. An extended analogy is on its way.
If you happen to be a baseball fan—or maybe even if you’re not—take a look at this article, published last week by ESPN.com columnist Bill Simmons. In it, he finally embraces the new statistical measurements that stat geeks have been heralding for years. Like Simmons, my father and I grew up on batting average, home runs, runs batted in, and other, accessible, easily measurable, but inherently flawed (and sometimes extremely misleading) statistics. Life was grand. You knew which players to adore—those with high batting averages or RBI totals—and which to hate. So what happened?
Well, Moneyball happened. Fans and management started to realize that high batting averages or tons of home runs on the back of a player’s baseball card didn’t necessarily translate into the team’s end goal: winning. Ultimately, new ways to analyze the game emerged. Some embraced it; others are still coming to terms with these ideas. But everyone pretty much agrees that fielding a team based on statistics like RBI totals or Earned Run Average (for a pitcher) alone doesn’t make sense.
Perhaps the perfect example of this shift in mentality: David Jonathan Drew is an outfielder for my beloved Boston Red Sox. Baseball fans know him as J.D. Drew (which is strange, since his initials are obviously D.J.D.; turns out the J.D. actually refers to his middle name and his last name,—Jonathan Drew=J.D.—and not to his first name and his middle name as is common). Anyway, last night I was watching Drew nonchalantly strike out against the Yankees’ C.C. Sabathia in the second inning when I remembered this quote from Simmons’ piece:
And it occurred to me that this shift in thinking is exactly like the one that needs to take place within online marketing organizations who are using SiteCatalyst, Discover, and other advanced web analytics tools to optimize their campaigns and user experience to drive conversion. Page Views might make sense as a metric in some limited contexts, but if you’re judging the success of your campaigns by how many page views they generate, that’s the wrong metric most of the time. Is the Visits metric useful in understanding general traffic trends? Absolutely—there is a reason we offer it. But do you want to optimize based on Visits? Probably not.
I’m not the first person to say this, so I don’t claim credit for a wholly unique outlook on the matter. That said, it’s critically important and warrants mentioning here and probably on every blog where online marketers—from newbies to gurus—look for info. So I’m saying it: You need to optimize around something that actually works and that conveys the real value of your efforts. (NOTE: This is a big reason why we presented on participation at Summit.)
The whole premise is exactly like J.D. Drew. According to the raw numbers, he’s a just-slightly-above-average ballplayer. Over his 11+ seasons in the majors, he’s a .283 hitter (that’s batting average), who averages 26 home runs and 85 RBI per full season of baseball. If those are your metrics—again, these are the metrics that are relatively easy to calculate and to understand for even casual fans (in the analogy, these are your HiPPOs)—Drew is definitely NOT worth the five-year, $70 million contract that he signed prior to the 2007 season. If he were a campaign optimized on Page Views, you’d throw him away like a rotten egg.
But, as Red Sox general manager Theo Epstein knew when he signed Drew, and as Bill Simmons learned, those are the wrong metrics. They don’t take into account the many varied situations and factors that contribute to success in baseball, and instead place all of their eggs in a few important-but-limited baskets. If you happen to hit a lot of home runs, then you’ll excel if that this the metric people are using. But that isn’t the best way to gauge a player’s value, just as increased traffic might suggest a successful campaign, but only if the goal of the campaign was to generate traffic. ROI (or at least ROAS) is probably a better measure of true value.
I won’t try to explain the “new” metrics of baseball—I don’t understand all of the math myself, frankly—but suffice it to say that when you look at Drew’s OPS+, he is the 24th best active player in baseball. His WPA (which Simmons doesn’t cover, but which attempts to quantify how much more likely your team is to win games with the given player in your lineup) puts him 22nd. And most impressively, his career WAR of 44.2 (i.e., his presence has theoretically netted his teams 44.2 more wins than a perfectly average alternative player) puts him 229th (out of nearly 17,000 players) in the entire history of baseball, ahead of a large number of current and future members of the Hall of Fame. So, yeah, he’s pretty good. And if Drew were an online campaign, you would invest more money in him. If he were a web page, you would devote more time and effort to drive people to him. If he were a segment of users, you would target him more aggressively—even if his Page Views and Visits weren’t as high as those of, say, Carlos Beltran.
What’s my point? I almost broke out in spontaneous applause during Josh James’ keynote address at Summit last month when he talked about online marketers leading the “next digital decade” with metrics that emphatically justify their activities. Don’t let yourself get sucked into believing that certain methods are working better than others because the “old metrics” tell a different story than proven, powerful metrics that your web analytics solution makes available to you with a bit of forethought and a solid implementation. There’s so much more out there available to you.
As Simmons put it:
You can’t be in online marketing in 2010 without embracing metrics that tell the real story—the valuable story. Fight it, and you’re just being stubborn.
If you’re interested in the metrics revolution taking place within the world of American professional sports, take a look at the following extra-credit reading:
And yes, I did just get to spend my afternoon at the office writing about baseball. I love my job.
As always, I welcome any questions, concerns, comments, etc. that you might have about any of these posts (or about anything else related to the Omniture Online Marketing Suite. Please feel free to comment on this or any other blog post, or to contact me via Twitter (@OmnitureCare) and I’ll do my best to get you the information that you need.