I’ll say it up front: bear with me here. An extended anal­ogy is on its way.

If you hap­pen to be a base­ball fan—or maybe even if you’re not—take a look at this arti­cle, pub­lished last week by ESPN​.com colum­nist Bill Sim­mons. In it, he finally embraces the new sta­tis­ti­cal mea­sure­ments that stat geeks have been herald­ing for years. Like Sim­mons, my father and I grew up on bat­ting aver­age, home runs, runs bat­ted in, and other, acces­si­ble, eas­ily mea­sur­able, but inher­ently flawed (and some­times extremely mis­lead­ing) sta­tis­tics. Life was grand. You knew which play­ers to adore—those with high bat­ting aver­ages or RBI totals—and which to hate. So what happened?

Well, Mon­ey­ball hap­pened. Fans and man­age­ment started to real­ize that high bat­ting aver­ages or tons of home runs on the back of a player’s base­ball card didn’t nec­es­sar­ily trans­late into the team’s end goal: win­ning. Ulti­mately, new ways to ana­lyze the game emerged. Some embraced it; oth­ers are still com­ing to terms with these ideas. But every­one pretty much agrees that field­ing a team based on sta­tis­tics like RBI totals or Earned Run Aver­age (for a pitcher) alone doesn’t make sense.

J.D. Drew
J.D. Drew is some­thing of an enigma—whether he’s aver­age or stel­lar depends on the met­ric you choose

Per­haps the per­fect exam­ple of this shift in men­tal­ity: David Jonathan Drew is an out­fielder for my beloved Boston Red Sox. Base­ball fans know him as J.D. Drew (which is strange, since his ini­tials are obvi­ously D.J.D.; turns out the J.D. actu­ally refers to his mid­dle name and his last name,—Jonathan Drew=J.D.—and not to his first name and his mid­dle name as is com­mon). Any­way, last night I was watch­ing Drew non­cha­lantly strike out against the Yan­kees’ C.C. Sabathia in the sec­ond inning when I remem­bered this quote from Sim­mons’ piece:

I longed for the old days when you could say things like, “I hate watch­ing J.D. Drew—when is that con­tract going to end?” and there wasn’t some dude lurk­ing behind me with Drew’s stel­lar OPS, VORP and WAR num­bers say­ing, “Well, actually … ”

And it occurred to me that this shift in think­ing is exactly like the one that needs to take place within online mar­ket­ing orga­ni­za­tions who are using Site­Cat­a­lyst, Dis­cover, and other advanced web ana­lyt­ics tools to opti­mize their cam­paigns and user expe­ri­ence to drive con­ver­sion. Page Views might make sense as a met­ric in some lim­ited con­texts, but if you’re judg­ing the suc­cess of your cam­paigns by how many page views they gen­er­ate, that’s the wrong met­ric most of the time. Is the Vis­its met­ric use­ful in under­stand­ing gen­eral traf­fic trends? Absolutely—there is a rea­son we offer it. But do you want to opti­mize based on Vis­its? Prob­a­bly not.

I’m not the first per­son to say this, so I don’t claim credit for a wholly unique out­look on the mat­ter. That said, it’s crit­i­cally impor­tant and war­rants men­tion­ing here and prob­a­bly on every blog where online marketers—from new­bies to gurus—look for info. So I’m say­ing it: You need to opti­mize around some­thing that actu­ally works and that con­veys the real value of your efforts. (NOTE: This is a big rea­son why we pre­sented on par­tic­i­pa­tion at Summit.)

The whole premise is exactly like J.D. Drew. Accord­ing to the raw num­bers, he’s a just-slightly-above-average ballplayer. Over his 11+ sea­sons in the majors, he’s a .283 hit­ter (that’s bat­ting aver­age), who aver­ages 26 home runs and 85 RBI per full sea­son of base­ball. If those are your metrics—again, these are the met­rics that are rel­a­tively easy to cal­cu­late and to under­stand for even casual fans (in the anal­ogy, these are your HiPPOs)—Drew is def­i­nitely NOT worth the five-year, $70 mil­lion con­tract that he signed prior to the 2007 sea­son. If he were a cam­paign opti­mized on Page Views, you’d throw him away like a rot­ten egg.

But, as Red Sox gen­eral man­ager Theo Epstein knew when he signed Drew, and as Bill Sim­mons learned, those are the wrong met­rics. They don’t take into account the many var­ied sit­u­a­tions and fac­tors that con­tribute to suc­cess in base­ball, and instead place all of their eggs in a few important-but-limited bas­kets. If you hap­pen to hit a lot of home runs, then you’ll excel if that this the met­ric peo­ple are using. But that isn’t the best way to gauge a player’s value, just as increased traf­fic might sug­gest a suc­cess­ful cam­paign, but only if the goal of the cam­paign was to gen­er­ate traf­fic. ROI (or at least ROAS) is prob­a­bly a bet­ter mea­sure of true value.

Theo Epstein
Red Sox GM Theo Epstein is chang­ing the way Boston fans think about play­ers. Who is chang­ing the way your orga­ni­za­tion thinks about online marketing?

I won’t try to explain the “new” met­rics of baseball—I don’t under­stand all of the math myself, frankly—but suf­fice it to say that when you look at Drew’s OPS+, he is the 24th best active player in base­ball. His WPA (which Sim­mons doesn’t cover, but which attempts to quan­tify how much more likely your team is to win games with the given player in your lineup) puts him 22nd. And most impres­sively, his career WAR of 44.2 (i.e., his pres­ence has the­o­ret­i­cally net­ted his teams 44.2 more wins than a per­fectly aver­age alter­na­tive player) puts him 229th (out of nearly 17,000 play­ers) in the entire his­tory of base­ball, ahead of a large num­ber of cur­rent and future mem­bers of the Hall of Fame. So, yeah, he’s pretty good. And if Drew were an online cam­paign, you would invest more money in him. If he were a web page, you would devote more time and effort to drive peo­ple to him. If he were a seg­ment of users, you would tar­get him more aggressively—even if his Page Views and Vis­its weren’t as high as those of, say, Car­los Beltran.

What’s my point? I almost broke out in spon­ta­neous applause dur­ing Josh James’ keynote address at Sum­mit last month when he talked about online mar­keters lead­ing the “next dig­i­tal decade” with met­rics that emphat­i­cally jus­tify their activ­i­ties. Don’t let your­self get sucked into believ­ing that cer­tain meth­ods are work­ing bet­ter than oth­ers because the “old met­rics” tell a dif­fer­ent story than proven, pow­er­ful met­rics that your web ana­lyt­ics solu­tion makes avail­able to you with a bit of fore­thought and a solid imple­men­ta­tion. There’s so much more out there avail­able to you.

As Sim­mons put it:

You can’t write about base­ball in 2010 (or play seri­ous fan­tasy or gam­ble or have an edu­cated con­ver­sa­tion) with­out embrac­ing saber­met­rics. Fight it, and you’re just being stubborn.

You can’t be in online mar­ket­ing in 2010 with­out embrac­ing met­rics that tell the real story—the valu­able story. Fight it, and you’re just being stubborn.

If you’re inter­ested in the met­rics rev­o­lu­tion tak­ing place within the world of Amer­i­can pro­fes­sional sports, take a look at the fol­low­ing extra-credit reading:

And yes, I did just get to spend my after­noon at the office writ­ing about base­ball. I love my job.

As always, I wel­come any ques­tions, con­cerns, com­ments, etc. that you might have about any of these posts (or about any­thing else related to the Omni­ture Online Mar­ket­ing Suite. Please feel free to com­ment on this or any other blog post, or to con­tact me via Twit­ter (@OmnitureCare) and I’ll do my best to get you the infor­ma­tion that you need.

4 comments
Tony John
Tony John

So true Ben. It is the same in my HR consulting field when it comes to hiring. There are much, much better ways to measure how good an employee will be compared to the standard job interview. But people like hiring people that they like, that they feel in "their gut" will be good. Somehow peoples' distrust of newer metrics and ways to measure performance inflate peoples' love of the old. This blog was so good, I think I just might send you $40...

zulf
zulf

hi Ben like your article, thought any chance you can provide further info on what '' OPS, VORP and WAR '' stands for. thanks

Ben Gaines
Ben Gaines

Ha! Your $40 payback from an old friend would be much appreciated, Tony. Interesting note about HR. I had never considered the application of the sabermetric movement in other fields, but it makes sense; there are just so much more data these days available to people in nearly all fields. Ironically, Theo's approach hasn't been working so well for our Sox over the first few weeks of the 2010 season.

Ben Gaines
Ben Gaines

Most of them are explained in the ESPN.com article that I mentioned at the beginning of the post, but they stand for: OPS = on-base percentage plus slugging percentage VORP = value over replacement player WAR = wins over replacement I hope that helps!