Embracing the new metrics: Web analytics, baseball, and you

I’ll say it up front: bear with me here. An extended anal­ogy is on its way.

If you hap­pen to be a base­ball fan—or maybe even if you’re not—take a look at this arti­cle, pub­lished last week by ESPN​.com colum­nist Bill Sim­mons. In it, he finally embraces the new sta­tis­ti­cal mea­sure­ments that stat geeks have been herald­ing for years. Like Sim­mons, my father and I grew up on bat­ting aver­age, home runs, runs bat­ted in, and other, acces­si­ble, eas­ily mea­sur­able, but inher­ently flawed (and some­times extremely mis­lead­ing) sta­tis­tics. Life was grand. You knew which play­ers to adore—those with high bat­ting aver­ages or RBI totals—and which to hate. So what happened?

Well, Mon­ey­ball hap­pened. Fans and man­age­ment started to real­ize that high bat­ting aver­ages or tons of home runs on the back of a player’s base­ball card didn’t nec­es­sar­ily trans­late into the team’s end goal: win­ning. Ulti­mately, new ways to ana­lyze the game emerged. Some embraced it; oth­ers are still com­ing to terms with these ideas. But every­one pretty much agrees that field­ing a team based on sta­tis­tics like RBI totals or Earned Run Aver­age (for a pitcher) alone doesn’t make sense.

J.D. Drew is some­thing of an enigma—whether he’s aver­age or stel­lar depends on the met­ric you choose

Per­haps the per­fect exam­ple of this shift in men­tal­ity: David Jonathan Drew is an out­fielder for my beloved Boston Red Sox. Base­ball fans know him as J.D. Drew (which is strange, since his ini­tials are obvi­ously D.J.D.; turns out the J.D. actu­ally refers to his mid­dle name and his last name,—Jonathan Drew=J.D.—and not to his first name and his mid­dle name as is com­mon). Any­way, last night I was watch­ing Drew non­cha­lantly strike out against the Yan­kees’ C.C. Sabathia in the sec­ond inning when I remem­bered this quote from Sim­mons’ piece:

And it occurred to me that this shift in think­ing is exactly like the one that needs to take place within online mar­ket­ing orga­ni­za­tions who are using Site­Cat­a­lyst, Dis­cover, and other advanced web ana­lyt­ics tools to opti­mize their cam­paigns and user expe­ri­ence to drive con­ver­sion. Page Views might make sense as a met­ric in some lim­ited con­texts, but if you’re judg­ing the suc­cess of your cam­paigns by how many page views they gen­er­ate, that’s the wrong met­ric most of the time. Is the Vis­its met­ric use­ful in under­stand­ing gen­eral traf­fic trends? Absolutely—there is a rea­son we offer it. But do you want to opti­mize based on Vis­its? Prob­a­bly not.

I’m not the first per­son to say this, so I don’t claim credit for a wholly unique out­look on the mat­ter. That said, it’s crit­i­cally impor­tant and war­rants men­tion­ing here and prob­a­bly on every blog where online marketers—from new­bies to gurus—look for info. So I’m say­ing it: You need to opti­mize around some­thing that actu­ally works and that con­veys the real value of your efforts. (NOTE: This is a big rea­son why we pre­sented on par­tic­i­pa­tion at Summit.)

The whole premise is exactly like J.D. Drew. Accord­ing to the raw num­bers, he’s a just-slightly-above-average ballplayer. Over his 11+ sea­sons in the majors, he’s a .283 hit­ter (that’s bat­ting aver­age), who aver­ages 26 home runs and 85 RBI per full sea­son of base­ball. If those are your metrics—again, these are the met­rics that are rel­a­tively easy to cal­cu­late and to under­stand for even casual fans (in the anal­ogy, these are your HiPPOs)—Drew is def­i­nitely NOT worth the five-year, $70 mil­lion con­tract that he signed prior to the 2007 sea­son. If he were a cam­paign opti­mized on Page Views, you’d throw him away like a rot­ten egg.

But, as Red Sox gen­eral man­ager Theo Epstein knew when he signed Drew, and as Bill Sim­mons learned, those are the wrong met­rics. They don’t take into account the many var­ied sit­u­a­tions and fac­tors that con­tribute to suc­cess in base­ball, and instead place all of their eggs in a few important-but-limited bas­kets. If you hap­pen to hit a lot of home runs, then you’ll excel if that this the met­ric peo­ple are using. But that isn’t the best way to gauge a player’s value, just as increased traf­fic might sug­gest a suc­cess­ful cam­paign, but only if the goal of the cam­paign was to gen­er­ate traf­fic. ROI (or at least ROAS) is prob­a­bly a bet­ter mea­sure of true value.

Red Sox GM Theo Epstein is chang­ing the way Boston fans think about play­ers. Who is chang­ing the way your orga­ni­za­tion thinks about online marketing?

I won’t try to explain the “new” met­rics of baseball—I don’t under­stand all of the math myself, frankly—but suf­fice it to say that when you look at Drew’s OPS+, he is the 24^th best active player in base­ball. His WPA (which Sim­mons doesn’t cover, but which attempts to quan­tify how much more likely your team is to win games with the given player in your lineup) puts him 22^nd. And most impres­sively, his career WAR of 44.2 (i.e., his pres­ence has the­o­ret­i­cally net­ted his teams 44.2 more wins than a per­fectly aver­age alter­na­tive player) puts him 229^th (out of nearly 17,000 play­ers) in the entire his­tory of base­ball, ahead of a large num­ber of cur­rent and future mem­bers of the Hall of Fame. So, yeah, he’s pretty good. And if Drew were an online cam­paign, you would invest more money in him. If he were a web page, you would devote more time and effort to drive peo­ple to him. If he were a seg­ment of users, you would tar­get him more aggressively—even if his Page Views and Vis­its weren’t as high as those of, say, Car­los Beltran.

What’s my point? I almost broke out in spon­ta­neous applause dur­ing Josh James’ keynote address at Sum­mit last month when he talked about online mar­keters lead­ing the “next dig­i­tal decade” with met­rics that emphat­i­cally jus­tify their activ­i­ties. Don’t let your­self get sucked into believ­ing that cer­tain meth­ods are work­ing bet­ter than oth­ers because the “old met­rics” tell a dif­fer­ent story than proven, pow­er­ful met­rics that your web ana­lyt­ics solu­tion makes avail­able to you with a bit of fore­thought and a solid imple­men­ta­tion. There’s so much more out there avail­able to you.

As Sim­mons put it:

You can’t be in online mar­ket­ing in 2010 with­out embrac­ing met­rics that tell the real story—the valu­able story. Fight it, and you’re just being stubborn.

If you’re inter­ested in the met­rics rev­o­lu­tion tak­ing place within the world of Amer­i­can pro­fes­sional sports, take a look at the fol­low­ing extra-credit reading:

• Mon­ey­ball (pur­chase from Ama​zon​.com)
• Money player (Boston Globe)
• Saber­met­rics Library (Saber​li​brary​.com)
• The No-Stats All-Star (New York Times)
• Houston’s G.M. Is a Rev­o­lu­tion­ary Spirit in a Risk-Averse Mind (New York Times)

And yes, I did just get to spend my after­noon at the office writ­ing about base­ball. I love my job.

As always, I wel­come any ques­tions, con­cerns, com­ments, etc. that you might have about any of these posts (or about any­thing else related to the Omni­ture Online Mar­ket­ing Suite. Please feel free to com­ment on this or any other blog post, or to con­tact me via Twit­ter (@OmnitureCare) and I’ll do my best to get you the infor­ma­tion that you need.