The Humble but Effective A/B Test

Personalization · By On · 3 Comments

Over the next few weeks, I’ll be using my blog posts to high­light the Test & Tar­get tools you have at your dis­posal, as well as call out the most effec­tive ways to use them.

I’ll start the series with the A/B test, one of the most fun­da­men­tal tools in your arse­nal. With mul­ti­vari­ate tests get­ting the lion’s share of atten­tion in the press, the effec­tive­ness of an A/B test is often over­looked. That’s too bad, since an A/B test is often the base­way to start testing.

First, let’s define some terms. For our pur­poses, I’ll define an A/B test as a live test in which you mea­sure the per­for­mance of two or more pages against each other. Exam­ples would be test­ing two ver­sions of your site’s home­page, or test­ing three ver­sions of a par­tic­u­lar land­ing page.

My OTTO Dig­i­tal clients are often sur­prised when I sug­gest start­ing with an A/B test. But once I explain my rea­son­ing, they quickly sign up with the plan. Here is why I rec­om­mend start­ing with an A/B test:

An A/B test encour­ages you to test rad­i­cal alternatives

Unlike mul­ti­vari­ate tests, which are designed to rig­or­ously test spe­cific ele­ments on a page, an A/B test encour­ages you to try totally new page designs. You should take advan­tage of this fact and test out ideas that dif­fer sig­nif­i­cantly from your con­trol. At OTTO Dig­i­tal we often tell clients that if you tack your tested alter­na­tives up on a wall, you should be able to tell them apart from across the room.

An A/B test can gives you fast results

With an A/B test, you divide traf­fic into just a few buck­ets. If you’ve cre­ated recipes with dra­matic dif­fer­ences, you’re likely to see a recipe or two emerge as a win­ner more quickly than you would with a more com­plex mul­ti­vari­ate test. For get­ting results fast, noth­ing beats an A/B test.

A/B tests are sim­ple to design and interpret

This is an impor­tant fac­tor to con­sider, espe­cially if site opti­miza­tion is a new dis­ci­pline at your com­pany. Few things set back a nascent test­ing effort more than an excit­ing test that is later revealed to be flawed in some way. An A/B test pro­vides a straight­for­ward setup, with straight­for­ward results.

An A/B test helps you focus your future MVT efforts

Once you run an A/B test and know which recipe is per­form­ing the best, you can fur­ther opti­mize that win­ner with a round of rig­or­ous MVT test­ing. You’ll now be test­ing off an improved base­line. A/B test­ing is unmatched in its abil­ity to reveal a win­ning con­cept and guide your future efforts.

So there you have it. Don’t hes­i­tate to run the hum­ble but effec­tive A/B test. It’s often the best first step on your road to site optimization.

  • http://www.marketo.com/b2b-marketing-resources/landing-page-test-calculator.php Jon Miller

    One way to tell whether A/B test­ing is right for you us to use an online cal­cu­la­tor to tell you how many test ver­sions a given page can sup­port and still give sig­nif­i­cant results in a timely fash­ion:
    http://​www​.mar​keto​.com/​b​2​b​-​m​a​r​k​e​t​i​n​g​-​r​e​s​o​u​r​c​e​s​/​l​a​n​d​i​n​g​-​p​a​g​e​-​t​e​s​t​-​c​a​l​c​u​l​a​t​o​r​.​php

  • http://www.leadliaison.com rev­enue gen­er­a­tion for b2b

    A/B test­ing is great for land­ing pages or pages that con­tain web forms. I’d also sug­gest using A/B test­ing for email cam­paigns. Most professional-grade email mar­ket­ing tools pro­vide A/B test­ing capa­bil­i­ties with click through rates and ana­lyt­ics to help you quickly mea­sure effec­tive­ness of these real-life tests.

  • http://www.kompatible-toner-kartuschen.de Toner online

    This is an impor­tant fac­tor to con­sider, espe­cially if site opti­miza­tion is a new dis­ci­pline at your com­pany. Few things set back a nascent test­ing effort more than an excit­ing test that is later revealed to be flawed in some way. An A/B test pro­vides a straight­for­ward setup, with straight­for­ward results.