Adobe
Adobe Digital Marketing Blog
  • Digital Marketing
    • Mobile
    • Social Media
    • Digital Advertising
    • Search Engine Marketing
  • Analytics
  • Personalization
  • Industries
    • Financial Services
    • Media & Entertainment
    • Retail & Travel
  • Executive Insights
    • Aseem Chandra
How to Make Testing Successful Newspaper Editors Become Online Marketers

The Humble but Effective A/B Test

Personalization · By John Broady On June 2, 2008 · 3 Comments

Over the next few weeks, I’ll be using my blog posts to high­light the Test & Tar­get tools you have at your dis­posal, as well as call out the most effec­tive ways to use them.

I’ll start the series with the A/B test, one of the most fun­da­men­tal tools in your arse­nal. With mul­ti­vari­ate tests get­ting the lion’s share of atten­tion in the press, the effec­tive­ness of an A/B test is often over­looked. That’s too bad, since an A/B test is often the base­way to start testing.

First, let’s define some terms. For our pur­poses, I’ll define an A/B test as a live test in which you mea­sure the per­for­mance of two or more pages against each other. Exam­ples would be test­ing two ver­sions of your site’s home­page, or test­ing three ver­sions of a par­tic­u­lar land­ing page.

My OTTO Dig­i­tal clients are often sur­prised when I sug­gest start­ing with an A/B test. But once I explain my rea­son­ing, they quickly sign up with the plan. Here is why I rec­om­mend start­ing with an A/B test:

An A/B test encour­ages you to test rad­i­cal alternatives

Unlike mul­ti­vari­ate tests, which are designed to rig­or­ously test spe­cific ele­ments on a page, an A/B test encour­ages you to try totally new page designs. You should take advan­tage of this fact and test out ideas that dif­fer sig­nif­i­cantly from your con­trol. At OTTO Dig­i­tal we often tell clients that if you tack your tested alter­na­tives up on a wall, you should be able to tell them apart from across the room.

An A/B test can gives you fast results

With an A/B test, you divide traf­fic into just a few buck­ets. If you’ve cre­ated recipes with dra­matic dif­fer­ences, you’re likely to see a recipe or two emerge as a win­ner more quickly than you would with a more com­plex mul­ti­vari­ate test. For get­ting results fast, noth­ing beats an A/B test.

A/B tests are sim­ple to design and interpret

This is an impor­tant fac­tor to con­sider, espe­cially if site opti­miza­tion is a new dis­ci­pline at your com­pany. Few things set back a nascent test­ing effort more than an excit­ing test that is later revealed to be flawed in some way. An A/B test pro­vides a straight­for­ward setup, with straight­for­ward results.

An A/B test helps you focus your future MVT efforts

Once you run an A/B test and know which recipe is per­form­ing the best, you can fur­ther opti­mize that win­ner with a round of rig­or­ous MVT test­ing. You’ll now be test­ing off an improved base­line. A/B test­ing is unmatched in its abil­ity to reveal a win­ning con­cept and guide your future efforts.

So there you have it. Don’t hes­i­tate to run the hum­ble but effec­tive A/B test. It’s often the best first step on your road to site optimization.

  • http://www.marketo.com/b2b-marketing-resources/landing-page-test-calculator.php Jon Miller

    One way to tell whether A/B test­ing is right for you us to use an online cal­cu­la­tor to tell you how many test ver­sions a given page can sup­port and still give sig­nif­i­cant results in a timely fash­ion:
    http://​www​.mar​keto​.com/​b​2​b​-​m​a​r​k​e​t​i​n​g​-​r​e​s​o​u​r​c​e​s​/​l​a​n​d​i​n​g​-​p​a​g​e​-​t​e​s​t​-​c​a​l​c​u​l​a​t​o​r​.​php

  • http://www.leadliaison.com rev­enue gen­er­a­tion for b2b

    A/B test­ing is great for land­ing pages or pages that con­tain web forms. I’d also sug­gest using A/B test­ing for email cam­paigns. Most professional-grade email mar­ket­ing tools pro­vide A/B test­ing capa­bil­i­ties with click through rates and ana­lyt­ics to help you quickly mea­sure effec­tive­ness of these real-life tests.

  • http://www.kompatible-toner-kartuschen.de Toner online

    This is an impor­tant fac­tor to con­sider, espe­cially if site opti­miza­tion is a new dis­ci­pline at your com­pany. Few things set back a nascent test­ing effort more than an excit­ing test that is later revealed to be flawed in some way. An A/B test pro­vides a straight­for­ward setup, with straight­for­ward results.

  • Follow Adobe Digital Marketing

    Fol­low @AdobeDigMktg
  • Popular Posts

    • Excellent Blog Post — Getting More from your Omniture Implementation.4
    • Tim Tebow and Mobile Marketing in 2012 (Part 1)2
    • Change the Conversation: What does “Efficiency” really mean?2
    • My work, My passion — Customer Analytics1
    Adobe Digital Marketing Blog

    Pages

    • Digital Marketing
    • Analytics
    • Personalization
    • Industries
    • Executive Insights

    The Latest

    • Using Dependent Code: Adding the Twitter Handle Name to Your Referring Traffic
      A common request I hear from customers is the desire to integrate […]

    More

    See how Adobe is changing the world through digital experiences. We are the leader in delivering solutions that let customers produce, distribute, and realize value from great content, whether in media and publishing or digital marketing.
    © 2012 Adobe Systems Incorporated. All Rights Reserved.
    Tweet