Opti­miza­tion is a fuzzy word. It’s been used to describe every­thing from decreas­ing page load time to improv­ing user expe­ri­ence. I’m guilty of using it as a catch-all as well, but today I want to focus on two key com­po­nents when it comes to opti­miz­ing your site for conversion:

1) Test­ing — the process of test­ing alter­na­tives on your site to see what con­verts best. Test­ing is most often imple­mented as simul­ta­ne­ously deliv­er­ing alter­na­tive con­tent to your vis­i­tors in a ran­dom­ized fashion.

2) Tar­get­ing — the process of dis­play­ing rel­e­vant con­tent to a par­tic­u­lar seg­ment or indi­vid­ual based on implicit and explicit vari­ables such as gen­der, search terms, and intent. Tar­get­ing is most often imple­mented as deliv­er­ing con­tent to your vis­i­tors in a rules-based fash­ion. For some com­pa­nies with enough site traf­fic, tar­get­ing can be deliv­ered to the indi­vid­ual in an auto­mated fashion.

Test­ing and Tar­get­ing: 1+1=3

Both efforts rely on the abil­ity to deliver con­tent dynam­i­cally on your site. Both are designed to increase key suc­cess met­rics on your site. How­ever, they are often sep­a­rated as two com­pletely dif­fer­ent ini­tia­tives within a mar­ket­ing orga­ni­za­tion. While peo­ple seem to be inter­ested in the idea of com­bin­ing them, they are usu­ally already siloed as dif­fer­ent projects with dif­fer­ent teams, bud­gets and time­lines by the time we have the dis­cus­sion. I think bring­ing test­ing and tar­get­ing together is really one of those cases where 1+1=3 though.

Let’s first talk about test­ing by itself. It’s a great way to fig­ure out what ver­sion your vis­i­tors pre­fer. If you’re just throw­ing new designs up on your site today on a wish and a prayer, test­ing is def­i­nitely a must. But then think about who your vis­i­tors are. Are they all peo­ple who are com­ing for the first time? Are they all vis­it­ing dur­ing work hours? Do they all enter on the same page look­ing for the same thing?

If your answer is no to any of these ques­tions, then you have to ask your­self if it makes sense to assume that they all pre­fer the same win­ner in any given test. The eas­i­est way to fig­ure this out is to seg­ment your report­ing. That doesn’t change any­thing about how the test is designed or imple­mented, but it can give you addi­tional learn­ings in the report­ing and analysis.

What if you found out that the rea­son a par­tic­u­lar ver­sion won was because it res­onated deeply with your new vis­i­tors com­ing organ­i­cally to your site? And bal­anc­ing out that great lift was actu­ally a neg­a­tive result with other smaller seg­ments such as direct mail and branded queries com­ing from paid search? This type of data is incred­i­bly valu­able for any orga­ni­za­tion try­ing to get deeper insights into what their vis­i­tors are look­ing for and respond­ing to. What makes it invalu­able is to then take action on those learn­ings by deliv­er­ing that tar­geted, win­ning expe­ri­ence to your organic, first-time vis­i­tors and then simul­ta­ne­ously begin­ning a new test on the rest of the traffic.

eHar­mony ran a test with us to under­stand whether adding tabbed nav­i­ga­tion to a land­ing page would be more effec­tive in get­ting users to con­vert. The hypoth­e­sis was that pro­vid­ing more infor­ma­tion about the ser­vice directly on the land­ing page would be a pos­i­tive change. Here are the dif­fer­ent versions.

What we found was that the test actu­ally per­formed worse for over­all traf­fic, and that peo­ple pre­ferred the sim­pli­fied page with­out the AJAX nav­i­ga­tion. Once we dug into our seg­ments though, we found a more com­plex and inter­est­ing story. We had set up geo-segments to track how peo­ple from dif­fer­ent coun­tries behaved, and it turned out that vis­i­tors from Canada sig­nif­i­cantly pre­ferred the nav­i­ga­tion. This learn­ing would never have been found with­out seg­men­ta­tion, and worse, the tabbed nav­i­ga­tion would have prob­a­bly been thrown out based on the over­all neg­a­tive lift.

What is Tar­get­ing With­out Testing?

What is tar­get­ing with­out test­ing though? Isn’t it the same as just putting your hypoth­e­sis out onto the site and hop­ing your hunch aligns with your vis­i­tors? For exam­ple, let’s say you own a retail cloth­ing site, and you want to tar­get vis­i­tors who look at acces­sories and then return to the home page. You may won­der which acces­sories to now rein­force upon their return, but what if you should instead be ask­ing your­self whether acces­sories is the right cat­e­gory to show? What if you should be fea­tur­ing women’s cloth­ing instead because there is a high cor­re­la­tion between the two or the poten­tial for higher mar­gin profit? Or what if show­ing 2 fea­tured prod­ucts instead of 6 would make all the dif­fer­ence? Do you know which price range you should be stay­ing between? All of these ques­tions are great oppor­tu­ni­ties to com­bine test­ing and tar­get­ing into a sin­gle ini­tia­tive focused on cre­at­ing con­ver­sion and rev­enue on your site.

Baby­Cen­ter ran a test with Test&Target to under­stand the impact of increas­ing rel­e­vancy on one of their most highly-trafficked key­words: “baby names.” Below is the con­trol ver­sion, along with the alter­na­tives tested against it. What’s inter­est­ing to note here is that all of the alter­na­tives have some ele­ment of tar­get­ing in them. Recipe B offers some rel­e­vant text, Recipe C adds some cat­e­gory links, Recipe D pro­vides top 5 names of 2005 for both girls and boys, and Recipe E builds on Recipe B with a few more bul­lets of infor­ma­tion. Can you guess which one won?

If you guessed Recipe D, then you are in sync with most of the mar­keters who see this case study. How­ever, the answer is Recipe B, the alter­na­tive that pro­vides the sim­plest rein­force­ment of just a head­line and a cou­ple lines of copy. Here’s the test data:

You can see that Recipe B pro­vided 65.32% lift against con­trol with very high sta­tis­ti­cal con­fi­dence. In fact, nearly every alter­na­tive per­formed bet­ter than the con­trol except for Recipe D, the ver­sion that peo­ple most often pick as the win­ner. It makes sense that we nat­u­rally grav­i­tate to Recipe D because it pro­vides us with the most rel­e­vant infor­ma­tion given that we are search­ing on the term, “baby names.” How­ever, since we are mea­sur­ing the suc­cess of the land­ing page based on those who sign up for the mag­a­zine, we have to bal­ance dri­ving both rel­e­vancy and engage­ment. With­out test­ing this tar­get­ing effort though, we would have no idea whether we picked the right mix.

If you are in the process of test­ing today, I’d strongly advo­cate set­ting up seg­ments so you can begin to see which ones behave dif­fer­ently. Tar­get­ing is a nat­ural exten­sion of test­ing, but don’t for­get to go back and test your hunches as well, whether they apply to one seg­ment or all of your traf­fic. Tak­ing action on your data is the best way to do more with less, a strat­egy we will all have to fol­low more closely through the imme­di­ate future.