As an ana­lyt­ics pro­fes­sional at a media com­pany, you have just received your login to Adobe Test&Target. You ner­vously enter your pass word and you are in. There before you is a shiny, yet for­eign inter­face. Now what?  It took you two years to become a pro with Site­Cat­a­lyst and now you feel com­fort­able quickly cruis­ing through the inter­face and cus­tomized reports you have built. But now you are back at square one with test­ing, only this time you can really do some dam­age with your new-found abil­ity to mod­ify con­tent on your site.  Where do you start?

Hav­ing helped dozens of clients get off the ground and run­ning with their first sev­eral tests, I can tell you the ques­tion you face is one that chal­lenges most new testers. My advice is to step back and remem­ber that you are not going to boil the ocean all at once. Pick a test that allows you to answer “YES!” to these three impor­tant questions:

1.     With the resources at your dis­posal, can you real­is­ti­cally launch this test in a short time?
2.     Is the change likely to pro­duce an impact?
3.     Will your orga­ni­za­tion actu­ally imple­ment the change if it proves to be a winner?

In my expe­ri­ence, a great place for media com­pa­nies to start that meets the three con­di­tions above is a lay­out test.

Lay­out tests may not require as many design resources as you think.
There is the ten­dency to con­fuse test­ing a lay­out with test­ing a redesign. Those are two sep­a­rate con­cepts. Whereas test­ing a redesign involves changes to loca­tion AND cre­ative, test­ing lay­out can involve sim­ply rear­rang­ing the exist­ing cre­ative on the page to see if there is a bet­ter posi­tion or order.  As such, many changes can be imple­mented with CSS, JavaScript, or jQuery that you can use a Test&Target mbox to deliver to the page. With the excep­tion of get­ting the mbox on the page to serve the offer, you may not even need to touch the source code. Another impor­tant thing to con­sider is that a lay­out test does not need to hap­pen on the home page. Instead it could be on a land­ing page, sec­tion front, or less polit­i­cally charged con­tent template.

These types of lay­out changes have increased impor­tance for media-based sites, which are gen­er­ally tem­plate dri­ven because the actual con­tent updates fre­quently. For this rea­son it is impor­tant to focus opti­miza­tion efforts on the tem­plate itself via lay­out and style changes. Test­ing the actual con­tent could poten­tially be very inef­fi­cient if the con­tent has a short shelf life.

Lay­out changes are very visual and are likely to have a large impact.
You can change the font from Arial to Arial Nar­row and peo­ple may not notice, but if the “Most Pop­u­lar Videos” mod­ule is in the left rail instead of the right rail, it changes how peo­ple inter­act with the page. Maybe vis­i­tors tend to scan top-left to bottom-right and your change puts that mod­ule in their line of site, or removes it. A test will help you prove that out.

 What Hap­pens if I Test a Los­ing Recipe
You can learn just as much from a los­ing recipe as a win­ning recipe. If your chal­lenger recipe pro­duces a 10% decline in site engage­ment, your test has saved you from mak­ing a blind change that would have reduced poten­tial ad rev­enue by 10% or more (assum­ing an ad-based CPM model).

A neu­tral test also tells you some­thing. If chang­ing the lay­out for the spe­cific tem­plate tested has no impact on engage­ment or suc­cess, a layout-intensive redesign may not be the most effi­cient way to increase engage­ment. For the sec­tion or tem­plate tested, you may want to try test­ing some­thing else less resource inten­sive that is still likely to pro­duce results, such as the num­ber of “related-articles” to display.

Test Mul­ti­ple Lay­outs
Many testers are frus­trated when their chal­lenger recipe does not work out because they are at a loss for what to do next, won­der­ing if they should just try a com­pletely new idea.  You can sig­nif­i­cantly increase your chance of find­ing a win­ner and learn­ing about your page by test­ing more than one chal­lenger. In fact, the addi­tional learn­ing you pull out of a test for doing third, fourth, and even fifth recipes will usu­ally far out­weigh the effort required to add those addi­tional recipes.

Return­ing to the “Most Pop­u­lar Videos” mod­ule lay­out exam­ple, if your chal­lenger recipe fea­tur­ing the mod­ule at the bot­tom of the right rail under­per­forms your default mod­ule at the top of the right rail, you do not know if chang­ing lay­out was a bad idea, or that par­tic­u­lar place­ment on the right rail was a bad exe­cu­tion of a good idea. Maybe the change pushed the mod­ule below the fold. There could be sev­eral rea­sons for the decline, but with only one chal­lenger it can be dif­fi­cult to drill into what caused the change.

If you test the mod­ule at top right (default), top-left (recipe B), bottom-left (recipe C), and bottom-right (recipe D), you are less likely to be thwarted by the bad exe­cu­tion of a good idea  and more likely to find some­thing that per­forms bet­ter than what you cur­rently have. 

Site opti­miza­tion through test­ing can be as com­plex as you make it. Do not bite off more than you want to chew with your first test. A lay­out test is a great way to run a sim­ple test that pro­duces an impact.

0 comments