Busi­nesses have embraced web­site test­ing and opti­miza­tion over the past decade for many rea­sons. Being able to quickly learn vis­i­tor pref­er­ences and adjust to them is one rea­son. Hav­ing an addi­tional tool to con­duct research in prepa­ra­tion for a major web­site over­haul is another. But the pri­mary rea­son most get started is that test­ing can lead to web­site improve­ments that trans­late into incre­men­tal sales.

So how does a test­ing team ensure that they use their finite resources to deliver sig­nif­i­cant impacts to the busi­ness? Let’s talk through ten tips for design­ing tests.

Tip #1 — Make sure the page you plan to test has sig­nif­i­cant traffic

This might be an obvi­ous one, how­ever there are two nuances that I want to under­line. First, “sig­nif­i­cant traf­fic” does not mean that you are lim­ited to test­ing on your home page. At the same time, many teams, after get­ting a test­ing pro­gram off and run­ning for­get to check the traf­fic lev­els for each page that they plan to test. What are the daily unique vis­i­tors to the page? What is the con­ver­sion rate for the page? Based on those num­bers, how long do you antic­i­pate need­ing to run the test before you will reach full con­fi­dence and be able to call the test com­plete? These are ques­tions that you should answer in advance.

Tip #2 — Ensure the page you plan to test is impor­tant to the con­ver­sion event you are try­ing to improve  

If you are an e-commerce web­site and your main con­ver­sion event is an online pur­chase, make sure the page you have cho­sen is part of the pur­chase fun­nel, or one that vis­i­tors com­monly visit as part of their research and pur­chase path.

Tip #3 — The page ele­ment that you plan to test needs to be noticeable

Pages vary dras­ti­cally from being min­i­mal­is­tic and clean to hav­ing a great num­ber of ele­ments on a page that scrolls down ver­ti­cally seem­ingly for­ever. The notice­abil­ity of ele­ments can be sub­jec­tive and dif­fi­cult to judge, but here are a few guide­lines. First, ide­ally the object you plan to test is on screen (above the fold) when the page is ren­dered on the visitor’s device, what­ever the size of that device’s screen. Next, if the page does have a large num­ber of ele­ments, the one you have cho­sen to test needs to be suf­fi­ciently large or promi­nently placed so that a change to it will be some­thing that your web­site vis­i­tors will notice.

Tip #4 — Alter­nate expe­ri­ences need to be dif­fer­ent in a mean­ing­ful way

One com­mon pit­fall in a test­ing orga­ni­za­tion is to get too indoc­tri­nated into “that’s how we do things around here” or to lose bat­tles to the brand police. The test­ing team should always strive to pro­duce alter­nate con­tent that is mean­ing­fully and impact­fully dif­fer­ent than the default con­tent. I often use an extreme anal­ogy when stress­ing this point — if you were shop­ping for new car­pet for your home, choos­ing between a light tan, medium tan, and a dark tan color is much less likely to draw a strong reac­tion than choos­ing between tan, orange, or pur­ple. This is an extreme exam­ple — not many peo­ple I know have orange or pur­ple car­pet, but it illus­trates what we should be striv­ing for in design­ing alternates.

Tip #5 — Ensure the busi­ness is pre­pared to push the win­ning recipe

Suc­cess­ful test­ing orga­ni­za­tions believe in the con­cept of test­ing. They deter­mine the win­ner based on a solid foun­da­tion of data, and are eager and will­ing to “push” the win­ning expe­ri­ence live once results are in. Early in the pro­gram, if the orga­ni­za­tion hasn’t yet fully embraced test­ing, it is best to con­duct tests that are sim­pler and smaller in scope. Crawl>Walk>Run

Tip #6 — If tar­get­ing, ensure your seg­ments are covered

If you are using seg­men­ta­tion strat­egy for serv­ing con­tent on your web­site, make sure that the page you have cho­sen to test is being vis­ited by your tar­get seg­ments at a rate suf­fi­cient to war­rant the test.

Tip #7 — Test areas of your site that chal­lenge cur­rent best practices

While in some cases test­ing may prove that your web­site vis­i­tors pre­fer an expe­ri­ence that dif­fers from a best prac­tice, my advice would be to always guide your web­site toward adher­ing to cur­rent best prac­tices and use test­ing to prove where in some cases your unique web­site vis­i­tors may dis­play dif­fer­ent pref­er­ences or behav­ior. As web­sites and devices are rapidly shift­ing, keep­ing up with best prac­tices in itself can be a challenge.

Tip #8 — Test as close to the “end con­ver­sion page” as you can

If you are an e-commerce site, test­ing as close to the page where vis­i­tors click “Com­plete pur­chase” or if you are a finan­cial insti­tu­tion, the page where prospects click “Sub­mit appli­ca­tion”, will deliver the most impact­ful results. It takes time for a busi­ness to learn and fully adapt the tools and processes nec­es­sary to con­duct test­ing. As these fun­nels or pages are crit­i­cal to the busi­ness, trust of the test­ing pro­gram must be in place before many busi­nesses will feel com­fort­able test­ing. But once there, these tests are the most poten­tially impactful.

Tip #9 — Con­sider the visitor’s device

As the world con­tin­ues to shift toward smart phones and tablet usage, busi­nesses must to con­sider how to best serve the needs of these dif­fer­ent expe­ri­ences. One approach is attempt to design some­thing flex­i­ble that works well for all com­put­ers, mobile phones, and tablets. Another approach is to design dif­fer­ent expe­ri­ences — 2 or 3 — for these sce­nar­ios. Look at the data for the page you plan to test and deter­mine which approach makes sense now for your busi­ness for this page.

Tip #10 — Make sure the test is achievable

After con­sid­er­ing all the aspects of how to design a test, a plan can get very com­plex. Based on where your busi­ness is with test­ing and your group’s abil­ity to design, code, exe­cute, and ana­lyze the results, be pre­pared to reduce scope and com­plex­ity of a test if it seems to have got­ten too ambi­tious. If you never launch the test, you can­not deliver impact­ful lift. Bal­ance the aspi­ra­tions of the team with the real­i­ties of your cur­rent resources and abilities.