Winning with Testing: Top 10 Tips
Businesses have embraced website testing and optimization over the past decade for many reasons. Being able to quickly learn visitor preferences and adjust to them is one reason. Having an additional tool to conduct research in preparation for a major website overhaul is another. But the primary reason most get started is that testing can lead to website improvements that translate into incremental sales.
So how does a testing team ensure that they use their finite resources to deliver significant impacts to the business? Let’s talk through ten tips for designing tests.
Tip #1 — Make sure the page you plan to test has significant traffic
This might be an obvious one, however there are two nuances that I want to underline. First, “significant traffic” does not mean that you are limited to testing on your home page. At the same time, many teams, after getting a testing program off and running forget to check the traffic levels for each page that they plan to test. What are the daily unique visitors to the page? What is the conversion rate for the page? Based on those numbers, how long do you anticipate needing to run the test before you will reach full confidence and be able to call the test complete? These are questions that you should answer in advance.
Tip #2 — Ensure the page you plan to test is important to the conversion event you are trying to improve
If you are an e-commerce website and your main conversion event is an online purchase, make sure the page you have chosen is part of the purchase funnel, or one that visitors commonly visit as part of their research and purchase path.
Tip #3 — The page element that you plan to test needs to be noticeable
Pages vary drastically from being minimalistic and clean to having a great number of elements on a page that scrolls down vertically seemingly forever. The noticeability of elements can be subjective and difficult to judge, but here are a few guidelines. First, ideally the object you plan to test is on screen (above the fold) when the page is rendered on the visitor’s device, whatever the size of that device’s screen. Next, if the page does have a large number of elements, the one you have chosen to test needs to be sufficiently large or prominently placed so that a change to it will be something that your website visitors will notice.
Tip #4 — Alternate experiences need to be different in a meaningful way
One common pitfall in a testing organization is to get too indoctrinated into “that’s how we do things around here” or to lose battles to the brand police. The testing team should always strive to produce alternate content that is meaningfully and impactfully different than the default content. I often use an extreme analogy when stressing this point — if you were shopping for new carpet for your home, choosing between a light tan, medium tan, and a dark tan color is much less likely to draw a strong reaction than choosing between tan, orange, or purple. This is an extreme example — not many people I know have orange or purple carpet, but it illustrates what we should be striving for in designing alternates.
Tip #5 — Ensure the business is prepared to push the winning recipe
Successful testing organizations believe in the concept of testing. They determine the winner based on a solid foundation of data, and are eager and willing to “push” the winning experience live once results are in. Early in the program, if the organization hasn’t yet fully embraced testing, it is best to conduct tests that are simpler and smaller in scope. Crawl>Walk>Run
Tip #6 — If targeting, ensure your segments are covered
If you are using segmentation strategy for serving content on your website, make sure that the page you have chosen to test is being visited by your target segments at a rate sufficient to warrant the test.
Tip #7 — Test areas of your site that challenge current best practices
While in some cases testing may prove that your website visitors prefer an experience that differs from a best practice, my advice would be to always guide your website toward adhering to current best practices and use testing to prove where in some cases your unique website visitors may display different preferences or behavior. As websites and devices are rapidly shifting, keeping up with best practices in itself can be a challenge.
Tip #8 — Test as close to the “end conversion page” as you can
If you are an e-commerce site, testing as close to the page where visitors click “Complete purchase” or if you are a financial institution, the page where prospects click “Submit application”, will deliver the most impactful results. It takes time for a business to learn and fully adapt the tools and processes necessary to conduct testing. As these funnels or pages are critical to the business, trust of the testing program must be in place before many businesses will feel comfortable testing. But once there, these tests are the most potentially impactful.
Tip #9 — Consider the visitor’s device
As the world continues to shift toward smart phones and tablet usage, businesses must to consider how to best serve the needs of these different experiences. One approach is attempt to design something flexible that works well for all computers, mobile phones, and tablets. Another approach is to design different experiences — 2 or 3 — for these scenarios. Look at the data for the page you plan to test and determine which approach makes sense now for your business for this page.
Tip #10 — Make sure the test is achievable
After considering all the aspects of how to design a test, a plan can get very complex. Based on where your business is with testing and your group’s ability to design, code, execute, and analyze the results, be prepared to reduce scope and complexity of a test if it seems to have gotten too ambitious. If you never launch the test, you cannot deliver impactful lift. Balance the aspirations of the team with the realities of your current resources and abilities.