In May we intro­duced a new series where Adobe experts pro­vide “Quick tips” to our com­mu­nity related to a par­tic­u­lar topic. May’s topic theme was “Test Every­thing” and you may have seen the tips we posted each week on our Face­book page. We’ve com­piled each of our weekly tips from May along­side great testing-related arti­cles and con­tent from across the web, all in one place for your view­ing plea­sure. Enjoy!

Adobe Quick Tips Test Everything

Week One: “As you set out to test every­thing remem­ber that you don’t want to test it all at once. The com­pa­nies that try to test it all now end up doing what we call ran­dom testing—they move from one area to the next and never see the large gains obtained by iter­at­ing. Test every­thing by not allow­ing any changes to hap­pen on your site with­out test­ing first. Then as you make changes remem­ber to iter­ate on those changes to truly learn and cap­ture the full mon­e­tary value wait­ing for you.”

The first Quick Tip in our Test Every­thing series comes from Rhett Nor­ton, a senior con­sul­tant and retail team lead at Adobe who works with top brands such as Red­box, Sta­ples and Wal­greens on their test­ing and opti­miza­tion campaigns.

Week Two: “Pri­or­i­tiz­ing your efforts based on data is the first stage in build­ing a suc­cess­ful test­ing pro­gram; this is the frame­work to a good test­ing method­ol­ogy and process. Once you know where to focus your opti­miza­tion effort, be sure to test mul­ti­ple options – don’t just stop at A/B, but cre­ate an expe­ri­ence C, D, etc. as this will give you more data and help you under­stand where to iter­ate for your next test. Don’t for­get to add seg­ments so you can see in your reports how your var­i­ous vis­i­tors reacted to the cam­paign – if they behave dif­fer­ently than your audi­ence as a whole, start tar­get­ing that expe­ri­ence to them! There’s a crawl, walk, run process to opti­miza­tion – be sure to have a method to the mad­ness or else you may be stuck on all fours.”

Gina Casagrande is a con­tent and con­ver­sion evan­ge­list at Adobe and has also spent time as part of the company’s con­sult­ing solu­tions orga­ni­za­tion, work­ing with top brands to opti­mize their con­ver­sion efforts and improve their mar­ket­ing ROI.

Week Three: “Each test you run should have a hypoth­e­sis, not just an idea. The dif­fer­ence is, a hypoth­e­sis is some­thing you can prove false through the results of your test. Hypothe­ses are the secrets to action­able insights, they help you under­stand the WHY behind the per­for­mance of your vari­ants – they iso­late the dif­fer­ence between vari­ants, help you under­stand your users, and smooth the path to repeat­able suc­cess. This keeps your site from look­ing like a ransom-note style hodge­podge of test win­ners that don’t hang together.”

Rachel Elk­ing­ton is a Senior Con­sul­tant and Opti­miza­tion Man­ager in Adobe’s Dig­i­tal Mar­ket­ing prac­tice. She loves using test­ing and tar­get­ing to empower her clients to meet their goals through under­stand­ing and influ­enc­ing user behavior.

Week Four: “Test­ing needs to work hand-in-hand with your cre­ative process, rather than derail­ing it. Make sure you’ve built mini progress tests into your design or con­tent devel­op­ment process, allow­ing for the test­ing of ele­ments as they’re being cre­ated to bet­ter focus on the most effec­tive con­tent per cus­tomer seg­ments. Iter­a­tive test­ing, or the con­tin­u­ous test­ing and opti­miza­tion of your site expe­ri­ence, should not only be a tech­nique for pub­lished mate­r­ial, but also a way of improv­ing the rel­e­vance of your exper­i­men­tal content.

Meth­ods for test­ing new mate­r­ial with­out inter­fer­ing with your cur­rent site expe­ri­ence are:

1) Tar­get the exper­i­ments to smaller sub­sets or groups of cus­tomers for a brief period, in higher traf­fic but not crit­i­cal pages of your site

2) Iden­tify the loca­tion of your tests on your site as experimental

3) Ask for feed­back directly from your cus­tomers. Often­times, empow­er­ing your vis­i­tors to respond to new mate­r­ial can improve engage­ment and the social shar­ing of your mate­r­ial, pro­vid­ing quicker and broader results.”

Drew Burns is the senior prod­uct mar­ket­ing man­ager for Adobe Test&Target and works to evan­ge­lize the prac­tice of iter­a­tive test­ing and con­tent tar­get­ing in the dig­i­tal mar­ket­ing world.

Week Five: “When start­ing out with your test­ing pro­gram dig into oppor­tu­ni­ties for user expe­ri­ence enhance­ments on lower fun­nel pages.  Increas­ing con­ver­sion within the check­out fun­nel will have an imme­di­ate impact on rev­enue and will also ensure that your test­ing efforts on upper fun­nel pages aren’t can­ni­bal­ized by high drop-off rates deeper in the site.  Start from the bot­tom and work your way up to max­i­mize cus­tomer through­put and quickly prove the value of your opti­miza­tion program.”

Whit­ney Lit­tle­wood is a Team Lead on the Adobe Dig­i­tal Con­sult­ing team who part­ners with clients to drive mea­sur­able return on their online mar­ket­ing invest­ment through the adop­tion of opti­miza­tion best practices.


CMO​.com: Case Study: Adobe Puts Own Soft­ware To The Test

Brooks Bell: The Golden Ticket of Test­ing: Under­stand­ing Con­fi­dence

Adobe Dig­i­tal Mar­ket­ing Blog: Hoard­ers — Web­site Edi­tion

Muddy Lemon: Intro­duc­tion to A/B Test­ing for Land­ing Pages

Brooks Bell: 5 Must-Read Opti­miza­tion Posts for the Week of April 30

Brooks Bell: The Golden Ticket of Test­ing: Under­stand­ing Confidence


Mar­ket­ing Land: How to Con­vince Your Man­age­ment Team to Test And Opti­mize Your Web­site

GrowS­mart­Biz: 13 Help­ful Guide­lines for A/B Test­ing

Wired: The A/B Test Results Are In

Wired: Test Every­thing: Notes on the A/B Rev­o­lu­tion

Adobe Dig­i­tal Mar­ket­ing Blog: BUYER BEWARE: No seg­ment fil­ter­ing in test results? Not an effec­tive test­ing or tar­get­ing tool

Online​-Behav​ior​.com: Split Test­ing Frame­work

Hope you enjoyed the com­pi­la­tion! In June 2012, we’re hon­ing in on personalization-related top­ics and strate­gies; so keep check­ing the Adobe Dig­i­tal Mar­ket­ing Face­book page for new tips, and stay tuned for a sim­i­lar round-up on per­son­al­iza­tion con­tent at the end of this month. What other top­ics would you like to see us cover? Are there other types of con­tent and resources that you’d like to see us aggre­gate here? Send us your thoughts.