The Collaboration of Testing Ideas

I thought for this post I would build on the theme of col­lab­o­ra­tion that Lily Chiu dis­cussed in her post It Takes a Test­ing Vil­lage. As a cit­i­zen of this Test­ing Vil­lage myself, I thought it could be help­ful to put together a list of web­site test ideas that I hope will be sup­ple­mented by all my fel­low Test­ing Vil­lage cit­i­zens out there like you. Test ideas are not too hard to come by once you get started with test­ing. When those ini­tial test results arrive and the learn­ings are shared, many ideas present themselves.

The ini­tial learn­ings will allow you to run fol­low up tests on other areas of your site to recon­firm the find­ings and also allow you to fur­ther lever­age the gains. The learn­ings could also present an oppor­tu­nity to run a cham­pion chal­lenger test by try­ing new con­tent against your new win­ner. Seg­ment analy­sis of your tests is another way to get the ideas flow­ing. You will see that these dif­fer­ent seg­ments can respond dif­fer­ently to test con­tent. When this hap­pens you will be pre­sented with many oppor­tu­ni­ties to lever­age those gains and learn more about these seg­ments of traf­fic which will even lead to more wins and test­ing ideas. But if you haven’t tested before, where would you start? When you step back and real­ize that you have an entire web­site to opti­mize it can be daunt­ing to decide where to dive in first.

Many com­pa­nies that I work with like to start with opti­miz­ing the traf­fic they are pay­ing for. This includes paid search traf­fic, email traf­fic as well as any affil­i­ate traf­fic. These areas are incred­i­ble areas to start with and are rel­a­tively easy to set up using tar­get­ing. These paid seg­ments of traf­fic shouldn’t be the sole focus of your opti­miza­tion efforts though. There are many other oppor­tu­ni­ties to opti­mize traf­fic that could lead to even big­ger returns on your investment.

Some of the best ideas that I see tested often come not from Mar­keters but rather the Cre­ative, Devel­op­ment, and IT groups within an orga­ni­za­tion. These groups live and breathe the con­tent and the set up of their web­site and they often have strong opin­ions as to what and how con­tent gets placed. Test­ing allows them to val­i­date their opin­ions or encour­ages a visit back to the draw­ing board. For these groups of peo­ple, I find the best approach to get the ideas flow­ing is dur­ing or after a walk through of the tech­nol­ogy used to test.

Just three weeks ago I was doing a kick off with a client where we were review­ing land­ing page oppor­tu­ni­ties for paid traf­fic when one of their IT man­agers who was in the room asked if the test­ing plat­form can be used to solve one of their inter­nal site issues. They have a site where the con­tent and prices are deter­mined by your sta­tus after log­ging in. For quite some time they wanted a way to mer­chan­dize spe­cific con­tent to these dif­fer­ent groups of peo­ple. After a quick review of how pro­files work, they have a few tests lined up that will show spe­cific con­tent based on their vis­i­tors logged in sta­tus. Since the vast major­ity of their traf­fic is return­ing vis­i­tors the poten­tial gains here could sig­nif­i­cantly out way any gains on paid traffic.

Test­ing does not have to live in a silo, nor should it. If you want to get the biggest bang out of your test­ing buck the best approach is coor­di­nate really well and have all stake­hold­ers of the web­site present ideas and test them. The shar­ing of test results is also key to mak­ing this happen.

Each site is dif­fer­ent though. They dif­fer in terms of traf­fic and what their key suc­cess met­rics are. For some spe­cific ideas on your par­tic­u­lar web­site I invite you to share your site with me and the other cit­i­zens here and we can look at it together here or in the Web­site Opti­miza­tion Group. In the mean­time, here are some gen­eral test ideas that could apply to many of the sites out there.

Acqui­si­tion:

• Ban­ner test­ing — opti­mize ads placed on third party sites such as affil­i­ate or media sites
• Route test­ing — send your paid traf­fic to dif­fer­ent land­ing pages to see what works best (home page, cat­e­gory or prod­uct page, or even inter­nal search results page)
• Ban­ner Rein­force­ment — rein­force ad or paid mes­sag­ing con­tent on the land­ing page or through­out every page the vis­i­tor can go to. (Mul­ti­page test­ing is quite inter­est­ing and has deliv­ered some pretty pow­er­ful results)
• Find your best search ad — run a mul­ti­vari­ate test on the head­lines, copy, and links. This allows you to quan­tify the impact of each of these elements!!
• Grow your email data­base — test loca­tion and pre­sen­ta­tion of email reg­is­tra­tion forms. Test pos­si­ble incen­tives to reg­is­ter­ing. (another cool use of pro­files is remov­ing this form all together if vis­i­tors already reg­is­tered thus free­ing up real estate for more content
• Email test­ing I — include sub­ject lines, time sent, and day sent as mul­ti­vari­ate variables
• Email test­ing II — test dif­fer­ent con­tent within emails or dif­fer­ent pro­mo­tional content
• Build a story — rather then repeat ban­ner ele­ments on the land­ing page, try build­ing a nar­ra­tive from the ad through the fun­nel of your site (another exam­ple of mul­ti­page testing)
• Incen­tive thresh­old — test dif­fer­ent incen­tive lev­els (includ­ing no incen­tive) for appli­ca­tions, reg­is­tra­tions, for order completion
• Affil­i­ate offer rein­force­ment — rein­force the affil­i­ate pro­mo­tion on the land­ing page (there is a great case study avail­able with Musi­cians Friend that high­lights the easy wins that are avail­able here)
• Viral Mar­ket­ing — test dif­fer­ent mar­ket­ing pro­mo­tions upon com­ple­tion of check­out (e.g. send your friends 10% off coupons)
• Form Opti­miza­tion — test how many fields you can add before hav­ing a neg­a­tive impact or where and how the form is presented

Con­tent:

• % Off or $$ Sav­ings — test not only what pro­mo­tion you offer, but also how you dis­play the pro­mo­tion (seg­ments are a must for this test)
• CTA (Call To Action) — test your CTA copy (Learn More, Buy now, Start here, etc…)
• CTA — test dif­fer­ent col­ors and SIZE. See if it holds true that red but­tons always win
• Cus­tomer Feed­back — test adding cus­tomer tes­ti­mo­ni­als and/or rat­ings and where and how often to place them
• Encour­age cus­tomers to act — test dif­fer­ent scarcity mes­sag­ing (e.g. “Lim­ited Time Only”, “Offer expires on…..”, or “While Sup­plies Last”). On a side note, many of my cus­tomers are using the tar­get­ing in the tool to switch out con­tent let­ting vis­i­tors know how much time is left to pur­chase in order to have their items arrive before the holidays
• Give the copy per­son­al­ity — test dif­fer­ent mes­sag­ing approaches: infor­ma­tive, aggres­sive, funny, pro­mo­tional, brand-focused, or ben­e­fits ori­en­tated (seg­ment analy­sis here would be well worth its time)
• Increase trust — test where and how to place con­fi­dence infor­ma­tion such as return pol­icy, ship­ping info, pay­ment details, cus­tomer ser­vice numbers
• Secu­rity Mes­sag­ing — incor­po­rate and test place­ment of trust logos such as Verisign, TrustE, or Hack­er­Safe (I have never seen this not pro­vide lift and place­ment does mat­ter
• Loca­tion, loca­tion, loca­tion — test not dif­fer­ent con­tent but rather dif­fer­ent place­ments of the same con­tent (quick to do as you don’t have to cre­ative alter­na­tive content)
• Per­son­al­ize the cus­tomer expe­ri­ence — tar­get con­tent based on the vis­i­tors on site behav­ior (if they show a pref­er­ence for a cat­e­gory or prod­uct, use pro­files to rein­force con­tent on other areas of the site)
• Pro­mo­tion manip­u­la­tion — test one large pro­mo­tion ver­sus sev­eral smaller promotions
• Rich Media — test rich media vs. simple/quick con­tent (Make sure the “rich” con­tent offers addi­tional infor­ma­tion or sig­nif­i­cant expe­ri­ence that can’t be com­mu­ni­cated in a flat format)
• Remove Con­tent — see if remov­ing con­tent on the site affects suc­cess metrics

Mer­chan­dis­ing:

• Bulk Shop­ping — test dif­fer­ent approaches to adding mul­ti­ple prod­ucts to your cart simultaneously
• Cart Aban­don­ment — when a vis­i­tor returns to the site, rein­force the fact that they have pre­vi­ously aban­doned their cart (Here is a post that walks through an exam­ple)
• Max­i­mize your profit mar­gin — Is free ship­ping or % off net of dis­count amount?
• Pro­mo­tion Thresh­olds — test pro­mo­tional thresh­olds (e.g. 10% off $50 order ver­sus 15% off $100 order)
• Searchan­dis­ing 1.0 — test dif­fer­ent items to appear at the top of inter­nal search result pages
• Searchan­dis­ing 1.1 — test tar­geted promotions/promotional ban­ners based on search term
• What prod­uct should be dis­played — (e.g. best sell­ers, hand picked vs. auto­mated, edi­tor picks, big brand vs. big seller, deal of the day, peo­ple who bought this bought that, peo­ple who viewed this bought that)

These ideas are just a start to what I hope will be an exten­sive and dynamic list that may help peo­ple out there start opti­miz­ing or even expand their cur­rent test­ing efforts to new areas. So my fel­low cit­i­zens, I ask you to please share any ideas you have and I will be sure to add them to a Mas­ter List that will be avail­able at the Web­site Opti­miza­tion Group. Here are some strate­gies to focus some of your ideas:

• Focus your test­ing objec­tives — Under­stand if your pri­or­ity is con­ver­sion, aver­age order value, or rev­enue per vis­i­tor. The more focused you are, the clearer your results will be. I per­son­ally rec­om­mend incor­po­rat­ing as many suc­cess met­rics as pos­si­ble into any test that you run. You may find that while your test pos­i­tively affects one met­ric, it may neg­a­tively affect another
• Make the most impact — look to pos­si­bly start at the bot­tom of a con­ver­sion fun­nel — not at the top. Improv­ing the reg­is­tra­tion or shop­ping cart gen­er­ally pro­vides greater impact on the bot­tom line
• What now? — If a par­tic­u­lar recipe or branch of your cam­paign wins, what are the learn­ings and how can they be used in the future
• Test plan — Plan test­ing “themes” instead of jump­ing from page to page, ele­ment to ele­ment, or cam­paign to cam­paign. Some good themes are reg­is­tra­tion drop off improve­ment, copy style and length, and PPC con­tent targeting
• Test what mat­ters — Small changes to small ele­ments always yield small results. For big impact, focus on prod­uct shown, pric­ing, pri­mary copy, images, offers and the ever impor­tant call to action
• Who should get into my test? — Start off test­ing by allow­ing many chan­nels or seg­ments of traf­fic into your test so as to get a sense of how they respond to dif­fer­ent con­tent. The more traf­fic that gets into your test, the quicker you will have sta­tis­ti­cally sig­nif­i­cant results

Addi­tion­ally, if you are a Test and Tar­get cus­tomer and would like to know how any of these cam­paigns are set up please let me know. I can pos­si­bly even add them to my demo account on www​.Tes​tand​Tar​get​ing​.com/​?​s​i​d​=​p​o​st3 so you could see them live and in action and view the set up for your­self in the Tool