There is a pesky fear that per­vades the mar­ket­ing world around the test­ing of dig­i­tal mar­ket­ing con­tent. The per­cep­tion is that test­ing will some­how replace or limit the cre­ative brain­storm­ing and “gut instincts” of a mar­ket­ing team or ad agency, and their desire to push the bound­aries of engage­ment.  That some­how “con­tent deter­mi­na­tion” or uncov­er­ing what works best with dif­fer­ent cus­tomers will make cut­ting edge cam­paigns less desirable.

On the con­trary, it allows com­pa­nies to quickly try new engage­ment styles and meth­ods, and see what “clicks” with their cus­tomers imme­di­ately, rather than wast­ing time with mate­r­ial that doesn’t.  Opti­miza­tion tools such as Adobe Test&Target have also become extremely user friendly;  they can be con­trolled com­pletely by a non-technical mar­ket­ing team, with IT involve­ment at an advi­sory level (free­ing up resources and offer­ing effi­cient con­trol by the busi­ness own­ers.)  In terms of invest­ment, many com­pa­nies are see­ing impres­sive lifts in con­ver­sion (10s to 1000s of per­cent­age lifts and beyond!) with minor adjust­ments to their web expe­ri­ences, within months and weeks of start­ing, recoup­ing yearly invest­ments quickly.  This is why opti­miza­tion and mar­ket­ing man­agers have quickly become evan­ge­lists of the test­ing process within their orga­ni­za­tions, and it’s always a blast to meet with them at dif­fer­ent events through­out the year to “geek out” over their lat­est findings.

I spent 3 inspir­ing days at the Hud­son Hotel in Man­hat­tan last week at an opti­miza­tion event called Click Sum­mit.  For those of you who are not famil­iar with it, I highly rec­om­mend sign­ing up: It is hosted by Brooks Bell, a com­pany which spe­cial­izes in assist­ing enterprise-level com­pa­nies and SMBs with orga­niz­ing, build­ing and per­fect­ing their online test­ing pro­grams within their cur­rent infra­struc­ture. Owner and Pres­i­dent Brooks Bell has a true pas­sion for test­ing and has been a strong evan­ge­list for the opti­miza­tion process for a decade.  For the past 3 years, she has gath­ered folks from the tech, media, retail, and finance indus­tries and other ver­ti­cals, as well as indus­try ana­lysts, for test­ing round­table dis­cus­sions at Click Sum­mit.  It calls into ques­tion why we all don’t get together more often: “A-ha” moments occurred every few min­utes at the event (no, we were not singing music from the 1980s rock band.) It was immensely help­ful to iden­tify com­mon obsta­cles in test admin­is­tra­tion, and how oth­ers are work­ing towards resolv­ing these issues.  We also shared best prac­tices in terms of build­ing momen­tum and being a good advo­cate for a test­ing cul­ture within our organizations.

Hud­son Hotel New York


Here are some of my “a-ha” moments, that “won’t be goooooooooone in a daaaaaaaaayyyyyyy” (OK, so now I am singing 80s music — couldn’t resist):

1)      The State of the Test­ing Union: First and fore­most, I was extremely intrigued by For­rester ana­lyst Joe Stanhope’sState of Online Test­ing” sur­vey pre­sen­ta­tion, in terms of cur­rent adop­tion of test­ing prac­tices and what the future will hold.  On aver­age, most com­pa­nies are just begin­ning to appre­ci­ate the impor­tance of test­ing and tar­get­ing in terms of improv­ing cus­tomer engage­ment and life­time cus­tomer value. 50% of sur­vey par­tic­i­pants were new to test­ing, but a sub­stan­tial num­ber were will­ing to spend upwards of $100K per year on a test­ing bud­get. Most are only run­ning 2–3 non-concurrent tests per month, and 16% of those that are test­ing are unclear as to what a mul­ti­vari­ate test is.  This points to the need for greater edu­ca­tion on more effec­tive test­ing prac­tices, beyond the sin­gle A/B test at high traf­fic loca­tions.  Look­ing to the future, Joe also men­tioned that there is a surge towards behav­ioral tar­get­ing, or tar­get­ing con­tent based upon highly pre­dic­tive anony­mous vari­ables rel­a­tive to a customer’s behav­ior.  This allows for greater per­son­al­iza­tion of con­tent to the indi­vid­ual based upon their sta­tis­ti­cal pref­er­ences.  Behav­ioral Tar­get­ing capa­bil­i­ties are now being offered in an auto­mated fash­ion, such as within Test&Target 1:1, which allows pre­dic­tive deliv­ery of the most engag­ing con­tent to anony­mous cus­tomer pro­files on highly traf­ficked webpages.

2)      Data is only use­ful if it’s action­able: whether we’re talk­ing ana­lyt­ics, or 3rd party cus­tomer infor­ma­tion accessed through your own data ware­house or a third party ven­dor, this data is only use­ful if you can use it to build out your user pro­files and make your seg­men­ta­tion and tar­get­ing more refined, improv­ing your accu­racy when deliv­er­ing rel­e­vant con­tent.  A strong major­ity of this data can be cus­tomized for aggre­ga­tion with the 1st party data within your test­ing tools, to bol­ster user pro­files and make them more complete.

3)      Take risks: rather than lim­it­ing your cre­ative con­tent in terms of exec­u­tives’ or mar­ket­ing team’s “gut feel­ings”, cre­ate hypothe­ses that test new engage­ment approaches and see if they click with your cus­tomers.  With test­ing, you often have the abil­ity to “fail fast”, or see what doesn’t work rel­a­tively quickly, and results don’t lie (as long as your tests are admin­is­tered cor­rectly.)  Look for a tool that allows flex­i­bil­ity and scal­a­bil­ity in terms of the quan­tity of con­tent and how you test it (it was great to hear that a lot of atten­dees felt that there is a lot of free­dom and flex­i­bil­ity with Test&Target – its open plat­form and scal­a­bil­ity allows for a larger quan­tity of vari­ables and seg­ments, vast oppor­tu­ni­ties for inte­gra­tion, as well as multi-page, onsite/offsite test­ing capabilities.)

4)      Take into account ancillary/adjacent con­tent or KPIs around your test: it can be easy to over­look the impact of our con­tent deci­sions fur­ther down the con­ver­sion fun­nel or cus­tomer expe­ri­ence. While an offer can be opti­mized and tar­geted for a par­tic­u­lar cus­tomer seg­ment, the next page or process on your web­site may be impacted neg­a­tively, and cus­tomer behav­ior should be mea­sured and tested to opti­mize around your test event.

5)      Deter­mine what not to test: don’t waste your time with mate­r­ial or loca­tions on your web­pages that don’t move the nee­dle.  Instead, set up tests (mul­ti­vari­ate tests are excep­tional for this) which show you what areas or con­tent ele­ments are not crit­i­cal in terms of user engage­ment, so you can focus on test­ing and tar­get­ing the mate­r­ial that is.  Time and bud­get savings!!!

6)      IP addresses offer pow­er­ful infor­ma­tion: an IP address can tell you a lot of infor­ma­tion about your vis­i­tor, even if their cook­ies do not.  For exam­ple, a sta­tic IP address can iden­tify that this per­son is access­ing through a com­pany account and infor­ma­tion about that com­pany.  It is very impor­tant to respect a customer’s pri­vacy, though; the data you are gath­er­ing should be eval­u­ated in terms of main­tain­ing and respect­ing a customer’s anonymity.  Good rule of thumb: keep abreast of stan­dard prac­tices and rules and reg­u­la­tions sur­round­ing pri­vacy, and how you might feel if par­tic­u­lar data were gath­ered by a com­pany you were engag­ing with.

7)      Free tri­als can be dou­bly use­ful: free tri­als offer great incen­tives to new cus­tomers to expe­ri­ence and ulti­mately pur­chase your prod­uct, but they also have the added bonus of allow­ing you to tar­get cus­tomers at par­tic­u­lar stages in the trial process.  This tar­geted con­tent can be tested to see what offers or infor­ma­tion are the most engag­ing for dif­fer­ent groups.  “Did you know you can do this?” emails can give addi­tional infor­ma­tion on over­looked fea­tures of your prod­uct, as well as sub­scrip­tion noti­fi­ca­tions, renewal info and updates as their trial is com­ing to an end.

Brooks Bell at Click Summit

My final “a-ha” moment from the event was under­scored in Brooks Bell’s open­ing remarks and described in depth by mar­ket­ing guru Seth Godin’s keynote address. Seth made allu­sions to “Mad Men”, how adver­tis­ers and mar­keters in the past would mark suc­cess by how much they could get an ad out there; the more it played, the more peo­ple saw it, whether the mate­r­ial was rel­e­vant or not.  He also spoke to how the tra­di­tional bell curve of cus­tomer con­sump­tion has now shifted out­wards, as cus­tomers’ desires and expec­ta­tions are pushed to the extremes due to the height­ened com­pe­ti­tion for atten­tion, the desire for per­son­al­ized expe­ri­ences and the viral nature of the web.   With so many con­tent choices and chan­nels, the tra­di­tional meth­ods of mar­ket­ing and adver­tis­ing are no longer ade­quate, as wit­nessed by the pre­cip­i­tous fall of the record labels when music found its way on the web.

So, what’s the solu­tion?  How can a com­pany quickly find its audi­ence, com­pete in terms of cre­ative approach and style, and deliver the best cus­tomer ser­vice across all dig­i­tal devices in the Wild West of the web?   One of the best invest­ments of time and money when rolling out a brand new cut­ting edge cam­paign is in test­ing and opti­miza­tion.  I’m thrilled to see so many com­pa­nies, involved in Click Sum­mit and other test­ing events, mak­ing so much head­way in terms of build­ing their test­ing pro­grams and lift­ing their cus­tomer sat­is­fac­tion and con­ver­sions to new heights.  The free­dom offered by “con­tent deter­mi­na­tion” through test­ing and tar­get­ing will allow them to adapt to the chang­ing trends of cus­tomer needs, as our dig­i­tal world con­tin­ues to evolve at such a rapid pace.