In “Your Guide to Choos­ing an Online Test­ing and Opti­miza­tion Solu­tion,” Brian Hawkins, a part­ner at Web Ana­lyt­ics Demys­ti­fied, out­lines the core con­sid­er­a­tions and best prac­tices in the selec­tion and inte­gra­tion of online test­ing and opti­miza­tion solu­tions. He even pro­vides a step-by-step require­ments check­list for iden­ti­fy­ing a solu­tion and partner—what’s impor­tant to the orga­ni­za­tion, what’s sup­ported by the ven­dor, and the effort required—to help mar­keters and stake­hold­ers deter­mine what’s core to build­ing and apply­ing their ini­tial cam­paigns. How­ever, ulti­mately, Hawkins states that the intro­duc­tion of these sys­tems must be wholly informed by a host of orga­ni­za­tional con­sid­er­a­tions, capa­bil­i­ties, exist­ing sys­tems, and cultures.

Where do you start? Accord­ing to Hawkins, it’s with a clear pro­nounce­ment and align­ment of orga­ni­za­tional goals, a clear path to inte­gra­tion, and a thor­ough assess­ment of prospec­tive partners.

Map Your Goals—And Your Cur­rent Abil­ity to Meet Them

You likely already have some sense of the desired out­comes of your test­ing and opti­miza­tion efforts. Are you an e-commerce site look­ing to increase con­ver­sion, raise the aver­age order value, or upsell cur­rent con­sumers? Want to deliver tar­geted con­tent to site vis­i­tors based on the types of finan­cial ser­vices they’re search­ing for? Increase video engage­ment, arti­cle con­sump­tion, or reg­is­tra­tions? Some­thing else?

Opti­miza­tion ser­vice providers can help you con­nect the dots when test­ing implications—what sys­tems exist and how they will help you deepen and mea­sure these goals—but it’s also crit­i­cal to look inter­nally. Who will man­age the process from your end? Does your orga­ni­za­tion have the know-how, or is train­ing required? How will you inte­grate the ana­lyt­ics that emerge into exist­ing data sets and processes? And, beyond Hawkins’ con­sid­er­a­tions, is your orga­ni­za­tion cul­tur­ally and sys­tem­at­i­cally mature enough and ready for these shifts? How much of an opti­miza­tion orga­ni­za­tion are you—and will your efforts be supported?

Deter­mine the “What” and the “How”

For those con­sid­er­ing online test­ing and opti­miza­tion for the first time, Hawkins pro­vides a sim­ple overview of the types of tests and test­ing chan­nels for ini­tial and ongo­ing efforts. Short, com­pelling case stud­ies illus­trate their respec­tive benefits.

Test Types Test­ing Channels
A/B tests Email
Mul­ti­vari­ate tests Mobile
Auto-optimizing Dis­play
Mutual exclu­siv­ity Inter­nal search and recommendations

As Hawkins explains, “tar­get­ing or seg­ment­ing sub­sets of vis­i­tors for analy­sis is the most effi­cient way to get as much value from your tests and test­ing efforts,” steer­ing mar­keters toward test­ing ser­vice providers who offer tar­geted test­ing and seg­men­ta­tion analy­sis uni­ver­sally. This type of sys­tem inher­ently pro­vides bet­ter results sooner while deliv­er­ing in-depth insights into poten­tial per­son­al­iza­tion points. The most basic tar­get­ing starts with envi­ron­men­tal variables—think browsers, oper­at­ing sys­tems, and gen­eral tem­po­ral infor­ma­tion. This is fol­lowed by geo­graphic data, real-time behav­ioral tar­get­ing, and even the abil­ity to inte­grate offline or inter­nal cus­tomer rela­tion­ship man­age­ment (CRM) data.

So how do you weigh these con­sid­er­a­tions and choose the opti­mal test­ing solu­tion? Think about what’s involved in inte­grat­ing poten­tial data sets into your exist­ing mar­ket­ing sys­tems. What do you need to sup­port and main­tain the inte­gra­tion? What will you actu­ally tap into, and what will your inter­nal and exter­nal stake­hold­ers need to see to deem the test­ing and sub­se­quent opti­miza­tion a success?

Eval­u­ate the ven­dors through these lenses and don’t get side­tracked by all of the “nice to haves” if your “need to haves” will get the job done. Then, of course, be sure you have the tools and the resources—AKA the mar­ket­ing manpower—to report, seg­ment, ana­lyze, and effec­tively uti­lize the results.

Think Col­lab­o­ra­tively

It’s sim­ple: chang­ing your site for the pur­pose of test­ing and opti­miza­tion can be a daunt­ing task for mar­keters and stake­hold­ers alike. It’s not only the change involved, Hawkins writes, but also the seem­ingly for­eign con­cepts such as HTML, JavaScript, style sheets, and Doc­u­ment Object Model manip­u­la­tion, that can over­whelm and off-road even the most ded­i­cated team.

To avoid these pit­falls, make the work­flow col­lab­o­ra­tive, clean, and as seam­less as pos­si­ble from the begin­ning. By select­ing the right tools and the right third-party ven­dor, you’ll have a com­pre­hen­sive roadmap in place that makes the frame­work and objec­tives clear to every­one involved. This col­lab­o­ra­tion must carry through to the imple­men­ta­tion stages of the process. Eval­u­a­tion is crit­i­cal, but imple­men­ta­tion “can make or break an organization’s use of a test­ing solu­tion.” Make sure those respon­si­ble for the exe­cu­tion and future inte­gra­tion are involved from the ear­li­est stages, and incor­po­rate their feed­back in the assess­ment and plan­ning processes. What it costs in startup time, it will more than make up for postpurchase.

The Check­list

Hawkins con­cludes with his thor­ough yet user-friendly require­ments check­list. The process, he writes, starts with the inter­nal audit and review period—determining what’s impor­tant, get­ting stake­hold­ers on board, and iden­ti­fy­ing an imple­men­ta­tion path—and then assess­ing solu­tion providers. His sam­ple check­list splits the review into three buckets:

  • Impor­tance: What are the must-haves ver­sus the nice-to-haves?
  • Sup­ported by ven­dor: What does the poten­tial part­ner offer, and what’s core, optional, and sim­ply not available?
  • Imple­men­ta­tion effort: On a high/medium/low scale, how much effort will go into the imple­men­ta­tion process?

From here, mar­keters can make their way through a pre­pop­u­lated list of tests, tar­get­ing capa­bil­i­ties, analy­sis capa­bil­i­ties, seg­men­ta­tion, automa­tion options, con­sid­er­a­tions, and more, assess­ing each sub­cat­e­gory with regard to their requirements.

For test­ing novices, it’s crit­i­cal to iden­tify a tool that not only checks these boxes but also keeps an eye on setup and exe­cu­tional best prac­tices and, more­over, pro­vides ample sup­port through­out. Hawkins calls out eight essen­tial capa­bil­i­ties for those new to the process:

  • Work­flow and col­lab­o­ra­tion best prac­tices enablement
  • Range of test type capabilities
  • Com­pre­hen­sive report­ing interface
  • Seg­men­ta­tion
  • Tar­get­ing
  • Real-time report­ing
  • Rich and robust tar­get­ing capabilities
  • Cus­tomiz­able graph­i­cal analysis

But not only new and rel­a­tively new orga­ni­za­tions need to take a step back and eval­u­ate their posi­tion. All orga­ni­za­tions should include the fol­low­ing in their assessments:

  • Inte­gra­tion capa­bil­i­ties, espe­cially with the ana­lyt­ics platform
  • Automa­tion of test content
  • Appli­ca­tion pro­gram­ming inter­faces (APIs)
  • Mutual exclu­siv­ity
  • Abil­ity to exe­cute tests across channels

No mat­ter the expe­ri­ence, com­fort level, or objec­tives, though, you also need to con­sider whether yours is an opti­miza­tion orga­ni­za­tion. The suc­cess of a test­ing and opti­miza­tion ini­tia­tive can only be as suc­cess­ful as the organization’s matu­rity, under­stand­ing, and avail­able sup­port. Where your orga­ni­za­tion stands dur­ing the plan­ning and eval­u­a­tion stages will dic­tate its success—don’t wait to “catch up” once the process begins.

Over­all, Hawkins’ piece gives a com­pre­hen­sive, digestible look at a com­plex process. Brands, agen­cies, schools, and con­tent providers must deliver the most rel­e­vant, most tar­geted infor­ma­tion, rec­om­men­da­tions, and solu­tions to their users, mak­ing online tar­get mar­ket­ing crit­i­cal to vir­tu­ally any orga­ni­za­tion. No mat­ter how you mea­sure ROI—through rev­enue, engage­ment, lead gen­er­a­tion, added con­tent con­sump­tion, or loyalty—the proper test­ing and opti­miza­tion solu­tions can deliver in spades.