Return on invest­ment, or ROI, is one of the most crit­i­cal yet chal­leng­ing met­rics that can be mea­sured dur­ing an opti­miza­tion cam­paign. ROI not only val­i­dates the results of your test­ing, but it also proves the value of the invest­ment in the opti­miza­tion pro­gram to the stake­hold­ers in the busi­ness. How­ever, it can often be chal­leng­ing to cal­cu­late ROI based on test­ing activ­i­ties, and we often fall vic­tim to “leav­ing money on the table” by not tak­ing into account all of the forms of ROI that opti­miza­tion produces.

Although ROI is most often cal­cu­lated as a finan­cial return on a bud­getary invest­ment made by the busi­ness, sev­eral other fac­tors play into the over­all eco­nomic ben­e­fit of an opti­miza­tion pro­gram. To truly under­stand the impact of your pro­gram on your company’s bot­tom line, it is impor­tant to take into account all of the ben­e­fits that result from both the effi­ciency and the effec­tive­ness of the pro­gram and its increase in cus­tomer engage­ment, con­ver­sion, and loyalty.

A cost-benefit analy­sis can not only pro­vide a valu­able way to pri­or­i­tize your test­ing efforts (i.e., rat­ing test ideas based on what will take the short­est amount of time to exe­cute with the great­est return), it can also help uncover ROI in terms of cost sav­ings rel­a­tive to employee time.  For instance, use of prod­ucts such as Adobe Tar­get reduce the amount of time it takes to go from test design to exe­cu­tion to analy­sis, reduc­ing the invest­ment needed to scale and sup­port your test­ing efforts. Being able to fil­ter and drill down deeply into the results by key seg­ments and suc­cess met­rics also helps define the con­tent that res­onates more quickly with con­sumers, sav­ing cre­ative resources and time. Not only are these hours/expense sav­ings impor­tant to eval­u­ate and add as ROI to your report­ing, but they also have pos­i­tive results in terms of employ­ees’ abil­ity to spend more time on other projects, or on improv­ing the pro­gram as a whole.

Another source of ROI gen­er­ated from opti­miza­tion is sav­ings in terms of cus­tomer ser­vice, call cen­ters, and ana­log processes rel­a­tive to the online busi­ness.  For exam­ple, prop­erly designed tests result­ing in improved cus­tomer expe­ri­ences means less time spent by a call cen­ter or sup­port team respond­ing to cus­tomers’ needs (or more accu­rately, com­plaints and con­fu­sion). Smoother inter­faces mean hap­pier users, and often result in increased sales. These folks are now spend­ing less time call­ing sup­port to fig­ure out bugs in the sys­tem or to clar­ify con­tent on poorly designed Web and mobile expe­ri­ences and more time con­sum­ing your con­tent, engag­ing with your brand and expe­ri­ences, and buy­ing your prod­ucts. This results in a win-win for the com­pany, dri­ving rev­enue while reduc­ing over­head costs.

A final way to look at ROI is in terms of risk mit­i­ga­tion. A new web­site, mobile site, or mobile app is a risky and heavy invest­ment.  Risk mit­i­ga­tion is of enor­mous inter­est to exec­u­tives because it buffers against poten­tially bad deci­sions, ensur­ing that the poten­tial for lost con­ver­sion is kept to an accept­able level. In fact, this can be used as an impor­tant learn­ing tool when you’re edu­cat­ing exec­u­tives on why they should trust the data above instinct. When a test result proves that one of their ini­tia­tives may have needed adjust­ment or work to opti­mize the cus­tomer expe­ri­ence, show­ing the poten­tial loss the exec­u­tive or busi­ness might have seen helps to prove why test­ing your assump­tions, how­ever edu­cated, is impor­tant and prag­matic.  As the opti­miza­tion pro­gram grows, the busi­ness matures and becomes more trust­ing of the test results; opti­miza­tion becomes an embed­ded part of the cul­ture and helps to drive the busi­ness strat­egy by jus­ti­fy­ing dif­fer­ent approaches first, in a smaller sam­ple, before they are pushed to vis­i­tors as a whole. This con­fi­dence grows as risk mit­i­ga­tion is mea­sured and busi­nesses can know for cer­tain that the con­tent they are putting out on the Web is the most effec­tive con­tent for their cur­rent goals. This goes a long way in grow­ing what we call a cul­ture of opti­miza­tion within the busi­ness, where all peo­ple and processes are at least in part dri­ven by the results of optimization.

So why then is ROI such an elu­sive tar­get? One rea­son is that many providers of opti­miza­tion soft­ware and ser­vices are afraid to expose ROI within their tools. In a sense, ROI is the final deter­mi­na­tion of the suc­cess of a pro­gram, and the soft­ware it runs on, and expo­sure of poten­tial short­com­ings in the soft­ware is a risk for the providers.

Another rea­son many pro­grams don’t include con­crete ROI esti­mates is that they do not employ valid method­olo­gies for cal­cu­lat­ing ROI. Many times ROI is a based on a series of pro­jec­tions and assump­tions, and not cal­cu­lated in a more sci­en­tific man­ner; there­fore, the unre­li­a­bil­ity of the pro­jec­tions pre­vents devel­op­ers from includ­ing the func­tion­al­ity within the software.

With Adobe Tar­get, we seek to expose the ROI met­ric with more rigor than other pro­grams. We allow for the design of con­crete met­rics within the test design that can then be assigned a cer­tain value and mon­i­tored through­out the test­ing process. Users can then go back and see what their ROI was using their cho­sen con­tent as well as what their poten­tial ROI would have been if they had gone with the best-performing con­tent as defined by the tests. This cre­ates a process around ROI that is tan­gi­ble and mea­sur­able, build­ing con­fi­dence in the tools and the results, and matur­ing the opti­miza­tion pro­gram and the busi­ness as a whole.

Accu­rate and reli­able ROI met­rics are, in a sense, the holy grail of test opti­miza­tion. The abil­ity of prod­ucts like Adobe Tar­get to enable a trans­par­ent assess­ment of ROI results in a more effi­cient and prof­itable busi­ness. With new inter­faces and improved report­ing, the results of your ROI analy­sis can be shared more eas­ily with your stake­hold­ers, fur­ther pro­mot­ing the ben­e­fits of your opti­miza­tion efforts and increas­ing your own ROI for the time spent design­ing and imple­ment­ing your tests.