I’ve been strug­gling with this ques­tion a lot lately as I talk to com­pa­nies try­ing to fig­ure out how to orga­nize inter­nally for test­ing. Some com­pa­nies want to own test­ing and start run­ning from day one, while oth­ers want to ramp up slowly with a few resources work­ing part-time on it. Some are con­tent to run just a few tests a year through IT, and still oth­ers want to out­source the whole oper­a­tion to a ven­dor or agency.

Should Test­ing Be Outsourced?

That last seg­ment is the one I grap­ple with the most. I don’t think the ques­tion is whether you can out­source test­ing and opti­miza­tion, but whether you should. For any com­pany seri­ous about their online chan­nel, I believe that their mar­keters should already be think­ing every day about how to improve the site and all acqui­si­tion efforts lead­ing to the site. Those ideas should be the fuel pow­er­ing the opti­miza­tion engine, so what would make a com­pany want to turn that over to some­body else to man­age? Does that mean there’s no fuel in-house, or that the fuel is bad grade? If so, that seems to point to a larger prob­lem for the com­pany than just whether test­ing can be outsourced.

None of this is to say that I don’t believe in the value of ven­dors, con­sul­tants and agen­cies con­tribut­ing to your opti­miza­tion efforts. I’ve seen many cases where 3rd-parties have helped pro­duce fan­tas­tic results for their clients. They bring their exten­sive expe­ri­ence with other com­pa­nies and fresh insights to help mar­keters think about their deci­sions from a test­ing per­spec­tive. But I’ve also seen com­pa­nies slash those par­ties off the bud­get with­out think­ing twice because nobody at the com­pany actu­ally cared about test­ing. Here are the main rea­sons why I don’t think test­ing can be fully outsourced:

1) No Test­ing Cul­ture - There is a huge unquan­tifi­able ben­e­fit to hav­ing a test­ing cul­ture instilled at a com­pany. It means that peo­ple are more thought­ful and less care­less because they are forced to make data-driven deci­sions. It also means that they feel more empow­ered to have ideas because they will get to see them in action on the site. An employee who sees that he or she has made hun­dreds of thou­sands of dol­lars for their com­pany through a great idea that got tested on the site will cer­tainly be moti­vated and moti­vate oth­ers to pro­duce more stel­lar results!

2) Less Learn­ings - If a 3rd-party is man­ag­ing a company’s test­ing efforts, then all the learn­ings that come out of that test­ing don’t usu­ally make it back out to the rest of team, espe­cially if those tests pro­duce neg­a­tive ROI. They are still incred­i­bly valu­able though, as they can inform how designs should be planned mov­ing for­ward. Imag­ine if an agency tests a big hero image against sev­eral small prod­uct images and finds one much more com­pelling than the other. That could change how other mar­ket­ing efforts such as site redesigns and direct mail pieces are designed, but only if that infor­ma­tion is circulated.

3) Eas­ily Removed - As I said before, divorc­ing opti­miza­tion from the com­pany makes it eas­ily remov­able as a nice-to-have instead of a must-have. This issue is not iso­lated to just out­sourc­ing the oper­a­tion though. I’ve also seen it hap­pen many times when there is only one per­son at a com­pany man­ag­ing test­ing. That cham­pion might be gen­er­at­ing many test­ing suc­cesses, but once they leave, nobody knows how to run it or wants to take on extra work. Beware of tribal knowl­edge! The most suc­cess­ful test­ing com­pa­nies main­tain a knowl­edge­base so they can eas­ily ref­er­ence pre­vi­ous learn­ings and not be back at square 1 if a sub­ject mat­ter expert leaves.

4) Less Col­lab­o­ra­tion - Test­ing forces peo­ple to talk to each other. Dis­con­nected land­ing page expe­ri­ences are often a result of dis­con­nected mar­keters. An acqui­si­tion mar­keter is incented to get peo­ple to the land­ing page. A site mar­keter is incented to drive peo­ple to con­ver­sion. There needs to be some com­mon objec­tive that forces them to talk to each other. I’ve also seen test­ing help align mar­ket­ing and IT. Instead of ask­ing IT to sim­ply put another tag on the page, mar­ket­ing should explain what that tag’s pur­pose is and that it is a 1-time effort that will actu­ally make their lives eas­ier mov­ing for­ward. That helps IT under­stand the value of imple­men­ta­tion and pro­vides con­text for why it’s dif­fer­ent from the other tags on the page.

Again, there are amaz­ing resources out there to help you start test­ing suc­cess­fully. This post is not meant to detract from the imme­di­ate value they offer to jump­start your opti­miza­tion pro­gram. Take advan­tage of the best prac­tices learned by oth­ers and use the valu­able strat­egy, advice and cre­ativ­ity they can pro­vide. But please, con­sider the ben­e­fits of using 3rd-parties to get your test­ing legs under you so that you can run on your own even­tu­ally! Spread the love and bring peo­ple into the opti­miza­tion process. It’s more fun to have a run­ning part­ner than to watch other peo­ple run­ning laps around you.

Pho­tos cred­its:
http://​www​.flickr​.com/​p​h​o​t​o​s​/​c​i​n​d​y​4​7​4​5​2​/​1​4​4​6​5​5​9​93/
http://​www​.flickr​.com/​p​h​o​t​o​s​/​a​v​a​l​o​n​s​t​a​r​/​1​6​3​5​0​2​7​44/