One of the chal­lenges that just about any group new to test­ing has is try­ing to get buy-in and sup­port from var­i­ous other groups, usu­ally with strong oppo­si­tion from UX and brand­ing teams, but also from just about any other group that inter­acts with the site. The largest rea­son for this is Loss Aver­sion, or “the disu­til­ity of giv­ing up an object is greater than the util­ity asso­ci­ated with acquir­ing it.” To put sim­ply, we get too caught up in what we lose that we miss what we gain. Peo­ple fear the lack of con­trol that open­ing up their ideas to analy­sis brings, and with that fear comes some of the biggest hur­dles that pro­grams need to overcome.

How many times have you had to try and get sign-off from a new group only to have them push back or say that doesn’t “feel right”. How many times have you wanted to bring test­ing to new peo­ple only to have them shy away the first moment what they were sure would win, loses? The irony of this is that some of the most ardent sup­port­ers of test­ing in mature pro­grams are the very peo­ple who were chal­leng­ing the pro­grams at the very start. Any­one that has built their rep­u­ta­tion on their artis­tic tal­ent, or by declar­ing “here is how we are going to do this” has to over­come their fear of los­ing that con­trol in order to gain the power and effi­ciency that test­ing can bring.

How many times have you gone into a con­fer­ence room with peo­ple to brain storm? Or gone to an off­site just to come back with a long list of who you are going to tar­get? Or some writ­ten rules on color guide­lines or the like. There is a lot of hard work that goes into those efforts, but the prob­lem is that they are filled with assump­tions and com­pro­mise. They are designed to either make every­one feel like they con­tributed, or to make the HiPPO happy. What test­ing does in the best cases is stop that cycle, so that you are no longer try­ing to fig­ure out the one way to make things work, but instead have open dis­cus­sions on what is fea­si­ble. It democ­ra­tizes ideas and is agnos­tic as to the value of them. The entire point is to be able to mea­sure the value of each idea against the oth­ers and fig­ure out the best one to go with. There is a great deal of fear, what if you are wrong? Does this make me look bad? My way has always been “right”, and so on. Peo­ple have built empires on this fal­lacy, often with no one hold­ing them account­able to the actual value of those ideas.

So how do you fight this? The first thing you must do is to get every­one to agree on what you are try­ing to accom­plish. This has to be a sin­gle thing that you can mea­sure and that is uni­ver­sal across the site (this is not about group A ver­sus group B, this is about every­one work­ing together to improve the site). I have seen so many pro­grams strug­gle, get no value, or end up in polit­i­cal quag­mires sim­ply because they refused this first step. It can be very dif­fi­cult or very easy, but at the end of the day, the sin­gle great­est deter­mi­na­tion of future suc­cess for opti­miza­tion is agree­ment on what you are try­ing to accomplish.

Once you have that mea­sure, then it becomes about tak­ing those ideas and mea­sur­ing them against that goal. Remem­ber you want to chal­lenge the com­mon the­ory, mean­ing you should include null assump­tions and things that con­tra­dict what you think will win. It serves you no good if you align every­one on find­ing an answer to a ques­tion if that ques­tion is irrel­e­vant or sub opti­mal. What is funny is that there is almost an inverse cor­re­la­tion between what peo­ple think will win, and what does win. Get opin­ions from mul­ti­ple sources, espe­cially from one or two from out­side the group that has owned that con­cept or por­tion of the site.

Step three is sim­ply test. But at the end of the test, don’t worry if you were wrong, and don’t make it about you ver­sus me. This is about every­one work­ing together to find what works. If every­one is work­ing for the same goal, then it is easy for every­one to get the credit and for every­one to align. The entire point was that the bet­ter the ideas feed into the sys­tem, the more diverse and risky those ideas are, the more you learn and the bet­ter the results you will have. You have to stop wor­ry­ing about who was right and instead encour­age peo­ple to be wrong. Being wrong gives you so much more than being right, and it gives you new learn­ing to share and bring value to other parts of the site.

If you do this enough, you will get to the point where you no longer need to have those large con­fer­ences or off sites, you just need to com­pile the fea­si­ble options and move for­ward with let­ting the test tell you where to go. It becomes less about try­ing to fit the square peg into the round hole (or in some cases, into no hole) and more about align­ing to move for­ward with what you learn. You will not end up at the feared 48 shades of blue axiom, but instead you will end up where you treat all fea­si­ble ideas as valu­able. It frees up your UX and cre­ative teams to try new things and to not worry about upset­ting their supe­ri­ors. It allows them the flex­i­bil­ity and the abil­ity to be “wrong”.

Every­one is fear­ful of the unknown and the risk of giv­ing some­thing up. What is impor­tant is to share the chal­lenge and the reward and to make it about adding value to what they were already doing, and to not blame any­one when they were wrong. Test­ing an idea is not about the loss of con­trol, it is about help­ing to make it as suc­cess­ful as pos­si­ble. This can’t be about you ver­sus them, or my idea ver­sus yours, in order to suc­ceed you need every­one to work on achiev­ing the same goal. Any sys­tem is only as good as its input, and any input with­out a proper sys­tem to facil­i­tate it will always lack value in the end. Encour­age new ideas, encour­age trial and error, as long as you have a sys­tem in place to mit­i­gate loss, you have so much more you can gain from learn­ing a new path or stop­ping a bad prac­tice on your site.


Clear and concise, very helpful information here Andrew, thanks. It's important to constantly remind yourself of the goal you're trying to achieve and check your ego and assumptions at the door.