One of the largest dif­fer­en­tia­tors between start­ing, mid­dling, and mature suc­cess­ful pro­grams is the speed at which they can exe­cute. The more you move, and the more effi­ciently that you move, the more value you get and the more you learn. You can’t get any­thing from run­ning 100 bad tests, but if you have even a lit­tle dis­ci­pline in your test design, then the faster you move, and the faster you can act on that data, then the more value you will receive and the greater your chance to learn and grow. If it takes you 2 months to just answer a sim­ple busi­ness ques­tion, how can you expect to keep up with the indus­try? Speed is the dif­fer­en­tia­tor in the mag­ni­tudes of value you receive once you have acted on and enabled the other dis­ci­plines of opti­miza­tion. You have to move fast to insure that you are get­ting the most from your program.

Deal­ing with hun­dreds of dif­fer­ent test­ing groups, you see a very dif­fer­ent approach to what is accept­able as far as the speed of get­ting a test live, from con­cept to exe­cu­tion. Some groups, it can take months to get a test live, while oth­ers are able to exe­cute in mere hours or min­utes. No mat­ter what you think is nor­mal, the real­ity is that every group needs to work every day to get faster and more effi­cient with their pro­gram. The sad truth is that most groups treat act­ing fast as a fail­ure of the sys­tem, let­ting the weight of their own size work against them to the point that it feels like you are sysy­phus when­ever you try to get dif­fer­ent teams to work together and an expe­dited man­ner. If you want to really get value from your pro­gram, you have to cre­ate the infra­struc­ture before you try to run, in order to max­i­mize speed and make sure that you are exe­cut­ing at a high enough level for success.

One of the most com­mon things I talk about with clients is that noth­ing will show your orga­ni­za­tional inef­fi­cien­cies more than try­ing to act quickly. Whether it is an out of touch IT group, a lack of train­ing, a need to get approval for every change, a slow unre­spon­sive design team, or a thou­sand other bar­ri­ers, every group has a large num­ber of road­blocks that stop them from mov­ing quickly. But this doesn’t mean that you can just accept these bar­ri­ers as being sta­tus quo; you have to max­i­mize each effort and make sure that you con­ver­sa­tions and ener­gies are spent on improv­ing the infra­struc­ture of your test­ing pro­gram, and not just on run­ning any one test.

There are many dif­fer­ent ways to orga­nize in order to suc­ceed, whether it is a L3PS model, a hub and spoke model, or many oth­ers, each group needs to fig­ure out what works best for them. It is the act of find­ing and cre­at­ing these effi­cien­cies that make pro­grams great, not that great pro­grams have these items avail­able to them. Great pro­grams have com­mon items in place and aligned in order to facil­i­tate speed and to make sure that bar­ri­ers and even­tual road bumps are dealt with as quickly as possible:

1) Spon­sor­ship – All pro­grams that really achieve lev­els of suc­cess have some­one higher up who is tak­ing an active role in the pro­gram and mak­ing sure that things are com­mu­ni­cated prop­erly in order to facil­i­tate the var­i­ous groups work­ing together. It is nearly impos­si­ble to really grow your pro­gram with­out some­one greas­ing the gears of your organization.

2) Con­sis­tent resources – Test­ing can’t be a project type of basis, it has to be a core team who are learn­ing and grow­ing their skills in order to become bet­ter at what they do. There are many dif­fer­ent ways to allo­cate these resources, but suc­cess­ful teams start with ded­i­cated time and usu­ally move to large teams of peo­ple who do noth­ing but work on opti­miza­tion efforts.

3) Tech­ni­cal infra­struc­ture – You need to make sure that your site can prop­erly track your sin­gle suc­cess met­ric, and that you can get a large major­ity of tests live with no tech­ni­cal involve­ment. This means that you need both global pieces of infra­struc­ture, but also prop­erly set up the pieces needed to tackle the key pages on your site.

4) Knowl­edge Trans­fer / Knowl­edge repos­i­tory – There is a large amount of edu­ca­tion needed in order to really move for­ward with your pro­gram. The pro­grams that have the most trou­ble are the ones that try to run opti­miza­tion like they do ana­lyt­ics, or visa-versa. You need to train peo­ple on how best to think about test­ing, how best to think about data, and how best to answer their key ques­tions. At the same time, you need some sort of liv­ing knowl­edge base that accu­mu­lates your knowl­edge and is avail­able for those that want to become more involved. Make sure that you are sep­a­rat­ing the lessons learned from the indi­vid­ual test results.

5) Pro­gram built around effi­ciency – So often where pro­grams fal­ter and slow­down is that they try to only test mas­sive tests that are built to do the one idea that some­one higher up wants to try. While you shouldn’t shun these tests all together, you will find that almost always those tests do not get the results that you expect and are not built for learn­ing. Instead test with the resources you have and try to max­i­mize the amount of learn­ing and recipes that you are get­ting from each test. The goal is not to run as many tests as pos­si­ble, as 50 awful tests will not be as valu­able as one well run test, but instead focus on max­i­miz­ing returns and keep­ing a steady cadence of test­ing to make sure you are always mov­ing forward.

6) Avoid Overly Tech­ni­cal pro­grams – This is the real killer of most groups. You have all the energy that you need to be suc­cess­ful, but all of it is wasted because you are try­ing to solve the wrong prob­lem. They try to over lever­age their IT teams in order to facil­i­tate test­ing, instead of build­ing out the proper infra­struc­ture through­out. For the most part (maybe 8 out of 10 tests) you should be able to train your mother in order to run the test. CSS and extremely sim­ple repeat­able JavaScript func­tions are your friend. If you are rely­ing on IT to write a long com­pli­cated JavaScript offer, or your team does not have the skill to jump in and get a test up on your site in less than an hour, you need to take a deep look into how you are going about your test­ing. It may seem like get­ting a test live that quickly is incon­ceiv­able with your cur­rent orga­ni­za­tion, but that is why it is so impor­tant that you are mov­ing for­ward to fix those prob­lems so that one day, it seems like just another aver­age task.

There are many other pit­falls that can hurt the infra­struc­ture of a pro­gram, from over­done QA processes, too many sign off points, block­ing or lack of involve­ment from senior lead­er­ship, unwill­ing­ness to lis­ten to num­bers or feed­back from your UX or brand­ing teams, to a thou­sand oth­ers. The real key is that you can­not ignore them, nor can you think they get solved with some magic elixir. It takes time to move your pro­gram for­ward, and there are always pot­holes and road bumps on the way. All pro­grams go through an evo­lu­tion, but at the end of the day, it is your will­ing­ness to move for­ward and try to over­come those bar­ri­ers that will deter­mine your success.

You have to want to move fast in order to do the things nec­es­sary to accom­plish it. You have to have the need for speed if you ever want to achieve it. It is doable, and there are lots of resources out there to help, but in the end, it all starts with your will­ing­ness to act.


Typo in the first paragraph. The more "efficiently" you move. Adverb.