Where then does test­ing fit into your orga­ni­za­tion? Is it just some­thing peo­ple do? Is it a cen­tral com­po­nent to be shared, or is it just some­thing that each group does on their own. All groups face this strug­gle when they dis­cover that you can’t just make it an addi­tional duty for some­one and get good value. When you have decided that you want to build a real opti­miza­tion team within your orga­ni­za­tion, you are chal­lenged with those very ques­tions. What I want to pro­pose are a cou­ple of frame­works that I have seen work to great suc­cess, and what fun­da­men­tally makes them suc­cess­ful. It may not be pos­si­ble to move moun­tains quickly to get to these struc­tures imme­di­ately, but it is impor­tant to under­stand why they work and to think of ways to move towards those directions.

One of the core chal­lenges is that test­ing really isn’t a full on mar­ket­ing dis­ci­pline, nor an ana­lyt­ics dis­ci­pline, IT dis­ci­pline or really any­thing else that is in most groups nor­mal orga­ni­za­tion struc­ture. To most groups start­ing out, test­ing is thought of as a fea­ture done in order to prove that their cur­rent efforts are bet­ter then their prior efforts. Test­ing instead is a unique dis­ci­pline that takes parts of all of those, but is also at its best when it is show­ing the inef­fi­cien­cies in your inter­nal processes and mind­set. for any group to suc­ceed, you need to have peo­ple, align­ment, and the cor­rect mind­set, oth­er­wise it all goes to waste. When test­ing is not allowed to be a new dis­ci­pline, it suf­fers expo­nen­tially the inef­fi­cien­cies of the dis­ci­pline it is placed under.

The first thing to under­stand is that for larger orga­ni­za­tions, test­ing works best in a hub and spoke model; mean­ing that you have a cen­tral team that then works with mem­bers of the var­i­ous busi­ness units to improve their actions. While this might be the best model, it only works if you have estab­lished clear rules and the cor­rect mind­set in those other groups as well as your own.

Edu­ca­tion will always be the pri­mary role of the test­ing team.

Each of the frame­works below shows the cen­tral team, that would then work with an indi­vid­ual or team in each struc­ture that fol­lows a sim­i­lar for­mat. Cen­tral align­ment allows you to sep­a­rate the resources and to insure that you are not just adding on test­ing to exist­ing duties. This for­mat allows you the ben­e­fit of cre­at­ing a cen­tral knowl­edge base, while lever­ag­ing local knowl­edge, resources, and struc­ture to work. The accu­mu­la­tion and shar­ing of knowl­edge, and the design of your efforts to accom­plish this task is the pri­mary goal of your larger struc­ture. For this to work, test­ing can not be dic­tated only by the busi­ness unit. It must instead be a col­lab­o­ra­tive effort where both sides work together to cre­ate a con­tin­u­ous cul­ture of opti­miza­tion, one that is focused on being “wrong” over being “right”. There can never be a time where just some­one com­ing up with an “idea” is allowed to be the end all of what is tested. No idea by itself is sacred, and no one, the CEO down to your jan­i­tor, should be allowed to just throw some­thing up because they think “it will work”.

Frame­work #1:

In this first model, you see that we have a man­ager of test­ing who works through or acts as a project man­ager for exter­nal resources (IT and cre­ative). This per­son may have ana­lysts work­ing under them, but fun­da­men­tally they work with other groups in a dot­ted line cre­at­ing a cross func­tional team that tack­les test­ing. While you may not have direct full time peo­ple on the team, in this struc­ture the same peo­ple work together reg­u­larly to advance the orga­ni­za­tions opti­miza­tion efforts. You will also notice that while they may work under ana­lyt­ics, they are not ana­lyt­ics, with a sep­a­rate team han­dling those respon­si­bil­i­ties. The dis­ci­plines are dra­mat­i­cally dif­fer­ent, and there is a lot of value for bring­ing dif­fer­ent data types together, but if you have just your stan­dard ana­lyt­ics team also doing test­ing, you will never achieve any­thing close to the value that you can and should receive.

The lim­i­ta­tions of this type of struc­ture is a heav­ier need for spon­sor­ship to allow the free­dom and align the teams on cen­tral goals. You will now have per­sonal or team goals for the var­i­ous busi­ness units that may be oppo­site of the cen­tral or opti­miza­tion team goals. It is extremely easy when resources are not “owned” for those to go to projects based on polit­i­cal or pop­u­lar rea­sons and not the value they may bring to the orga­ni­za­tion. It is easy to talk about work­ing together and hav­ing access to resources, but that tends to last only as long as there is not a fire that some­one feels needs to be put out. Other lim­i­ta­tions are the con­stants pull to do other types of work, espe­cially for report pulling, as well as the need to make it clear what peo­ple are mea­sured on despite being on very dif­fer­ent teams. Despite those lim­i­ta­tions, a lot of good can come if you have clear and strong lead­er­ship, account­abil­ity, and the right peo­ple in place. This is one of the most com­mon struc­tures for mature groups, and is one that will allow some level of suc­cess and expan­sion through­out the organization.

Frame­work #2:

In this sec­ond exam­ple, you see that you have a full opti­miza­tion team, one with­out dot­ted lines and one who is inde­pen­dently part of the entire data team. Here you have tech­ni­cal and cre­ative resources, but who only tan­gen­tially part of the larger mar­ket­ing and IT teams. These resources are not directly part of their team, but the same group con­tin­u­ously work together to grow and expand the orga­ni­za­tions test­ing efforts. The ben­e­fits of this struc­ture is the abil­ity to really develop the skills of the mem­bers, the cen­tral role within the larger orga­ni­za­tion, and the abil­ity to have a sep­a­rate char­ter and to really focus on improv­ing the site as a whole, not just the smaller com­po­nents within.

This is the pre­ferred struc­tures that I have seen for larger orga­ni­za­tions. This allows for con­sis­tent resources, the abil­ity to do the right thing for the site, not just the busi­ness unit, and an inde­pen­dent role for ana­lyt­ics and just mar­ket­ing. It doesn’t cre­ate con­fu­sion for the team to do things that help only one group or per­son, and it allows for a clear line between test­ing and any other group. The goal is to work together to improve, not for teams to fight over who gets credit. The lim­i­ta­tions are the need to con­stantly be work­ing with and edu­cat­ing the var­i­ous busi­ness units, and the com­pli­ca­tions of own­ing the impact for the team’s actions.

In order for either frame­work to func­tion prop­erly, the exec­u­tive spon­sor must take an active role in keep­ing peo­ple aligned and account­able for site goals, not just per­sonal goals. You will never be able to func­tion if you cant first deal with petty infight­ing and a lack of account­abil­ity towards a com­mon cen­tral goal. If you do not have that, then don’t wait for it to hap­pen and instead seek out spon­sor­ship to make it hap­pen. You don’t need or even want higher level exec­u­tives to be deal­ing with the day to day oper­a­tions, but you need an umbrella by which to sep­a­rate the dis­ci­plines, align the resources, and make sure that the sys­tem itself is being used to its high­est func­tion. As much as we may want to pre­tend we live in a world where every­one works together towards a com­mon goal, it is rare that this is the case. The exec­u­tive sponsor’s pri­mary respon­si­bil­ity is to raise the level of dis­course away from petty indi­vid­ual goals and to hold peo­ple account­able for actions that make every­one better.

These are but two of the more com­mon exam­ples I have seen for suc­cess­ful pro­grams. In all cases, the real suc­cess is far more about devel­op­ing skills, chal­leng­ing ideas, get­ting buy-in and account­abil­ity, and more than any­thing treat­ing opti­miza­tion as an inde­pen­dent func­tion that never starts and never ends. The team is meant to be the best friend of all busi­ness units, not their worst enemy. It is meant to show them the effi­ciency of their efforts and to make sure that what they think mat­ters really does. If you are just going to lever­age test­ing to just push the same ideas, to prove value for other tools, or to make some­one look good then you have no chance of build­ing a world class orga­ni­za­tion. This means that you have to fight the bat­tles to get past the ini­tial resis­tance and edu­cate peo­ple as to why their ideas won’t work and why you must chal­lenge all ideas in order to under­stand the value, not just directly but rel­a­tive to other courses of action.

There are no magic bul­lets to make your orga­ni­za­tion a great opti­miza­tion orga­ni­za­tion. No mat­ter what struc­ture and path you choose to go down, it takes time and a lot of hard work to get peo­ple to under­stand how to make it work. The hard­est task is always going to be get­ting past poor mis­con­cep­tions and petty inter­nal bat­tles for con­trol and who gets to claim suc­cess. Once you have got­ten past those points, align­ing in a way that insures suc­cess is the next step on the path to a great pro­gram. Pro­grams fail when they stop fight­ing nec­es­sary fights and just go with the flow. If you want to make your orga­ni­za­tion the best it can be how­ever, you can never stop improv­ing and you can never stop try­ing to get peo­ple to align in the best ways possible.