If you think about the trin­ity of key ele­ments to a suc­cess­ful opti­miza­tion orga­ni­za­tion, the one that really deter­mines the abil­ity to drive the oth­ers are the peo­ple. It’s not enough to get peo­ple that are skilled or know the tool, those can be trained; what is most impor­tant are to find peo­ple that will and can chal­lenge the sta­tus quo and help change your orga­ni­za­tion for the bet­ter. You need peo­ple whose mis­sion in life is to find bet­ter ways to do things and who do not accept the sta­tus quo. If you keep doing what you were doing, how do you expect to get better?

Your peo­ple and the roles they fill, are what will allow you to change your orga­ni­za­tions view of test­ing and orga­nize in effi­cient ways to accom­plish tasks. So what peo­ple do you need? What roles do you need to fill and how do you know that you are doing the right things in those roles? There are com­mon key roles within each test­ing pro­gram. All these roles might be filled by one per­son or they may be filled by 5 peo­ple each. It is not the num­ber of peo­ple, but how they think and act that is most impor­tant. What mat­ters most is the abil­ity to put the right hat on when needed and to think and act in a way to be suc­cess­ful. Suc­cess here is defined as mak­ing the com­pany the most money in the most effi­cient way pos­si­ble. If your goal is not to assist your com­pany in mak­ing money over mak­ing your­self look good, then please ignore the suc­cess and fail­ure part of this post.

If you want to suc­ceed, you need each of these roles to be iden­ti­fied, lever­aged cor­rectly, and to work together. Hav­ing just one bad apple, or let­ting some­one turn into a vil­lain, in your group will guar­an­tee that you are lim­it­ing your results.

Pro­gram Sponsor –

Role – This is the per­son whose neck is on the line to make things work. They are there to align teams and to get every­one work­ing towards a com­mon goal. In order to accom­plish this, your spon­sor needs both the will­ing­ness and power get change done in your orga­ni­za­tion. They do not need to be involved in every day to day activ­ity, but when it comes time to make sure that other groups are not upset by chal­leng­ing their immi­nent domain and to make sure that there is not infight­ing, these need to be the real lead­ers to get that done. They also help make sure the other roles are doing their job and mak­ing sure that the pro­gram is not being used to just to make one per­son or group look good. Their last duty is to make sure that things don’t just work right one time, but over the long haul, and that the pro­gram is assist­ing the entire orga­ni­za­tion to change how it views and accepts testing.

You will be suc­cess­ful if – You take the time to under­stand that dis­ci­plines of test­ing and how to make it work in your orga­ni­za­tion. You make sure peo­ple are held account­able for their actions and for deliv­er­ing results, and make sure that there is con­stant effi­cient effort with all parts of the group. You stop peo­ple from just try­ing to make sto­ries up about how “right” they are and instead actively try to get peo­ple to appre­ci­ate being wrong. You make it clear that you and the team are not there to be yes men to other groups, but that by work­ing together that you can all work towards a com­mon cen­tral goal.

You will fail if – You don’t get involved or get too involved. There is a fine line to set­ting the stage and allow­ing oth­ers to do what they need. You will also not suc­ceed if you allow the pro­gram to cower to other pow­er­ful people’s whims or do not deal with the need for IT, design, ana­lyt­ics, and mar­ket­ing to work together towards a sin­gle com­mon site goal. You will also fail if you are more inter­ested in build­ing your empire then actu­ally get­ting results. More often than not pro­grams fail when peo­ple try to make things look bet­ter for them over actu­ally achiev­ing results and if you are not will­ing to stop that behav­ior, then you will not have a suc­cess­ful pro­gram. If you accept just get­ting an out­come over get­ting the best out­come, then your pro­gram will never be truly successful.

Test­ing Owner –

Role – This is the per­son who runs the day to day oper­a­tion of the test­ing pro­gram. Unlike the spon­sor who sets the higher level agenda and holds peo­ple account­able, your test­ing owner is the per­son who makes the magic hap­pen and makes sure that the pro­gram is run day to day. Liais­ing with other groups, set­ting the daily agenda, mak­ing sure con­sis­tent resources are lever­aged and mak­ing sure that the pro­gram does not get too caught up indi­vid­ual tests or ideas are your pri­mary duties. In order to do that, you will spend a large major­ity of your time edu­cat­ing and work­ing with other groups to change their per­cep­tions of what makes a suc­cess­ful test, that you are mea­sur­ing the value of every action. Cre­at­ing the momen­tum and chang­ing people’s per­spec­tives to insure a con­sis­tent opti­miza­tion process and not just a series of projects is your day to day directive.

You will be suc­cess­ful if – You can nav­i­gate the polit­i­cal waters and keep the many plates spin­ning. If you can do all that and make sure that every action is not wasted on non-strategic things, then you have a real chance to be a rock star. You must work proac­tively to edu­cate peo­ple about the dis­ci­plines of test­ing and how it is both dif­fer­ent and valu­able to the other dis­ci­plines in your orga­ni­za­tion. Being proac­tive and not let­ting oth­ers set the con­ver­sa­tion based on their agen­das is the pri­mary direc­tive of this role.

The num­ber one chal­lenge is not the day to day activ­i­ties, but keep­ing the con­ver­sa­tion at the right level and not allow­ing any of the small details such as tech­ni­cal imple­men­ta­tion, test ideas, resources, or ques­tions from out­siders to derail the pro­gram. It’s easy to do once, but doing it every day for months and years is the real discipline.

You will fail if – You are more inter­ested in play­ing nice or pro­mot­ing your­self then get­ting the right answer. If you get caught up in the con­cepts from read­ing the lat­est blog or con­fer­ence or think­ing that just get­ting a sin­gle answer from a test some­how makes you suc­cess­ful, then you are on the path to fail­ure. You have to stop your­self and your team from think­ing that just the act of get­ting a result some­how makes you successful.

Most pro­grams fail to see higher value because the peo­ple in this role are so des­per­ate to get a sin­gle result that they try to make it like other groups, or they start reward­ing other groups for just find­ing some­thing bet­ter and not for find­ing the best use of resources. They get so frus­trated by the pol­i­tics that they start mak­ing excuses, to oth­ers and espe­cially to them­selves, that they have accom­plished more then they really have. More then any­thing, it is the fail­ure to stand up and to do the tough things like chal­lenge peo­ple and not allow pop­u­lar opin­ion to drive things, even if they might help you polit­i­cally get ahead, that is the lead­ing killer of programs.

Tech­ni­cal resource –

Role – This is the per­son who knows how every­thing gets done and can make magic hap­pen with code. They own the cre­ation of test code and the process by which is works with other code on the site. They should be part of every dis­cus­sion work­ing on the effi­ciency of efforts and talk­ing about the best of the 50 ways to accom­plish this task. They can also be help­ful in liais­ing with other IT resources to make sure that every­one is on the same page.

You will be suc­cess­ful if – You are part of the con­ver­sa­tion but under­stand that just because you can do some­thing doesn’t mean you should. You will need to make sure that we are look­ing at things from how they fit together in a dis­ci­pline based view as opposed to what can we do. In order to do that, you will be part of the con­ver­sa­tion talk­ing about effi­ciency of not just doing one thing, but mak­ing sure that all tests are decon­structed and that you are test­ing mul­ti­ple fea­si­ble alter­na­tives, not just what is being asked orig­i­nally of you. They are also strongly work­ing with and edu­cat­ing the IT group to make sure that other projects do not inter­fere with and are enhanced by your test­ing efforts, nor are test­ing efforts slowed down by get­ting stuck in IT processes designed for very dif­fer­ent types of projects.

You will fail if –– You think that your job is just to do what oth­ers tell you and that process mat­ters more than out­comes. You will also fail if you try to do test­ing like other IT projects, espe­cially QA. Many peo­ple will want to reward you for mak­ing their ideas come to life, but fun­da­men­tally if you are not stop­ping bad ideas, if you think it is some­one else’s prob­lem, then you are pro­mot­ing your­self and not adding value. If you can­not find a way to make things work prag­mat­i­cally in a speedy man­ner, then you have no chance of succeeding.

Opti­miza­tion Analysts –

Role – The opti­miza­tion ana­lysts is the per­son who owns the reports and the out­comes of the test­ing pro­gram. They may also be the one that sets things up in the tool and who helps project man­age day to day activ­i­ties. When it comes time to make calls on out­comes, they take lead, cre­at­ing reports and help­ing edu­cate oth­ers on what was learned from each and every action. They will need to know a lit­tle bit about many dif­fer­ent things and should be able to hold peo­ple account­able to estab­lished rules of action, while at the same time talk­ing to other groups on their level.

You will fail if – You set the way that things are done before a test, not after. If you don’t cower to every ques­tion that is asked or pre­tend that you know some­thing that you don’t. If you are will­ing to remind peo­ple to focus on what mat­ters and not just dive into every ques­tion because you could find some data. If you work tire­less in find­ing bet­ter ways to do things.

Ulti­mately, you are suc­cess­ful here by what you con­vince oth­ers to not do far more then what you do.

You will not suc­ceed if – You try to answer every ques­tion after the fact or if you allow oth­ers to change mea­sure­ments to meet their per­sonal agen­das. If you do not under­stand the biases of peo­ple and how they are try­ing to use your and the data your pro­vide for their own self-promotion. If you are not will­ing to step up and chal­lenge peo­ple or to remind them of what mat­ters, then you will never suc­ceed. If you get caught up on the lat­est “buzz” or “expert” and for­get to work towards the right goals, then you will also not succeed.

Designer –

Role – You work with the test­ing team to cre­ate the mate­ri­als needed for each test. You make sure that you are test­ing not just the con­cept but also the exe­cu­tion of each con­cept by pro­duc­ing mul­ti­ple ways for each item thrown your way. You own the rela­tion­ship with your design teams and you work with the team to make things look good enough to go live on the site, while mak­ing sure that you are open to new things that don’t meet cur­rent stan­dards. You will be the one mak­ing sure that you have a large amount of dif­fer­ent fea­si­ble alter­na­tives for every effort. You will also be the one help­ing com­mu­ni­cate what is learned and how it changes or breaks cur­rent “best prac­tices” or style guides for your group.

You will be suc­cess­ful if – You make sure that you are test­ing out every­thing that is fea­si­ble and that the alter­na­tives are dif­fer­ent enough from each other. You get excited by the chance to try new things and know that the worst thing you can do is try to get “approval” for ideas.

You will fail if – You get too caught up in what has been there before or style guides or “expe­ri­ence” or any of a hun­dred other excuses and fail to test out things that go out­side of everyone’s com­fort zone. You don’t get excited by the chal­lenge of being cre­ative and get too caught up on request­ing or want­ing feed­back. You will also fail if you allow tests to get too caught up on con­tent and fail to address the other much more effi­cient parts of the user expe­ri­ence such as real estate, func­tion, and presentation.

There are of course many other roles and the nature of each of the ones listed is dif­fer­ent for each orga­ni­za­tion. What does not change are the dis­ci­plines to suc­ceed. Make sure that you are aware of who is doing what for your group, and even more impor­tantly you are doing the things to suc­ceed and avoid­ing the pit­falls that each per­son is bound to come across. You won’t be per­fect day one, but that doesn’t allow any­one in any role to avoid the respon­si­bil­ity of get­ting bet­ter and mak­ing things work for you.