One of the great mys­ter­ies of the ana­lyt­ics space is the use of words that have almost no real mean­ing. Words like opti­miza­tion, ana­lyt­ics, mar­ket­ing, social, value, per­son­al­iza­tion, pre­dic­tive, and seg­ment have dif­fer­ent mean­ings to dif­fer­ent peo­ple. They become use­ful jar­gon to direct a con­ver­sa­tion, but when it comes down to giv­ing them a real mean­ing, so many groups strug­gle because it is a very per­sonal def­i­n­i­tion. When we do find a mean­ing for those words, it is usu­ally an old tired one that has lost all rel­e­vance in the mod­ern world. To me, the most com­monly abused term is efficiency.

What does effi­ciency mean? Is it just an out­come? Is it some­thing that you can actu­ally mea­sure? If it is as sim­ple as just ROI, why do we fail then to really mea­sure against it? I want to present a sim­ple way to think about effi­ciency, your actions towards improv­ing it, and then give you real world ways to use that to mea­sure your actions and to improve the “effi­ciency” of your organization.

Here is the way I sug­gest mea­sur­ing efficiency:

This gives you a value, which you can then mea­sure against oth­ers. The dif­fer­ence between val­ues shows you what is effi­cient and what is not effi­cient. It is strongly related to ROI, but sep­a­rates its com­po­nents and allows you to look at any action, not just rev­enue. We are given the choice to inter­act with 1 or all 3 parts, and we can mea­sure our abil­ity to do so on the same scale.

It is impor­tant to under­stand the 3 com­po­nents, to make sure that every­one is on the same page.

Scale – The size of the pop­u­la­tion that is impacted.
Impact – This is the mea­sure of record­able lift or gain. This is your abil­ity to influ­ence. This must be towards a site wide goal, not just a depen­dent goal such as the next page or clicks.
Cost – This is how much in time, energy, money or other resources it takes to acquire and main­tain the impact listed above.

To do this how­ever, you must always keep all three things in mind, not just one.

Scale reminds us that a high increase of a small group is often less valu­able then a small increase to a large group. We can try to increase the scale of some­thing, but with­out know­ing the impact or the cost to achieve it, we have nothing.

Impact reminds you that you can’t only look at lift. If you hear that you got a 12% lift, then you are still miss­ing two really impor­tant pieces of infor­ma­tion. If the 12% is to 100 peo­ple or 100,000 peo­ple, it dra­mat­i­cally changes the outcome.

The cost to achieve those two pieces tells us if we actu­ally did some­thing valu­able or not. If it takes you 2 hours and $20 to achieve this out­come, or if takes you 6 months, 500 man hours, 1.2 mil­lion in new prod­ucts and has a long term main­te­nance cost, then it is not going to be as valuable.

In order to enforce a con­ver­sa­tion around max­i­miz­ing return, you must first change the con­ver­sa­tion so that you are no longer dis­cussing only one of these met­rics at a time. Do not accept a con­ver­sa­tion that only tells you lift, or that only tells you a pop­u­la­tion with­out know­ing the abil­ity to impact that group and the cost to achieve that change. Do not just blindly hear that you have like­li­hood to change a met­ric, under­stand that you have to know the cost and scale of doing so. Do not just hear that a group has a dif­fer­ent behav­ior, under­stand that you need to know the scale of impact and the cost to change them to under­stand the effi­ciency of that action.

So this may seem like a very sim­ple def­i­n­i­tion for a com­plex issue, but it gives you the abil­ity to truly view the world dif­fer­ently. To quote Jim Horn­ing, “Noth­ing is as sim­ple as we hope it will be.” We like to pre­tend we think these things all the time and that they are obvi­ous in every con­ver­sa­tion, yet time and time again we drop the entire con­text in the name of push­ing an agenda. There are hun­dreds of con­ver­sa­tions every day that talk about met­rics that have noth­ing to do with improv­ing per­for­mance (e.g. bounce rate) or that only talk about a sin­gle por­tion of per­for­mance (lift). Stop those con­ver­sa­tions, and remind peo­ple that reduc­ing costs or increas­ing scale is just as effec­tive as improv­ing your impact. Do not assume that every­one is putting every­thing in the right con­text, because they aren’t.

So what is effi­ciency? It is sim­ply the act of mak­ing sure that you are improv­ing this ratio, and you are remem­ber­ing that you can not look at only one aspect to answer a ques­tion. We can’t fail to mea­sure actions against each other. These are not just iso­lated events. It is act­ing in a way that you keep both the denom­i­na­tor and numer­a­tor equiv­a­lent in your dis­cus­sions and actions, and that you do every­thing in your power to reduce low value actions and increase high value actions. Once you have an action, mea­sure it against other actions, and con­tinue to bal­ance the dis­cov­ery of the value of actions against your exploita­tion of the higher value ones.

Being effi­cient is sim­ply tak­ing resources away from low value actions and towards high value actions. The very con­cept implies that you will stop doing cer­tain actions and that you will do new ones you aren’t cur­rently doing. It is the entire dis­ci­pline of know­ing that what you are doing today is wrong, and that there is always a bet­ter way to do things.

It is not the con­cept but the con­stant dis­ci­pline of fol­low­ing it and hold­ing your­self and oth­ers account­able that will truly define your out­come. Noth­ing here is rev­o­lu­tion­ary, other than elim­i­nat­ing all of the other fac­tors and excuses peo­ple love to through­out in their argu­ments. It gives you the way to mea­sure dif­fer­ent out­comes against each other, and because of that, you can truly see what the value of var­i­ous actions are against each other.

If you are dis­ci­plined in your track­ing, hon­est in your impact, and will­ing to eval­u­ate actions as how they help your site, and not just you, you will arrive at amaz­ing con­clu­sions that will shift your orga­ni­za­tion. The only way to improve is to change, so do not fear change, embrace it. Do things you aren’t sure about, chal­lenge com­mon think­ing, do the exact oppo­site to see what the value of what you are doing really is. Mea­sur­ing things in this sim­ple a form is not sexy or “advanced”, and it can seem juve­nile, but it is only by doing the small things well that you will ever suc­ceed at all those large things peo­ple promise rev­o­lu­tion­ize the world.