Per­son­al­iza­tion is such a buzz­word right now that it is nearly impos­si­ble to have a con­ver­sa­tion in the dig­i­tal mar­ket­ing space with­out it com­ing up. Every­one is on this quest for a “per­son­al­ized expe­ri­ence” or to make sure that they are doing what every other group is doing. You con­stantly hear about all this new tech­nol­ogy and all these new ways to accom­plish this task. There is more tools and infor­ma­tion about our users now than ever before, and yet there are very few groups or peo­ple who actu­ally can dif­fer­en­ti­ate between suc­cess and fail­ure for personalization.

The most fun­da­men­tal thing peo­ple for­get about “per­son­al­iza­tion” is that I can “per­son­al­ize” an expe­ri­ence in almost infi­nite ways. I can change copy, I can change work flow, I can change lay­out or fea­tures of the expe­ri­ence. Even bet­ter, I can do this for the same user in a thou­sand dif­fer­ent ways. I am a return­ing user to your site… but I am also a user in the after­noon, who came from Google, who has been on the site 12 times, who has made 3 pur­chases, and who is using Fire­Fox. So the ques­tion is not CAN I per­son­al­ize an expe­ri­ence, at this point there are a thou­sand dif­fer­ent tools and ways to do so. So the sim­ple act of cre­at­ing an expe­ri­ence is not the goal, the goal is to do so in the way that gen­er­ates the great­est ROI for my organization.

The ques­tion needs to be, how do I dis­cover the most valu­able way to change the experience?

What we need to incor­po­rate in any con­cept of per­son­al­iza­tion is a way to mea­sure these dif­fer­ent con­cepts against each other. We have to build into every process a period of dis­cov­ery, using tools that allow us to know the two most valu­able pieces of infor­ma­tion when it comes to per­son­al­iza­tion:

What is my abil­ity to change their behavior?

What is the cost to do so?

There is no way to acquire that infor­ma­tion with­out actively mak­ing changes and see­ing the out­come. Mea­sur­ing that dif­fer­ent groups have dif­fer­ent behav­ior is easy, but what does that tell you about your abil­ity to change that behav­ior? Just because one group of users pur­chase twice as often as another, how do you know your abil­ity to change that behav­ior? How do you know that a dif­fer­ent expe­ri­ence will do any­thing more than a sta­tic sim­i­lar expe­ri­ence for both?

And that is the dif­fer­ence between suc­cess and fail­ure when it comes to per­son­al­iza­tion. Are you just serv­ing up an expe­ri­ence because you can? Or have you done the active acqui­si­tion of knowl­edge that shows not only that it improves per­for­mance, but that it is the best way to increase performance.

I want to give a func­tional exam­ple so that you can see this in action. Let’s take the exact same con­cept and see it exe­cuted under both ways of thinking.

Let us say that it is com­ing up on the hol­i­day sea­son, and you want to serve up a hol­i­day ship­ping mes­sage to peo­ple who have pur­chased on your site before.

If my goal is increased rev­enue, then the steps would be as follows:

1. Cre­ate mul­ti­ple exe­cu­tions of the mes­sage (how do you know if the con­cept or the exe­cu­tion is the issue with one offer?)

2. Take 2 to 3 other mes­sages that could be used there (one will most likely be your default con­tent), other con­cepts such as spe­cific prod­ucts or spe­cific site offer­ings. Hope­fully you are just reusing exist­ing content.

3. Serve all the offers to EVERYONE

4. Look at the results by seg­ment and cal­cu­late the total gain by giv­ing a dif­fer­en­ti­ated experience:
i. If you are cor­rect, then the high­est per­form­ing recipe for the pre­vi­ous shop­per seg­ment will be one or both of the ship­ping mes­sages. Default con­tent would then be the win­ner for the non-purchaser seg­ment (the com­pa­ra­ble segment).
ii. If you are wrong, then any other seg­ment will have a higher win­ner for any of the offers. Be open to per­mu­ta­tion win­ning that you never thought of. Being wrong is always going to pro­vide the great­est return

5. Push live the high­est rev­enue pro­duc­ing oppor­tu­nity found

Let us see how groups that get lit­tle, no, or neg­a­tive value from “per­son­al­iza­tion” do the same task:

1. Push the sin­gle piece of cre­ative to the repeat pur­chaser segment.

2. Hope

See the fun­da­men­tal prob­lem is that in the sec­ond sce­nario you have no way of know­ing if it is valu­able, or not. Blind belief that you are pro­vid­ing value is not the same as pro­vid­ing value. Most groups think that if they just report the out­come, or the rate of action of that group, that it some­how rep­re­sents the value of that action. It doesn’t. Value only comes from the improve­ment of per­for­mance by that action. If you aren’t actively acquir­ing that infor­ma­tion, then you have no way of know­ing the value of any action. Even worse, we are adding cost and we suf­fer­ing from oppor­tu­nity cost from the gain we should be getting.

I want to show some sim­ply math to show you the dif­fer­ence in the two groups. Let us say in the first test we have the 5 dif­fer­ent expe­ri­ences and that we are look­ing only at 10 dif­fer­ent com­pa­ra­ble seg­ment groups (seg­ments only mat­ter if there is a dif­fer­ent out­come for the com­pa­ra­ble group). This might include things like new/returning, work hours/non-work hours, search/non search, Firefox/chrome/Internet Explorer, or any other of the infi­nite ways of divid­ing your users using any and all of the infor­ma­tion that is avail­able to you. You can always do more, but for the sake of argu­ment and of effi­ciency, 10 dif­fer­ent pools of the same pop­u­la­tion is enough. Seg­ments are only valu­able for tar­get­ing if we serve things to the com­pa­ra­ble seg­ment. If I assume that every­thing is purely ran­dom, then I have a 1 in 5 chance of my offer being the best. I also have a 1 in 10 chance of my seg­ment being the MOST valuable.

(1÷5) * (1÷10) = 2%

So if every­thing is ran­dom, then I have a 2% chance that I picked the best out­come (the one that dri­ves the high­est rev­enue for my site), which means that in 98% of the sce­nar­ios, I have cost my site money. But let’s assume that you are REALLY good at pick­ing seg­ments and con­tent based on your expe­ri­ence and your analy­sis. Hav­ing worked with nearly 300 dif­fer­ent orga­ni­za­tions, expe­ri­ence shows that the best of peo­ple who aren’t rely­ing on causal data are no bet­ter than 2 times ran­dom guesses for choos­ing a bet­ter option (they guess a right answer twice as often than just the ran­dom sample).

Most groups do not fall into that cat­e­gory. In real­ity, most groups actu­ally are worse than ran­dom at choos­ing the best option.

That means the math is only:

(2÷5) * (2÷10) = 8%

Let’s say you are the best per­son in the world at what you do, with great analy­sis and all sorts of tools, so that you are three times better:

(3÷5) * (3÷10) = 18%

So if you are absolutely amaz­ing at what you do, then 18% of the time, you will have guessed the right mes­sage for the right group. 82% of the time, another out­come is bet­ter and most likely sig­nif­i­cantly bet­ter. You can reduce that to 0% of the time a bet­ter per­form­ing option with a few sim­ple steps and accept­ing that we do not always under­stand the pat­terns before us. If we go back to ran­dom chance, then 20% of the time just doing noth­ing (your default offer) actu­ally per­forms bet­ter for every­one. If you are the bet­ting type, which would you take? 8% ver­sus 100%? Espe­cially when the scale of impact can be massive.

Remem­ber that in all sce­nar­ios you are going to get an out­come, so that can’t be the mea­sure of suc­cess. The process of find­ing the right answer is far more impor­tant than a con­ver­sa­tion around the func­tion of a tool. Nor can dis­cussing only the impact of one seg­ment, since we are not com­par­ing it to oth­ers in con­text. The ques­tion is did doing this one thing pro­vide MORE value than doing another action (or doing noth­ing), and the only way to answer that is to com­pare out­comes. All of the down­side is when you look at “per­son­al­iza­tion” as just a func­tion that you make a deci­sion on and just do. All of the upside is when you dis­cover value and then exploit it. There is noth­ing more valu­able then when you are wrong, but the only way to dis­cover that is when you cre­ate a sys­tem that enables it.

The dif­fer­ence between a suc­cess and a fail­ure with per­son­al­iza­tion comes down to this:

If the goal is to make money, the ques­tion to ask is not to ask CAN I do per­son­al­iza­tion, but how do I put steps in place to ensure that I am hav­ing both a dis­cover and an exploita­tion phase to my actions?