In my quest to tackle the major biases that plague our indus­try, the sec­ond one I want to tackle is Expec­ta­tion Bias or “the ten­dency for exper­i­ments to believe, cer­tify, and pub­lish data that agree with their expec­ta­tions for the out­come of an exper­i­ment, and to dis­be­lieve, dis­card, or down­grade the cor­re­spond­ing weight­ings for the data that appear to con­flict with those expec­ta­tions”.

In sta­tis­tics and math, you deal with the con­cept of expected out­come. You take the pay­out, fac­tor in the per­cent chance of it hap­pen­ing, and then fac­tor in the num­ber of chances you are tak­ing to reach there. This is the exact rea­son that the lot­tery is such a bad invest­ment, because it has a mas­sive out­come, but such a low like­li­hood of reach­ing that con­clu­sion that your expected pay­out is never going to hap­pen (Neglect of prob­a­bil­ity bias). Just because one per­son did win the lot­tery does not mean that most peo­ple win the lot­tery. The same is true for just read­ing the sto­ries or copy­ing oth­ers. There is a want to push out and tell peo­ple about how advanced you are, as well as the want to see imme­di­ate pay­offs (hyper­bolic dis­count­ing). There is also the need to dis­tin­guish your­self or your offer­ing and to dis­count oth­ers if an action only works 1 out of 20 times, and even then you don’t know if it is the best out­come even for that group, what pos­si­ble good is it going to do you?

Obvi­ously, the prob­lem then becomes know­ing the like­li­hood and the rel­a­tive value of the outcome…

Good luck get­ting that infor­ma­tion if you don’t have direct inter­ac­tion. Not only are peo­ple hes­i­tant to share if they know, but peo­ple are inher­ently wired to not seek or know that infor­ma­tion them­selves (choice sup­port­ive bias). What does hap­pen how­ever is a giant group of ankle biters… peo­ple or groups (espe­cially agen­cies or “experts”) who promise that they have all the answers, who tell you that if you lis­ten to them, every­thing will be golden, and that all of their clients are super suc­cess­ful. It is not evil, it is not even that they know what they are act­ing self­ishly, they are just wired to think in these terms, as any human is to want to believe them. This phe­nom­e­non and the play of it on the human psy­che is a recur­ring theme and one that leads to very inef­fi­cient and under per­form­ing pro­grams, where peo­ple are left either defend­ing poor results or seek­ing yet another magic answer to all their problems.

Suc­cess­ful con­sult­ing and suc­cess­ful pro­grams focus on the dis­ci­plines of suc­cess, not just the actions. They talk about both good and bad out­comes, and how to get more of one and less of the other. They focuses on what defines suc­cess­ful actions and alter­na­tives, not just what you are doing or a sin­gle pos­si­ble alter­na­tive. Chal­lenge any­one or any­thing that only talks about “suc­cess sto­ries” or who can’t talk about how often things work and how that com­pares to alter­na­tive processes. There exists a feed­back loop where bad prac­tices are shared and cham­pi­oned just because they are being acted on, and not because they are valu­able. It allows them to tell sto­ries that show that they got some­one to do… some­thing. It is up to you to stop your path down that mobius strip, and instead chal­lenge your­self and oth­ers to think dif­fer­ently and to think in terms of find­ing a bet­ter way and a bet­ter answer. Pro­grams have to be able to learn, grow, adapt and deal with real world prob­lems, which means that any sin­gle “answer” is just play­ing to your hubris and leav­ing you open to future failure.

One of my favorite say­ings is “Infor­ma­tion is not knowl­edge”. Seek­ing out bet­ter infor­ma­tion and chal­leng­ing the sto­ries and prac­tices of oth­ers, chal­leng­ing your self to learn and grow and change your biases and your want to be “right” is the only way for your pro­gram to truly move for­ward. Focus on the sys­tem, focus on learn­ing, focus on fix­ing why you do what you do, and never stop, never let go, and never let your­self fall for these traps. That is how you succeed.

2 comments
John Hunter
John Hunter

Focus on learning, make that the primary objective. It is good to confirm what you believe, but it doesn't help tremendously. Instead you should be excited when things contradict what you believe. This gives you the chance to learn. This doesn't mean being confused and unclear. It doesn't aid learning to accept crazy ideas with little evidence when you have a huge pile of evidence saying the opposite. But those that continue to learn throughout their life are able to accept and revel in new and contradictory ideas. And to seek out new areas that they are ignorant of. I find those that believe in simple, clear answers are interesting in hiding from the challenge of dealing with the mess of complex issues that impact us everyday. You can survive perfectly fine ignoring much of what is happening and holding to overly simple ideas that look good on a power point slide. But you often are greatly limiting what you could understand.