When choosing the next fallacy to cover, I faced a tough choice as there are so many different fallacies that describe the same human behavior: The belief that we know or can answer things we can’t by assigning pattern or reason to things without actual cause. We are wired to want to explain why things happen, but in order to accomplish that task, we ignore or use only data we want and we supplant our own points of view as the core reason things happen. We believe that the world is far more established and easy to understand then it really is. My favorite fallacy that covers this behavior is the Texas Sharpshooter Fallacy, which is when someone assigns pattern or reason to random chance.
The name Texas Sharpshooter comes from this “story”:
A cowboy takes aim at a barn and starts shooting randomly. When he is done, we walks up and notices that there are a large number of holes in one area and fewer holes in another. He then paints a bull’s-eye over the area where there are a large number of holes. To anyone walking up, it looks like he was a good shot and mostly hit where he was aiming.
Now while I am sure that we can all think of cases where others have done this with data, the first thing you need to understand is that we all do this… all the time. We see patterns and rationalize our own actions, whether it is why we do things in a certain order or even why we believe certain “truths” about the world. We rationalize decisions after we make them, and while they are not all random, our understanding of why we do things is often flawed at best and completely delusional other times. The human brain actually engages the rationalization part after the action part, meaning that we always act, then think of why we act, not the other way around. We draw circles around the patterns of our own behavior and then accept those circles as the logic that lead to the decision. This makes our understanding of why people do things often extremely flawed, since so much of how we view others behaviors is through the context of our own “understanding” of what drives our own actions. We so want to come up with a why, and we dive so deep, that we miss the point that we will never truly know. Nor does it matter, sense we are describing a pattern, one that we can engage and interact with and build rules around, without needed to know all the causes of that pattern.
One of my favorite examples of this in the real world is a psychology professor in Baltimore that does the same demonstration each year. He starts his lecture by bringing a chicken up in a cage on stage. The cage has a feeder that is set to dispense food pellets at random time intervals. He then covers the cage and talks for an hour and half. At the end of the presentation, he takes the cover off and without fail, the chicken is found doing some behavior over and over again; it has convinced itself that this behavior is why the food comes out. The food comes out no matter what it does, and it has no control, but it has convinced itself that it is in control of the situation. We are all like that, we have to explain things so bad that we will believe anything, or will paint bulls-eyes, where they aren’t to make ourselves feel like we have more control then we really do.
We like to believe we are smarter then that chicken, but we aren’t. In our world, data is our food, so we assign patterns to explain changes in what we observe. Data becomes a crutch to accomplish this task. We so want to have a story to tell others and ourselves that we find one in the data. We believe that because conversions went up, the message must have “resonated” or because one group has a different winner then another group, that it is because of their socioeconomic status or because they are more familiar with technology. We have no way of knowing this, but we convince ourselves and others that this is the reason why. The reality of the situation is that we need “why” to help us feel like we understand, but acting and using data in no way requires a why so much as it requires a willingness to act.
Looked at from a data perspective, this means that when we see a noticeable meaningful change, often from testing, we are left to think of why it happened. People are fascinated with the “why?” often at the cost of what comes next. The reality is that we are always going to be looking only at a noticeable change and then apply rationalization after. We get so caught up in the why that we miss the truth that we will never really know nor does it matter. Having a clear plan of action for our data means that we never need to know the why to be successful, and in fact insures that the more we dive in and try to answer it, the more we are wasting resources. Acting on data requires willingness and alignment, it is decided before something happens. Rationalization is what happens afterwards. Why does not change your need to act on the data, nor does it allow you have some sudden insight into human behavior. At best you have a single data point, at worst you are painting bull’s-eyes around holes and calling them insight.
Marketers have been trying to figure out the “why” for a long time, and while there is a lot of people that claim to know, the reality is at best we have pattern, and at worst we have stories we present to make ourselves look good. You can not derive pattern from a single data point, yet we are obsessed with trying to do that very thing. If we are honest with how we go about collecting data, and we are open to consistent and meaningful action from testing, then why will never matter. If we are following the data and disciplined, then we know how we are going to act based on the results, not why the results happened. If you are disciplined in how you think about users, then you know that a story or a single data point will never tell you anything. If we really want to make things personal, then we won’t force “personas” on people, but instead let data tell you the casual value of changing the user experience and for whom it works best.
At its worst, the Texas Sharpshooter Fallacy represents our need to show that we are more in control or know more than we really do. We use the need to explain why to make stories and to help communicate our value to others. My background is in historical analysis, and one of the first things you learn is how little value comes from the first person narrative. It shows far more about the fallacies of the person speaking then it does for providing real information about what really is happening. Data at its heart is meant to improve situations, not to allow you to come up with a story that satisfies your world view.
Why is not a question that you can ever truly answer, yet most people in marketing are obsessed with a Sisyphean quest to answer it. The reality is that it is a question that has nothing to do with how you act on data or the disciplines needed to be successful. We do not need to know why for everything, even if it seems to hold all the answers. We just need to know what to do with what is in front of us and to appreciate how little we really know about the world in which we live.