June 12, 2009
Customers & Bullet Holes
In talking with painter James Christensen, Photoshop engineer Jerry Harris picked up an anecdote I found interesting:
During World War II, Allied bomber losses were high, so the powers that be demanded a fix. The engineers set out to eyeball every bomber they could, gathering great statistics for each bullet hole. After a long study they decided to add more armor plating to the areas that had the highest concentrations of holes. A bit after these improved planes were deployed, they received some startling news: more planes were going down than before. At this point I thought, “Did they make them too heavy?”
Then the light bulb went on for someone: they had measured every bullet hole in every plane at their disposal, but they’d failed to realize it was the ones that they did not have access to that mattered. It was the ones that did not return that needed to be scrutinized. They needed to improve the armor in the places that the returning planes had no bullet holes.
Sounds like it might generally relate to product marketing, and user studies: go investigate the customers that don’t return for seconds, i.e. upgrade.
I can’t vouch for the story’s veracity, but the lesson seems sound. This is part of why one needs to listen to customers, but only up to a point: they’ll tell you how to please the customers you have–but not the ones you don’t have.