This article explores a few cognitive biases I’ve experienced first-hand as well as strategies for mitigating their influence. These biases are by no means the only ones that I think designers should be aware of – there are many, many others as well. That being said, I think these are the most common ones that designers must deal with during the design process. They are also the ones that have the potential to influence design decisions the most, which ultimately impact the products and services we help to build.
Consider What You Don’t See
During World War II, the Statistical Research Group at Columbia University was asked by the U.S. military to examine the damage done to bombers that had returned from missions. Their mission was to determine where armor could be added to the bombers to increase the protection from the flak and bullets. Planes that returned were carefully examined and damage was compared across all the planes. The consensus was that additional armor should be added to the areas of the planes that show the most common patterns of damage: the wings, tailfin, and middle of the planes.
Luckily, a statistician on the project by the name of Abraham Wald countered this conclusion by pointing out that the planes being examined were only those that survived, therefore calculations were missing a critical set of data (the planes that didn’t make it back). As a result, Wald recommended that armor get added to the areas that showed the least damage to reinforce the most vulnerable parts of the planes.1
The case study of World War II bombers highlights our tendency to concentrate on the people or things that made it past a selection process and overlook those that do not, typically because of their lack of visibility. This can lead to false conclusions in several different ways. This tendency is known as survivorship bias and it can show up during a critical part of the design process.
How it occurs
Research is a critical part of the design process because it forms the backbone of understanding in regards to end-users and how the product or service we’re designing will fit into their lives. The data we receive during this time is fundamental to the success of a project — as it helps to guide subsequent design decisions. One critical way that survivorship bias can creep into the design process is through the participants we choose (or leave out) of design research. A lack of participant diversity will inevitably lead to a lack of data diversity. If you’re only considering one perspective, it will reduce the reliability of the data.
Design feedback is another crucial part of the design process and also one that survivorship bias can take hold. The best design is that which considers a diversity of perspectives, specifically in regards to the people who will be using it. For example, focusing too much on only the positive feedback from our peers is likely to result in solutions that aren’t resilient enough. If the team with which we are receiving feedback lacks diversity, so will too the input we receive during the design process.
Factoring in failure
While survivorship bias is common within the design process, there are ways to counter it as well. When we focus too much on success stories, positive metrics, or ‘happy paths’ in our designs, we lose sight of how the design responds when things fail. Consideration of only positive feedback is likely to lead to a very one-sided design approach. It’s best to factor in failure, consider where things can go wrong, centralize the edge cases in the design process, and seek out diverse perspectives during design reviews. In other words, we can make our designs more resilient by considering the unhappy path just as thoroughly as the happy one. In the process of designing for the less ideal scenarios, we address the fundamental features needed for everyone.
Recognize the limits of quantitative data
We must remember quantitative data is only relevant to the actions currently available and therefore has the potential to limit our thinking. As Erika Hall points out in Just Enough Research, “By asking why we can see the opportunity for something better beyond the bounds of the current best”.2 We should consider what the quantitative data isn’t telling us to make more informed design decisions.
Related
A bias in which a sample is collected in such a way that some members of the intended population have a lower or higher sampling probability than others.
A mental shortcut that relies on immediate examples that come to a given person’s mind when evaluating a specific topic, concept, method, or decision.
