business career entrepreneur success

People Overestimate the Quality of Evidence

Let’s see how people overestimate the quality of evidence and how people overestimate the likelihood events that seem typical, familiar or memorable.

We see patterns and generalize them

In large part this overestimation happens because we are so good at inductive reasoning, drawing evidence from one example and applying that evidence for some more general judgment or decision. We see patterns and generalize those patterns. We are so good at drawing evidence from an event in fact that will use that evidence even when it shouldn’t apply.

Suppose that you’re the CEO of a company that sells bicycles in you are making the final decisions about the release of a new bicycle targeted toward male casual writers, people who tend to ride the bikes only a few hours per month. Your marketing team comes to your office with a video testimonial from a rider who road tested the new bicycle and a focus group.

Before they show you the video they tell you this writer is not typical of your target market segment, he rides every day. Now, if you care about the quality of the evidence you shouldn’t let this video can influence your decision too much. This writer isn’t typical of the group you’re targeting so the evidence should be considered high quality, but here’s the surprising thing, people still use this evidence.

People are equally influenced by evidence

In fact, in laboratory experiments that test these sorts of decisions people are equally influenced by evidence regardless of its quality. An example the bicycle from CO it wouldn’t matter whether the video was from a writer identified as outside the target market identified is inside the target market or from rider whose typicality wasn’t mentioned.

Of course real-world marketers are very aware of who’s likely to buy their products so they go to great trouble to collect data from consumers in those target market segments, but the basic point still holds: we are easily influenced by anecdotes and stories, even if those anecdotes shouldn’t be relevant to our decisions.

Let me give you a simple example: think about a time in your past that you spent on a plane waiting for your flight to take off. If you flown enough times in your life you’ve probably had at least one memorable flight delay that time when you sat on the plane as a taxied to the runway then there was a mechanical problem then there was all the delay then there was icing on the wings then you had to wait for the deicer then there was another delay because of the weather and so on.

Decision might be clouded by extremes of emotion

So let me ask you a question: How unpleasant is flying? Is taking a flight something you look forward to or something you dread? There’s no right answer here: different people have different attitudes toward flying, but I want to explain what I’d just done: when I asked you to think about time you spent on the plane what comes to mind most naturally are the vivid unusual and frankly rare events like the delay that never ends.

When people bring those rare and memorable events to mind on average those events are seen as stronger evidence than they should be. After thinking about an extreme flight delay people would be more likely to think that flying is pretty unpleasant. But, now suppose that you are instead asked to describe a typical flight from your past. You wouldn’t bring to mind an extreme event like an extensive flight delay.

Instead you probably think about some flight that went more or less as expected. If people are first asked to think about typical events in the past than their judgments about the future arch is clouded by extremes of emotion. Anecdotes on extreme events don’t necessarily provide good evidence, events can seem extreme because they are atypical and unrepresentative in which case they shouldn’t shape our decisions.

When thinking about whether to accept the new job, don’t just think about the ideal circumstances something like: I’ll be in line to be VP within five years, I’ll get to travel on the annual company trip. Force yourself to think about what’s typical, along with what could be extremely good or extremely bad. Considering a broader range of evidence can help you make better decision.