business career entrepreneur success

Shared Errors in Judgement

Crowds fail when they have a shared error in judgment and when that’s the case there’s a deeper problem the people who are most confident about their decisions are often the most wrong.  

Systematic perceptual biases

In a clever study people were asked to make judgments about simple drawings, for example which of two line drawings was created from a longer line. The researchers chose the questions so there would be some than most people get correct and so that there were others than most people would get incorrect. These mistakes occurred because of systematic perceptual biases that we all share.

For example, people tend to overestimate the length of shapes composed of curved lines converge shapes composed of straight lines and the tendency vertical lines is longer than horizontal lines. When the crowd tends to get a judgment right, the people who are more accurate tend to be more confident but when the crowd tends to get a judgment wrong then confidence is higher in people who are more inaccurate.

Confident is not accurate

We think the confident people are more likely to be right and we place our trust in, but just because someone is confident about their judgment doesn’t make them accurate, they are just more likely to be like the rest of the crowd. We can be overconfident in our own judgments as well and that can also cause the wisdom of crowds to fail.

When betting on football games, casual gamblers generally prefer to bet on the favored team even though the advantage of the favorite is essentially negated by the point spread or betting line. In one experiment, those point spreads were manipulated to help the underdog team, for example changing the bed from a 50-50 gamble the something were the underdog might have a 60% chance of winning the bet.

We tend to undervalue that information from other people

Casual gamblers still picked the favorites and they picked the favorites even when they were warned about that bias. Gamblers just like picking favorites probably because of a combination of two effects we’ve discussed in this course: a preference for betting on things are more familiar and the ease with which they can envision the favorite winning compared to the underdog.

Systematic biases like this one can reduce or eliminate any advantage of crowds and crowds fail when people ignore them. In many of our decisions we have the opportunity to collect information from other people but we tend to undervalue that information.

Adjusting judgement based on advice

When people receive advice from someone else about a judgment, weather guesses about the number of M&Ms in a jar or predictions about investment they tend to adjust their judgment by only about 25% toward whatever the advisor recommended this number 25% is itself an aggregate across three types of people:

  • most people ignore advice and they keep their original judgment,
  • some split the difference between the original judgment and the advisors, and
  • a small minority actually adopt the advisors position.

This pattern of results means that people treat their own judgments as much more accurate than someone else’s, they don’t use information from other people as much as they should even when those other people should be just as good at the judgment for the decision. And people don’t recognize the importance of sample size as I introduced in the electron evidence.

Using the wisdom of crowds

When given the choice of following the advice given by one confident person or the aggregate of several less confident people, a majority of people choose to follow the confident person. Remember people tend to overestimate the quality of their evidence and they underestimate the importance of the quantity of evidence.

But, confident isn’t necessarily predictive of accuracy, in fact situations where one confident person disagrees with a group of less confident people are precisely those in which the competent person is most likely to be wrong. Using the wisdom of crowds can lead to good decisions in some circumstances and poor decisions in other circumstances.

The main challenges lie knowing when to use the wisdom of crowds and  how to improve the process of decision-making so the crowd sourced decision-making can be more effective.