The idea of the wisdom of crowds has become so well-known that is hardly questioned any more. Many people from politicians to venture capitalists now assume the crowd simply do better than individuals when forming judgments and making decisions, but in today’s article all present a more nuanced view.

Who wants to be a millionaire?

“Who wants to be a millionaire?” is one of the most popular game shows in history. Its basic premise, which is has been by now  replicated in dozens of countries, is that a single contestant answers a series of multiple choice trivia questions for ever increasing amounts of money.

The questions are drawn from any area of knowledge: sports, geography, literature, anything at all. The contestant has several lifelines to help with difficult questions. For example he or she can poll the audience members to get their collective answer and if the contestant makes it all the way to the end of the game without missing a question they win the titular prize of $1 million or pounds etc.

The first few questions are generally very easily answered they’re simple general knowledge questions that allow the television audience to get to know the contestant and that allow the contestant to answer a few questions with confidence before they get the harder trivia. On one episode of the French version of Who wants to be a millionaire the contestant faced a typical early question: “Which of these revolves around the earth”/

The potential answers were: “the Moon, the Sun, Mars and Venus”. As these answers were revealed the contestant visibly sighs and swallows. He repeats the question “which of these revolves around the earth”. His eyes scanned the different options then he says: “I’m going to ask the audience!” The host instructs the studio audience to pick up their electronic controllers and place their vote.

Then after a few seconds of upbeat music the audience poll is revealed 56% answer that the Sun rotates around the earth 42% say that the Moon rotates around the earth and negligible fractions answer Mars or Venus. The contestant nods his head and then starts talking his way through his answer Mars definitely not, Venus no. After a pause he says the response is “the Sun”. He goes with the crowds response and he loses.

The wisdom of crowds

The Moon of course rotates around the earth but he was led astray by his own ignorance or perhaps nervousness and by the majority opinion of the studio audience. Why did the audiences majority judgment lead them astray? It’s probably not because the audience didn’t know that the Moon rotates around the Earth. Instead the audience was split: slightly less than half told him the correct answer to help them, but slightly more than half told him the incorrect answer probably on purpose as a punishment for soliciting their help on such an easy question.

In many cases groups of individuals make more accurate judgments and better decisions than single individuals. This phenomenon which is come to be known as the wisdom of crowds explains why the studio audience in Who wants to be a millionaire usually does provide the correct answer. It is also been applied to many real-world problems: estimating numbers, developing consumer products, betting on sports and political events and brainstorming for creative solutions.

The idea of the wisdom of crowds has become so well-known that is hardly questioned any more. Many people from politicians to venture capitalists now assume the crowd simply do better than individuals when forming judgments and making decisions, but in today’s article all present a more nuanced view. As is often the case the true nature of this phenomenon is a bit more complex than the popular story.

Sometimes the crowd does very well indeed, but other times the crowd goes systematically astray for predictable reasons. Wisdom of crowds is an old idea dating back more than a century that’s continually being shaped by new research. The spark of the idea came from a short paper published in 1907 by the English social scientist and statistician Francis Galton.

Throughout his long life Galton had been interested in averages as a statistician he invented the measure that is used to explain variation on averages we call the standard deviation. He showed how extreme values of data tended to move back toward an average as more data were collected. That’s a regression to the mean and he measured countless people’s faces and body parts trying to identify average characteristics of the groups from which they came.

Vox populi

His essay published in the scientific journal Nature described data he collected from judgments of the dressed weight of an ox as made by visitors to the Plymouth livestock show. Almost 800 people have made judgments, some of those people weren’t were ranchers or other experts while others were just random visitors of the exhibition. Each though had purchased a ticket and written down there guess and hoped of a large price.

Galton scientific report was prompted by an unusual feature of those guesses: even though people’s individual guesses vary considerably the median value of the guests was very close to the true weight. In statistics the median is just the middle of some set of numbers. In this competition it’s the guess that was exactly in the middle range of guesses with half of the people guessing more and half guessing less.

That median was 1207 pounds, the actual dressed weight of the ox was 1198 pounds, only 9 pounds from the median guess. Let me describe this result a different way. Suppose that you were at that same fair more than 100 years ago. You know nothing about oxen, you are no expert books butcher, no rancher, perhaps you’ve never even seen an ox before. But unlike the other fairgoers of that livestock show you look at many other people’s guesses.

You take the middle of all those guesses and write that down as your guests, you would do much better than the typical butcher rancher or other expert. Galton described this effect as the vox populi the voice of the people now it’s called the wisdom of crowds. It’s not magic, it arises because there’s always error in people’s guesses. In many situations, some people guess too high and others too low.

Those different guesses will bracket the true value. When that happens taking the middle guess from a crowd is more likely to be close to the true value just taking a guess from a single person.