business career entrepreneur success

How Much Evidence Does One Need

There is a distinction between randomness and evidence. We want to use evidence to make good decisions, but we often have a hard time knowing when to trust the evidence in front of us and when to be skeptical. Let me introduce you to an amazing example of when not to trust the evidence that we see.

The scientist that was wrong

In the late 19th century one of the most famous astronomers in the world was it amateur not professional scientist, his name was Percival Lowell. Lowell was born into a very wealthy family in Boston, his wealth allowed him to pursue his intellectual passions, chiefly astronomy and he combined a passion for science the businessman’s pragmatism.

At that time, astronomical observatories were typically located where people were, in large cities or in universities. Lowell decided to found an observatory on a mountainside in the middle of nowhere, near Flagstaff in what was then the Arizona territory. The sky there was a dark and clear in the new telescope he installed there was one of the largest refracting telescopes in the world.

It can gather more light than almost any other in short time Lowell and his observatory colleagues became world leaders in astronomy. They mapped the surface of Mars, identified many new asteroids and even discovered tantalizing hints that sparked the search for what is now called the dwarf planet Pluto. Lowell was, by any measure, an exceptional scientist with the careful keen eye, but Lowell ended his career in scientific disgrace.

In 1896 Lowell published a map of the surface of Venus, it showed a large dark spot slightly to the right of the center of the planet as viewed from Earth. Radiating away from the spot where a set of spoke like dark paths each twisting and turning in a complicated pattern. Lowell interpreted what he saw as evidence, in this case, evidence for an alien civilization on Venus.

The civilization that was not there

The central spot was a massive capital city and the spoke like paths were roads and other structures that connected the capital to other communities throughout the planet. Even at that time this was a controversial even extraordinary claim. Others looked into their telescopes, admittedly often smaller telescopes that were sites for observing but they saw no cities no canals, and the idea that there was a civilizational Venus that just seemed preposterous to most scientists of the day.

We now know based on all of our modern observations of Venus that there are no cities no roads visible from space. In fact through modern visible light telescopes Venus actually appears largely featureless, it’s so bright and has such thick cloud cover that you can’t see any service details. Percival Lowell one of the most passionate careful and technologically sophisticated astronomers of his day got it wrong.

He made two mistakes. First he placed too much trust in evidence that only he could see compared to what everyone else claimed and he should have been more skeptical given that it was at least somewhat unlikely there would be an advanced civilization on the nearest planet. In this article I’ll discuss the nature of evidence: how should we incorporate evidence into our decisions and judgments and in what ways do we make mistakes. Let’s begin with a brief consideration of information, that’s one of those words that clearly means something to each of us even if it is a bit difficult to define.

The engineer, Claude Shannon, is often considered the founder of information theory after World War II he was working on the mathematics of communication, how messages are transported from place to place, he defined information as roughly a reduction in uncertainty.

Having the right information is evidence

Suppose for example the communication channel could send to possible signals say the dots and dashes of the Morse code. If someone was waiting for the first signal to arrive they would have uncertainty about those two possibilities then when the signal did arrive say is a dash, the two possible outcomes would be reduced to one That reduction of two possibilities to one outcome is called a bit of information.

Information reduces uncertainty and information can be meaningful or meaningless. When Shannon developed his theory of communication he wasn’t necessarily interested in meaningful communication messages that tell somebody something. In fact, he explicitly stated that whether messages have meaning isn’t relevant to the engineering problems of sending and receiving them.

But when we make decisions we only care about meaningful information. Suppose you have an old pre-digital television set hiding in your attic. When you turn on that said and it doesn’t receive any signal it will show a speckled flickering pattern, all the small pixels in that television will be flashing randomly.

Now does this flickering pattern carry information? Yes! In fact that random pattern carries much more information then any real sort of television signal. To help you think about this imagine that the television was instead showing a tranquil beach scene from a static camera pointing out toward the ocean as waves rolled slowly and if you saw one frame of that beach video you know a lot about the other frames.

Having good and bad quality information

The beach and the ocean would tend to be in the same places from moment to moment, only a little of the information the video changes from frame to frame. When the television only shows flickering noise however one frame of noise tells you nothing about the next frame.

You’d need a lot more information to represent every pixel in every frame of the video, but that information is meaningless, it doesn’t help us predict what will come next. For decision-making we care about information that helps us make better decisions, that helps us decide on one course of action or another, we care about meaningful information which I’ll call evidence.

It’s ideal if we have a great deal of high quality evidence supporting our decisions but that isn’t always the case. Sometimes we have only a little evidence, but it’s of high quality and other times we have a lot of evidence but it’s of low-quality. You can think of the pros and cons of these two possibilities in a single example: suppose that you are trying to decide whether to go to a new downtown restaurant for dinner tomorrow.

If you call a close friend who recently went to dinner there you’d get one person’s opinion, but that opinion would be very trustworthy, or you could look up reviews of that restaurant online. You might get many people’s opinions although each opinion wouldn’t be that trustworthy individually. Both of these are valuable for decisions, you want high quality information when you can get it and you want lots of information when you can get it, but you don’t always have lots of high-quality information.

When people have to decide between two options, one with a little high quality evidence and the other with a lot of low-quality evidence they often make systematic mistakes. In particular people tend to overestimate the quality of their evidence and they underestimate the value of having lots of evidence, even if it’s low quality.