There’s another reason that people overestimate the quality of their evidence toward a decision: people sent to seek out evidence that confirms their existing beliefs, instead of evidence that could refute those beliefs. That’s called confirmation bias.

Challenging the same beliefs

We see echoes of confirmation bias in the popular media in our conversations and in ourselves. When people take a strongly held position like a political hot button issue, they interpret new evidence in what ever way best fits their existing beliefs. So new evidence tends to reinforce people strongly held beliefs, even when that same evidence might be seen by a neutral party challenging the same beliefs.

In one study research participant’s were presented arguments for and against a controversial issue such as new taxes to support the local schools. Regardless of their own belief they were asked to evaluate the arguments as objectively as possible so they could explain each argument to another person and after each argument was presented they rated it strength.

Confirmation bias doesn’t go away as people gain knowledge

As you might expect, people show the confirmation bias, arguments that confirm their beliefs were rated as strong while arguments that challenge their beliefs were rated as weak, but the experiment didn’t stop there. The researchers also found that the confirmation bias was strongest in the people who had the most knowledge about politics as measured by a separate test, that is confirmation bias doesn’t go away as people gain knowledge, it gets stronger.

Several factors contribute to the confirmation bias one possibility is that people preferentially seek out streams of information that tend to support their prior beliefs. There’s good evidence of this matters. For example, people prefer news programs attend to share their political slant not challenge it, and they avoid other potential sources of conflicting information.

People actively reinterpret evidence

Another possibility is that when people see evidence against a strongly held position they simply ignore it. That’s usually not true though. When shown to arguments one confirming their beliefs the other disconfirming people tend to spend relatively little time reading and thinking about the confirming evidence, instead they spend their time looking at disconfirming evidence not ignoring it.

So that leads to a third possibility: people actively reinterpret evidence that would otherwise argue against their belief. When they read an argument against their position they don’t just accept that argument. Instead they start thinking about counter arguments, they internally generate new evidence in support of their prior belief, which in turn actually strengthens their original position.

We counter argue disconfirming evidence

This gives us two main reasons for the confirmation bias: we prefer sources of evidence that tend to confirm our existing beliefs and we counter argue disconfirming evidence strengthening original beliefs in that process. Now it’s very difficult to eliminate the confirmation bias, none of us wants to be constantly challenged in our beliefs and we all want to feel their beliefs are rational justified all one can hope to do is to minimize it as much as possible.

There’s one approach that helps it’s not very hard if you’re willing to try it, you just have to switch sides. By that I mean you should force yourself to take the other position at present its case without counterargument. If you passionately believe that taxes should be raised to support the public schools then force yourself to argue the virtues of reduced taxes as well as you can.

Think about the evidence from another perspective

You might argue that reduced taxes will lead to a stronger local economy more local jobs, a more vibrant cultural scene and eventually a larger tax base for the schools
do your best to identify the most reasonable and strongest arguments or that other position. The key here is to avoid counter arguing for your existing beliefs, instead put yourself in the shoes of another person, a reasonable member of your community who happens to take the other side.

That will ll disengage you from the desire to be consistent from your own passions and force you to think about the evidence from another perspective. Ideally you want to think about alternatives to your prior beliefs as early as possible so that you are too biased in the evidence you acquire, but it’s still good practice for all of us to challenge our beliefs, even strongly held ones.

Let’s now move from the quality of evidence to the amount of evidence. I mentioned that people tend to underestimate the value of having lots of evidence, even if that evidence is of low quality. In many cases the amount of evidence is given by the statistical term sample size which describes how many data points have been measured.