confirmation bias

Confirmation Bias

Critical Thinking

Among our critical thinking questions were:

Does the evidence really support the claim?

Is there other evidence that I should consider?

Sufficient Conditions

Consider the following claim: “you can get to CUHK from Lingnan using public transportation.”The claim is true if the MTR goes from Lingnan to CUHK.The claim is also true if there is a bus from Lingnan to CUHK.In fact, the claim is true if there is any way to get from Lingnan to CUHK.


A claim X is a sufficient condition for the claim “you can get to CUHK from Lingnan” when if X is true, “you can get to CUHK from Lingnan” is true. It’s true that if “the MTR goes from Lingnan to CUHK” is true, then “you can get to CUHK from Lingnan” is true. Therefore, “the MTR goes from Lingnan to CUHK” is a sufficient condition for “you can get to CUHK from Lingnan.”


A helpful way of thinking about this:

Whenever “If A then B” is true:

A is a sufficient condition for B.

Examples

“If today is Thursday, then we have class” is true.So, today’s being Thursday is a sufficient condition for our having class.“If you are convicted of murder, you will go to jail” is true.So, being convicted of murder is a sufficient condition for going to jail.

Examples

But the following are not true:

“If we have class today, then today is Thursday.”(we also have class on Monday)

“If you go to jail, then you were convicted of murder”(Some people in jail were convicted of other crimes)

Necessary Conditions

Suppose that “if A, then B” is true. We know that this means that A is a sufficient condition for B.

It also means that B is a necessary condition for A.

Requirements for Office

Consider the requirements for Chief Executive in Hong Kong:• Must be at least 40 years old.• Must be a Hong Kong permanent resident.• Must not have right of abode in a foreign

country.• Must have resided in Hong Kong for 20

consecutive years.

Necessary Conditions

Requirements for office are things that are necessary for office– things that must be true for you to hold office.So we know that if someone is the Chief Executive, then that person is at least 40 years old.In other words, being at least 40 (being a permanent resident, etc.) is a necessary condition for being Chief executive

“I know horoscopes can predict the future, because I’ve seen it happen.”

“Positive thinking can cure cancer, because I know someone who used it, and got better.”

“Everyone knows you do worse in your second year in college, you see it all the time.”

Necessary and Sufficient Evidence

If claims like these are true, for example, if it’s true that horoscopes predict the future, then the evidence in question is necessary.

If horoscopes predict the future, then there must be cases where a horoscope predicted something, and then it happened.

Necessary and Sufficient Evidence

But such evidence is not sufficient for establishing the truth of these claims.

When a horoscope predicts that X will happen, and then X happens, that doesn’t prove anything. Were there times horoscopes predicted things that didn’t happen? Were there things that horoscopes should have predicted, but never happened?

Contingency Tables

Contingency Tables

A contingency table is used to plot predictions of the form “If A happens, then B will happen.”

In the upper left corner of the table (Prediction = Yes, Observation = Yes) are cases where A happens and B happens. These are “true positives” and they confirm the prediction.

Contingency Tables


In the lower right corner (Prediction = No, Observation = No) are cases where A does not happen, and B does not happen. These are “true negatives” and also confirm the claim in question.

Contingency Tables


Two sorts of cases disconfirm the prediction. First there are false positives in the upper right corner, where the predicted outcome (Prediction = Yes) does not match the observed outcome (Observation = No).

Contingency Tables


Second, there are “false negative” cases. These are also outcomes where the prediction (Prediction = No) fails to match the observation (Observation = Yes).

Perfect Correlation Claims

For “absolute” claims (If A happens, B will always happen, A and B are perfectly positively correlated), all cases must fall in the upper left or lower right corners.

The prediction must always match the outcome.

Imperfect Correlation Claims

For probabilistic claims (A and B are imperfectly positively correlated), you must compare the proportion of true positives to predicted positives to the proportion of false positives to predicted negatives.

(How you should compare them depends on the base rates…)

For example…

Suppose I claim that students are more likely to get an A on the final if they take notes in class than if they don’t take notes.

I point out 5 students who took notes and got an A, and 18 students who didn’t take notes and didn’t get an A as evidence for this claim. But whether the claim is true depends on more than just these true positives and true negatives…

I’m RightGet an A Don’t get an

ATotals

Take notes 5 10 15

Don’t take notes

2 18 20

Totals 7 28 35

I’m Right

Here, it looks like taking notes matters. 5 out of 15 note-takers get A’s (33%) and only 2 out of 20 students who don’t take notes get A’s (10%).

However, with the same number of true positives, I could also be very wrong. Let’s add 10 students to the “false positive” and “false negative” squares…

I’m WrongGet an A Don’t get an

ATotals

Take notes 5 20 25

Don’t take notes

12 18 30

Totals 17 38 55

I’m Wrong

Even though the number of true positives and true negatives stayed the same, we now see that a measly 5 out of 25 note-takers got A’s (down to 20% from 33%), but a full 12 out of 30 students who didn’t take notes (40% up from 10%) got A’s. Students who didn’t take notes are more likely to get A’s, in this scenario.

Main Idea

The basic point is that you cannot use examples that agree with a claim to show that that claim is true.

You must also look at examples that disagree with the claim. If there are enough of these, then the claim might be false, even if there are lots of examples that support it.

CONFIRMATION BIAS

Confirmation Bias

Even though evaluating predictions requires looking at both the rates of true positives among predicted positives, and the rates of false positives among predicted negatives, human beings have a tendency to only consider true positives (and to a lesser extent, true negatives) when evaluating predictions or other claims of imperfect correlation.

Wason Selection Task

Around ½ of people studied say “D” and “5”.

About 1/3 say just “D”.

Only about 1/20 get the right answer: “D” and “2”!

Searching for Confirmation

People have a preference for positive answers that confirm their theories, even though negative answers that disconfirm their theories might give the same amount of information.

For example, suppose A picks a number between one and ten, and you’re supposed to guess what it is. I suggest that the number is 3.

The psychological research shows that you’d be more likely to ask “is the number odd?” than “is the number even?”– even though both answers are equally informative.

In this case, the preference for confirmation does not matter:

Asking the positive question and the negative question give you the same information. If people prefer the positive question, that doesn’t harm them at all. But the preference can harm them if the positive question gives less info.

A Strange Example

Americans were asked:

Which of these pairs of countries are more similar to one another?

1. West Germany, East Germany2. Sri Lanka, Nepal

They said (1), West Germany and East Germany. Others (Americans) were also asked:

Which of these pairs of countries are more different from one another?

1. West Germany, East Germany2. Sri Lanka, Nepal

They also said (1).

Americans thought that West Germany and East Germany were both more similar to each other than Sri Lanka and Nepal and less similar to each other than Sri Lanka and Nepal.

How is that possible?

First, when considering the question ‘which are more similar?’ the subjects looked for all the positive evidence that West Germany and East Germany were similar, and all the positive evidence that Sri Lanka and Nepal were similar.

Since Americans know nothing about Asian countries, they had no positive reason to think Sri Lanka and Nepal were similar.

Similarly, when asked ‘which are more different?’ the subjects considered the positive evidence that West Germany and East Germany were different and the positive evidence that Sri Lanka and Nepal were different.

Again, having no knowledge of Sri Lanka or Nepal, Americans chose (1), because of all the positive evidence in its favor.

But it cannot be true that East Germany and West Germany are both more similar and more different than Sri Lanka and Nepal.

What the subjects did not do is consider the relevant negative evidence that would disconfirm their hypotheses.

The Problem of Absent Data

Sometimes it’s not just that we only look for or evaluate the positive evidence, but that there is no negative evidence. This can lead us to think we have very well-confirmed beliefs when we do not.

Hiring Job Applicants

Suppose you’re hiring job applicants for your shoe company. You think people who haven’t studied a musical instrument would not be good employees.

Who do you hire? The people who have studied music, of course! And if they’re successful at your company, do you have good reason to believe that you were right?

No! You have positive evidence– people you predicted would be successful, who are successful– but you have no negative evidence.

What about all the people you didn’t hire, the ones who didn’t study music? They might have been successful too. They could’ve been more successful. You don’t know.

Absent evidence is all around us.

Suppose you decide to major in accounting instead of philosophy. You find that you are very happy studying accounting. Did you make the right choice? You can’t know. You could have been more happy studying philosophy. There’s just no evidence.

Self-Fulfilling Prophecies

The Prisoner’s Dilemma

There are two general strategies to playing the prisoner’s dilemma. You can view the game as one where the goal is for everyone to do well, and thus play “cooperatively” or you can view the game as one where the goal is for you to do better than your opponent, and thus play “competitively”.

The Prisoner’s Dilemma

Believing that the goal is selfish, to win more for yourself, is a self-fulfilling prophecy.

If you play against other selfish players, they will of course play competitively.

But even if you play against cooperators, they will have to play competitively, or get 0 every round. So it will seem as if there are no cooperators.

Another Example

Suppose you think I’m not a very nice person, so you avoid me.

If you avoid me, then you’ll never have a chance to correct your initial impression.

So if I’m a nice person, you’ll never find out if you start out thinking I’m not.

confirmation bias

Documents