thinking about probabilities ccc8001. assignment watch episode 1 of season 1 of ancient aliens
TRANSCRIPT
Thinking about Probabilities
CCC8001
Assignment
Watch episode 1 of season 1 of “Ancient Aliens.”
Assignment
Find one thing that is said, shown, or presented in the episode that is misleading. I want you to describe it to me, then to explain why it is misleading.
Assignment
What you describe to me can be misleading for any reason, not just a reason we’ve talked about in class. Just describe it, and tell me the reason why it is misleading.
PROBABILITY BIASES
Conjunction Fallacy
Which of the following is most likely to happen? a. There will not be a final exam in this class.b. There will not be a final exam in this class, because the instructor has to leave the country.c. Lingnan University closes and there will not be a final exam in this class.
The Killer
Ah Jong is a martial arts expert. He’s a high ranking member of the Triads, and he’s killed hundreds of people. Friends describe him as “dangerous.”
The Killer
What Is Most Likely?
(a) Ah Jong sews women’s dresses.(b) Ah Jong sews women’s dresses so the police won’t think he’s a gangster(c) Ah Jong sews women’s dresses as part of his court-ordered rehabilitation.
Conjunction Fallacy
Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.Which is more probable?a. Linda is a bank teller.b. Linda is a bank teller and is active in the feminist movement.
Conjunction Fallacy
The correct answer is (a):
a. There will not be a final exam.a. Ah Jong sews women’s dresses.a. Linda is a bank teller.
Conjunction Fallacy
Suppose: 20,000 people meet the description (“31 years old, single, outspoken, and very bright. She majored in philosophy…”).
How many of them are feminist bank tellers? Pick any number: 5,000.
Conjunction Fallacy
How many of them are bank tellers (feminist or not feminist)?
It has to be more than 5,000. That’s the number of feminist bank tellers. You have to add in the number of non-feminist bank tellers who meet the description. Let’s say that’s only 1 person.
Conjunction Fallacy
Then the probability that Linda is a bank teller AND is active in the feminist movement is 5,000 out of 20,000 or 25%.
And the probability that she is a bank teller [maybe a feminist, maybe not] is 5,001 out of 20,000 > 25%.
Mathematics
Always, the probability of two events happening (Linda being a bank teller AND Linda being a feminist) is less than the probability of just one of those events happening (for example, Linda being a bank teller).
The illusion that the opposite is true especially occurs in cases where one event explains the other.
George
For example, suppose I tell you that there is a man named “George.” George turned water into wine, healed the sick, brought a dead person back to life, and came back to life himself after he died.
What is the probability that George did all these things? How likely is it?
George
You can say whatever you like. 1%, 10%, 99%.
But suppose I add to the story. I say “George was the son of God. That’s why he had all these powers.”
George
Many people will say that it’s more likely that George was the son of God AND did all these things than it is that he did all these things. But that can’t be true.
“A & B” is always less (or equally) probable than A, or than B. For A & B to happen, A has to happen and also B has to happen.
Debiasing
We can avoid this bias if we ask the question differently:
There are 100 persons who fit the description above (that is, Linda’s). How many of them are:Bank tellers? ____ of 100Bank tellers and active in the feminist movement? ____ of 100
Frequencies
This shows that it’s good to translate percentages and probabilities into frequencies (number of X out of number of Y).
We are less susceptible to representativeness bias when things are phrased in this way.
REPRESENTATIVENESS
Representativeness
Our (false) judgment that Linda is more likely to be a feminist bank teller than to just be a bank teller is an example of how we judge the truth of claims based on how “representative” they are.
Representativeness
Consider again our case of coin flips that seem non-random, due to clustering.
Since coins land 50% heads and 50% tails, “XO” and “OX” are representative of this even split, whereas “XX” and “OO” don’t represent it. So sequences with clustering seem non-random, even if they are (random).
Representativeness
Representativeness influences our other judgments as well.
It’s hard to accept that two very tall parents tend to, on average, have less tall children (as regression to the mean requires). Children who are as tall as their parents are more representative of their parents’ heights.
Heuristics and Biases
Representativeness is often a good heuristic.
A heuristic is a strategy that is easy to use in problem solving but doesn’t always work when applied.
There is often no good reason to distinguish between heuristics and biases.
Representativeness is a good heuristic (sometimes) because (sometimes) things are representative.
THE BASE RATE NEGLECT FALLACY
Base Rate Fallacy
• There are ½ million people in Russia are affected by HIV/ AIDS.
• There are 150 million people in Russia.
Base Rate Fallacy
Imagine that the government decides this is bad and that they should test everyone for HIV/ AIDS.
They develop a test with the following features:• If someone has HIV/ AIDS, then 95% of the
time the test will be positive (correct), and only 5% of the time will it be negative (incorrect).
• If someone does not have HIV/ AIDS, then 95% of the time the test will be negative (correct), and only 5% of the time will it be positive (incorrect).
The Test
If someone has HIV/ AIDS, then :• 95% of the time the test
will be positive (correct)• 5% of the time will it be
negative (incorrect)
The Test
If someone does not have HIV/ AIDS, then:• 95% of the time the test
will be negative (correct)
• 5% of the time will it be positive (incorrect)
You Get Tested
Suppose you are a Russian who gets tested for HIV/ AIDS under the government program.
The test comes out positive. How likely are you to have HIV/ AIDS?
Test = Yes Test = No
HIV/AIDS = Yes
True Positives
False Negatives
HIV/ AIDS = No
False Positives
True Negatives
True Positives
We know that there are 500,000 people in Russia who have HIV/AIDS. How many will get a positive test result?
The Test
If someone has HIV/ AIDS, then :• 95% of the time the test
will be positive (correct)• 5% of the time will it be
negative (incorrect)
True Positives
500,000 x 95% = 475,000
Test = Yes Test = No
HIV/AIDS = Yes
True Positives
False Negatives
HIV/ AIDS = No
False Positives
True Negatives
Test = Yes Test = No
HIV/AIDS = Yes
475,000 False Negatives
HIV/ AIDS = No
False Positives
True Negatives
False Negatives?
How many people who have HIV/AIDS will test negative?
500,000 – 475,000 = 25,000500,000 x 5% = 25,000
Test = Yes Test = No
HIV/AIDS = Yes
475,000 False Negatives
HIV/ AIDS = No
False Positives
True Negatives
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
False Positives
True Negatives
True Negatives
We also know that there are 150 million – 500,000 people in Russia who do not have HIV/AIDS.
How many of them will correctly test negative?
The Test
If someone does not have HIV/ AIDS, then:• 95% of the time the test
will be negative (correct)
• 5% of the time will it be positive (incorrect)
True Negatives
(150,000,000 – 500,000) x 95% = ?
True Negatives
149,500,000 x 95% = ?
True Negatives
149,500,000 x 95% = 142,025,000
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
False Negatives
True Negatives
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
False Negatives
142,025,000
False Positives
How many people who do not have HIV/AIDS will falsely test positive for the disease?
149,500,000 – 142,025,000 = 7,475,000149,500,000 x 5% = 7,475,000
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
False Positives
142,025,000
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
7,475,000 142,025,000
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
7,475,000 142,025,000
You tested positive. What percentage of people who tested positive truly had HIV/AIDS?
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
7,475,000 142,025,000
Totals
Test = Yes Test = No
HIV/AIDS = Yes
475,000 25,000
HIV/ AIDS = No
7,475,000 142,025,000
Totals 7,950,000
Your Chances
True Positives ÷ All Positives =475,000 ÷ 7,950,000 = 6%
Whether a test is good or worth doing depends not only on how accurate it is (95% true positive, 95% true negative), but also on how prevalent the condition being tested for is.
Very rare conditions require very sensitive tests, whereas very prevalent conditions only need minorly accurate tests.
Base Rates
The “base rate” is the percentage of people in the population who have a certain property.
The base rate of HIV/AIDS cases is the percentage of people who have HIV/AIDS in the population. The base rate of bearded people is the percentage of people who have beards in the population, etc.
Base Rate Neglect
The “base rate neglect fallacy” is the fallacy of ignoring the base rate when making a judgment.
For example, if I assumed you were a terrorist, because you tested positive, I would be committing the base rate neglect fallacy. I should assume you’re still probably not a terrorist.
Base Rates
As we have seen, base rates matter. If the base rate of a condition is very low (small percentage of AIDS cases), then even very accurate tests (95% true positive, 95% true negative) can be useless.
In our example only 6% of people who tested positive for AIDS had the disease!
Base Rate Neglect
The base rate neglect fallacy has nothing at all to do with the number of people there are in the world. Nothing.
It has to do with the probability of a variable taking on a certain value, for instance, the probability that someone’s height = 1.5m, the probability that someone has AIDS, etc.
Base Rates
This is the “base rate” of people who are 1.5m tall, and the “base rate” of people with AIDS.
If 1 in 100 people are terrorists, then the rate of terrorists is 1 in 100 and the probability that a randomly selected person is a terrorist is 1 in 100.
Base Rates
We call this the base rate, because it is the probability that someone is a terrorist when we don’t know anything else about them.
It might be that the base rate of terrorists is 1 in 100, but the rate of terrorists among people who are holding rocket launchers is 1 in 2, and the rate of terrorists among retirees is 1 in 500.
Tests
The base rate neglect fallacy happens when we have a test that is meant to detect the value of a variable.
For example we might have a test that tells us whether someone has AIDS or not, or whether someone is driving over the speed limit, or whether they are drunk.
Reliability of Tests
Here is the important, and crucial fact. Please learn this:
As the base rate of X = x decreases, the # of false positives on tests for X = x increases.
Tests are less reliable when the condition we are testing for becomes rare (low base rate).
Base Rate Neglect Fallacy
The base rate neglect fallacy happens when:
1. There is a low base rate of some condition.2. We have a test for that condition.3. Someone tests positive.4. We assume that means they have the
condition, ignoring the unreliability of tests for conditions with low base rates.
Base Rate Neglect
Humans have a tendency to ignore base rates.
Base Rate Neglect
John is a man wearing outstanding gothic inspired clothing with long black hair who listens to death metal.
How likely is it that he is a Christian and how likely is it that he is a Satanist?
Prosecutor’s Fallacy
The base rate neglect fallacy is often called the prosecutor’s fallacy, as I shall explain.
Murder!
Let’s suppose that there has been a murder.
There is almost no evidence to go on except that the police find one hair at the crime scene.
You are the Suspect
If someone is the killer, there is a 100% chance that their DNA will match the hair’s DNA.
The police have a database that contains the DNA of everyone in Hong Kong.
They run the DNA in the hair through their database and discover that you are a match!
Comprehension Question
If you have been following along you should be able to answer this question:
What is the probability that you are the murderer, given that you are a DNA match for the hair?
Answer
If you said 100%, then you have just committed the base rate neglect fallacy.
The correct answer is “Much lower, because the base rate of people who committed this murder out of the Hong Kong population as a whole is 1 in 7 million.”
Perfect Conditions for Fallacy
Here’s what we have:
1. A low base rate (only 1 person who committed this murder in the world).
2. A test for whether someone is the murderer.3. You, who’ve tested positive on this test.4. And the police who think you did it!
Let’s Look at the Numbers
We know that if you are the murderer, then there is a 100% chance of a DNA match.
But what is the false positive rate? How likely is a randomly selected person will match the DNA?
False Results
Here’s a quote from “False result fear over DNA tests,” Nick Paton Walsh, The Guardian:
“Researchers had asked the labs to match a series of DNA samples. They knew which ones were from the same person, but found that in over 1 per cent of cases the labs falsely matched samples, or failed to notice a match.”
Let’s assume that half of the cases where “labs falsely matched samples, or failed to notice a match.” were cases where they falsely matched samples.
So the probability of a false positive is ½ x 1% = 0.5%, or 5 in 1,000.
Since there are 7 million people in Hong Kong, we expect about 0.5% x 7 million = 35,000 of them to match the hair’s DNA.
Actually, it’s 35,000 + 1, because the true killer is a match, and not by accident.
So we expect that there are 35,001 DNA matches in all of Hong Kong.
And only one of them is the murderer. So what is the probability that you are the murderer?
1 in 35,001. That’s way less than 100%.
Important Things to Remember
There are three important things to remember:
1. If the test is more accurate (fewer false positives), then it’s more reliable
2. If the base rate is higher, the test is more reliable.
3. If the police have other reasons to suspect you, the test is more reliable.
1. If the test is more reliable…
Theoretically, DNA tests only return a false positive about 1 in 3 billion times.
In that case, we’d expect only .002 false positives in all of Hong Kong.
So your chances of being guilty would be 1 in 1.002, or 99.8%. Still, that’s lower than 100%.
2. If there base rate is higher…
Maybe the person who died was stabbed 5,000 times, once each by 5,000 different people. So there are 5,000 murderers.
Then with the previous false positive number at 35,000, you have a 5,000 in 40,000 chance of being one of the killers, or 12.5%.
3. If the police have some other reason to suspect you…
To figure out your chances of being guilty, we looked at the probability that a randomly selected person from HK would be a DNA match. We were assuming you were randomly selected.
But what if you weren’t randomly selected? What if the police tested you because you had a reason to kill the victim?
Reason to Suspect You
Then we would have to look at not the probability that a randomly selected person would match, but the probability that a person who had reason to kill the victim would match.
Suppose there are 5 people who had reasons to kill the victim, and the killer is one of them.
Much Higher Chance
Then your chances are:Let K = you’re the killer and M = you’re a matchP(K/ M) = [P(K) x P(M/ K)] ÷ P(M)= [(1/5) x 100%] ÷ P(M)= 0.2 ÷ [(1 + 0.025) ÷ 5]= 97.6%