Review

What is Critical Thinking?

There are two basic decisions to make in life:

1. Decide what to believe: What do I believe?

2. Decide what to do: What do I do?

What Should I Believe?

Is there any evidence to support the claim?

Is the evidence reliable and trustworthy? How reliable is it? Should you accept it?

Does the evidence actually support the claim?

Is there other evidence you should consider?

What Should I Do?

What outcomes can my choice lead to?

Does the outcome of my decision depend on factors other than what I choose to do?

What is the likelihood that deciding to take a specific action will lead to a specific outcome?

Which outcomes do I most prefer?

CONTEXT

The Importance of Context

When someone provides you with evidence for a truth-claim, you have to ask:

Does this evidence really support the claim?

Is there other relevant evidence I should look for before assessing the claim?

Taking Quotes out of Context

Every quote is taken out of its original context. That’s why it’s a quote and not a reproduction. This is fine if the person providing the quote has also provided enough context that the quote does not mislead you into thinking someone meant something they did not mean.

Numbers, charts, and figures can be taken out of context as well, in misleading ways.

Trends and Truncated Charts

When you see a chart describing what happened in the last 10 years, or last 10 months, or last 10 weeks, you’re seeing the information out of context. What happened before then?

When the y-axis of a chart does not start at zero (a “zero baseline”), you’re seeing what’s called a “truncated”, “torn” or “gee whiz” graph. You’re seeing the information it presents out of context.

Misleading Figures

Absolute numbers can be misleading. India has about 330,000 people in prison. A lot or a little?

Rates can also be misleading. Russia has 615 prisoners for every 100,000 people. A lot or a little?

Even comparisons can be misleading. If country X has twice as many people in prison today than in 1990, is that bad? What if they have twice as many residents than in 1990?

COGNITIVE BIASES

The Clustering Illusion

Here is 20 random coin flips:

XXXXOOXOOXXOOXOOXOOO

That doesn’t look random. But it is. The coin lands the same as the previous toss 10 times and different from the previous toss 9 times.

Confabulation

The human mind sees patterns where there is only randomness. It also freely invents reasons and explanations to “make the world make sense.”When we encounter random data, we see a pattern that isn’t there. And we explain why there should be a pattern. This can make our bad beliefs difficult to abandon.

Regression to the Mean

Whenever two variables are imperfectly correlated, extreme values of one variable tend to be paired with less extreme values of the other.

The regression fallacy involves attributing a causal explanation to what is nothing more than regression to the mean.

Confirmation Bias

Even though evaluating predictions requires looking at both the rates of true positives among predicted positives, and the rates of false positives among predicted negatives, human beings have a tendency to only consider true positives (and to a lesser extent, true negatives) when evaluating predictions or other claims of imperfect correlation.

The Problem of Absent Data

Sometimes it’s not just that we only look for or evaluate the positive evidence, but that there is no negative evidence. This can lead us to think we have very well-confirmed beliefs when we do not.

Bias

Our expectations often influence how we evaluate claims and evidence. We easily accept as true those things that we expect to be true, but are much more skeptical about things that are unexpected.

Disconfirmation Bias

Disconfirmation bias is the tendency to subject evidence against your views to a greater degree of scrutiny than evidence in favor of your views.

It is a double-standard for evidence evaluation.

Conjunction Fallacy

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.Which is more probable?1. Linda is a bank teller.2. Linda is a bank teller and is active in the feminist movement.

Conjunction Fallacy

Always, the probability of two events happening (Linda being a bank teller AND Linda being a feminist) is less than the probability of just one of those events happening (for example, Linda being a bank teller).

The illusion that the opposite is true especially occurs in cases where one event explains the other.

Representativeness

Our (false) judgment that Linda is more likely to be a feminist bank teller than to just be a bank teller is an example of how we judge the truth of claims based on how “representative” they are.The representativeness heuristic is partly responsible for other biases as well: “clusters” are not representative of random sequences, and testing positive for AIDS is not representative of not having AIDS (base rate).

Base Rates

The “base rate” is the percentage of people in the population who have a certain property.

The base rate of terrorists is the percentage of terrorists in the population, the base rate of HIV/AIDS cases is the percentage of people who have HIV/AIDS in the population, etc.

Base Rates

As we have seen, base rates matter. If the base rate of a condition is very low (small percentage of terrorists), then even very accurate tests (100% true positive, 99% true negative) can be useless. .

Base Rate Neglect

The “base rate neglect fallacy” is the fallacy of ignoring the base rate when making a judgment.

For example, if I assumed you were a terrorist, because you tested positive, I would be committing the base rate neglect fallacy. I should assume you’re still probably not a terrorist, because the base rate of terrorists is so low.

LOGIC AND FALLACIES

Arguments

Philosophical argument consists of two parts: the premises and the conclusion.

The premises are the ‘evidence’ that are given in support of the conclusion.

The conclusion is the ‘claim’ that the premises are supposed to support.

Deductive Validity

We say that an argument is deductively valid when it has the following property:

If the premises of the argument are true, then the conclusion of the argument must be true.

A valid argument is “truth-preserving”: the truth of the premises gets passed on to the conclusion.

Inductive Validity

We say that an argument is inductively valid when it has the following property:

If the premises are true, then the conclusion is likely to be true.

An inductive argument probably preserves truth.

Invalidity

An argument that is not valid is called invalid.

Valid: If the premises are true, then the conclusion must be true.

Invalid: The premises can be true while the conclusion is false.

Deductive Logic

The goal of deductive logic is to identify deductively valid argument forms. We can use these as a formal test for validity: if an argument has a certain form, then that argument is deductively valid.There is no formal test for (deductive) invalidity: no way of looking at the form of an argument and telling that the premises do not guarantee the conclusion.

Fallacies

A fallacy is an invalid argument, usually one that might mislead someone into thinking it’s valid.

Straw Man Fallacy

The Straw Man Fallacy is when you misrepresent your opponent, and argue against the misrepresentation, rather than against your opponents claim.

Assuming the Original Conclusion

Assuming the original conclusion involves trying to show that a claim is true by assuming that it is true in the premises. It has the form:

X is true. Why? Because X.

False Equivocation

Equivocation (or “false equivocation”) is when one word is used with two meanings in the same argument, rendering it invalid.

False Dilemma

An argument commits the false dilemma fallacy when it presents two options as the only options, even though there are actually more options.

Fallacy of the Mean

The fallacy of the mean is the assumption that a “middle point” between two views is the right one.

Distribution Fallacy

The distribution fallacy is committed when one assumes that individuals have the properties of groups to which they belong.

Lingnan has an excellent philosophy department.I am a philosopher at Lingnan._________Therefore, I am an excellent philosopher.

Composition Fallacy

The converse of the distribution fallacy is the composition fallacy, assuming that groups have the properties of the individuals that compose them.

For example: “A point doesn’t have any length; lines are made out of points; therefore, a line doesn’t have any length.”

Ecological Fallacy

Here’s an “ecological inference”.

Countries where, on average, people consume more fat have higher rates of breast cancer.

Therefore, there is a correlation between the amount of fat a person consumes and his/ her chances of breast cancer.

Argument from Ignorance

The argument from ignorance goes like this:

“You can’t prove that God doesn’t exist. Therefore God exists.”

It assumes that because there is no argument against a position, that that position must be correct.

Genetic Fallacy

The genetic fallacy seeks to evaluate a claim on the basis of its origin.

So, for example, someone might say, “Eugenics is wrong, because the Nazis began it and did horrible things for its sake.”

Eugenics may be wrong, but the fact that the Nazis began it is irrelevant to this claim.

Appeal to Motive

Sometimes people argue that a certain claim must be false, or an argument invalid, because of the motives of the person making the claim/ argument.

Naturalistic Fallacy

The Naturalistic Fallacy assumes that just because something is natural, it is good, and just because something is not natural it is not good. Some natural things are bad, some non-natural things are good.

THE SCIENTIFIC METHOD

Scientific Method

Last time we discussed the hypothetico-deductive method, which consisted of four steps:1. Formulate a hypothesis2. Generate testable predictions3. Gather data4. Check predictions against observations

Good Theories

A good theory should:• Have predictive power• Explain the relevant phenomenon in terms of

underlying causal mechanisms• Be fruitful• Be simple• Be coherent

Causation

Much of science is concerned with discovering the causal structure of the world.

We want to understand what causes what so we can predict, explain, and control the events around us.

Correlation

Two variables A, B that are not independent are said to be correlated.

A and B are positively correlated when P(A/ B) > P(A). If B happens, A is more likely to happen.

A and B are negatively correlated when P(A/ B) < P(A). If B happens, A is less likely to happen.

Correlation

Other relationships between variables are often called correlation as well.

A and B are positively correlated when increases in A correspond to increases in B.

A and B are negatively correlated when increases in A correspond to decreases in B.

Causation and Correlation

One thing that can lead two variables A and B to be correlated is when A causes B.

For example, if having a cold causes a runny nose, then having a cold is correlated with having a runny nose:

P(cold/ runny nose) > P(cold)

Causation ≠ Correlation

But causation does not imply correlation. If A and B are correlated there are several possibilities:

• A causes B• B causes A• C causes A and C causes B• A and B are only accidentally correlated

Types of Scientific Studies

There are two basic types of scientific studies:• Observational studies look at data in order to

determine whether two variables are correlated.

• Controlled experiments involve two groups: the experimental group, which receives the treatment we’re interested in, and the control group, which does not.

Observational Studies

Importantly, observational studies can only show whether two variables A and B are correlated. They cannot show whether A causes B, or B causes A, or some third cause causes both, or if the correlation is accidental.

Confounding Variables

A controlled experiment lets you “control for” confounding variables. You can make the control group and the experimental group have equal numbers of people from each age group.

Then you know that if more people in your experimental group die, it wasn’t due to their age (the other group had similar ages).

Controlling

In an observational study, there is no way to rule out B causing A rather than A causing B. Does wine reduce the risk of cancer, or does a lowered risk of cancer increase wine consumption?

If experimenters control who gets wine, then we can rule out the hypothesis that in our study, lowered cancer risk causes wine drinking.

Controlling

In an observational study, there is no way to rule out a common cause for two correlated variables A and B.

In an experimental study, the common cause is ruled out, because the experimenter is the one who causes (“controls”) whether people have A or not.

Randomization

Ideally, an experiment controls for as many variables as possible.

To a large extent, this is done by randomly assigning individuals in the study to either the control group or the experimental group. This way, the members of the group are less likely to share features other than chocolate eating.

Maximal Similarity

In general, the control condition and the experimental condition should be as similar as possible, and differ only in the variable being tested.For example, if the experimental group is given the happiness test by a beautiful woman and the control group is tested by a grumpy professor, that might be the real reason for a difference in scores, not the baby pictures.

“Blinds”

In experimental studies, we say that the participants are blind if they do not know which group they are in: the control group or the experimental group. It’s not always possible to have blind participants, but this is considered best practice.Blinding helps control for the placebo effect and subject bias. Non-blinded studies tend to overstate effects.

Blind Experimenters

Ideally, in experiments the researchers are blind to which group subjects are in.

This prevents the experimenter from accidentally indicating to the subjects which group they are in.

It also prevents experimenter bias.

Placebo and Nocebo

‘Placebo’ is Latin for “I will please,” and the placebo effect is when a treatment that doesn’t by itself cause any improvement leads to positive expectations in the patient that cause improvement.

‘Nocebo’ means “I will harm,” and the nocebo effect is when an inactive treatment causes harm, because we believe that it will.

The Placebo Effects

• 4 sugar pills are better than 2!• Pink sugar pills make you more alert than blue

ones.• Saltwater injections improve headaches better

than sugar pills.• Paying more money for the same drugs makes

you feel better.• Placebos in fancy packages work better…

Ethics of Placebos

There are lots of reasons that we should discourage treatment that is no better than a placebo. If the treatment is:• Expensive• Harmful• Deceptive• Used instead of effective alternatives

The Baseline

Scientists often test small groups, and then aggregate (put together) all the data later. When you have a small group of people– for example 20 or 30, there is a high probability that by random chance either the control group or the experimental group will be doing better.

This is called “the baseline.”

Controlling for the Baseline

You can “control for the baseline” by testing how much people improve over the course of the trial, instead of just testing whether they’re doing well or poorly at the end of the trial.

Null Hypothesis

The hypothesis that there is no causal connection between the variables being studied is called the null hypothesis.

Our goal is to reject the null hypothesis when it is false, and to accept it when it is true.

P-Values

One way to characterize the significance of an observed correlation is with a p-value. The p-value is the probability that we would observe our data on the assumption that the null hypothesis is true. p = P(observations/ null hypothesis = true)We say that an experimental result with p < .05 is statistically significant.

Statistical Significance

What does p < .05 mean?

It means that the probability that our experimental results would happen if the null hypothesis is true is less than 5%.

According to the null hypothesis, there is less than a 1 in 20 chance that we would obtain these results.

Importance

Just because the results of an experiment (or observational study) are “statistically significant” does not mean the revealed correlations are important.

The effect size also matters, that is the strength of the correlation.

Effect Size

In class, we considered two measures of effect sizes:If the probability that a non-smoker will get lung cancer is 2% and the probability that a smoker will get lung cancer is 10%, then the relative risk of lung cancer for smokers relative to non-smokers is 10%/2% = 5. The chance of getting lung cancer for smokers is 5 times the chance for non-smokers.

Sample

In statistics, the people who we are studying are called the sample. (Or if I’m studying the outcomes of coin flips, my sample is the coin flips that I’ve looked at. Or if I’m studying penguins, it’s the penguins I’ve studied.)

Our question is then: what sample size is needed for a result that applies to the population?

Representative Samples

The opposite of a biased sample is a representative sample.

A perfectly representative sample is one where if n% of the population is X, then n% of the sample is X, for every X.

For example, if 10% of the population smokes, 10% of the sample smokes.

Random Sampling

One way to get a representative sample is to randomly select people from the population, so that each has a fair and equal chance of ending up in the sample.

For example, when we randomize our experiments, we randomly sample the participants to obtain our experimental group. (Ideally our participants are randomly sampled from the population at large.)

Sample Size Determination

There are two questions we need to ask:

How many people do I need to study to obtain statistically significant results?

How big should my sample be to accurately estimate effect sizes in the population at large?

Sample Size Determination

If we want to know how many people to look at, we should determine:

1. What level of confidence we want2. How big we want our confidence interval to

be.

Common Choices

Common choices for these numbers are:

1. We want to be 95% confident of our estimation.

2. We want our confidence interval to be 6% wide (e.g. between 42% and 48%).

One Calculation

As we calculated in class, you need about 1,000 people to be 95% sure that the vote counts you estimate from the sample are within 6% of the actual voting behavior of the population.

Things to Note

This doesn’t mean that studies with less than a thousand people can’t tell us anything—

What they tell us will just be either less confident than 95% or have greater error bars than 6%.

If a confidence interval of 20% is fine, you only need 100 people.

The Problem of Multiple Comparisons

The problem of multiple comparisons is the fact that if you look at a data set, you will find lots of correlations completely by random chance.

Thus science prohibits “finding hypotheses in the data.” We can’t look at some data and see if there are any correlations. We have to pick a correlation we’re looking for in advance, and see if it exists in new data.

Why New Data Is Important

It would be really unlikely if (i) I propose a correlation(ii) I test it against some new data(iii) The new data confirm the correlation(iv) All of that was just an accident

Compare this to the fact that it is really likely to find random correlations in the data.

Meta-Analysis

A meta-analysis is an analysis of analyses.

In clearer terms, it is a study that looks at lots of different experiments that have been conducted on the same problem, and tries to “put together” all of the findings.

Cherry Picking

Cherry picking is another name for “the fallacy of incomplete evidence,” and it’s related to confirmation bias and selection bias.

If I want to prove that a treatment works and I pick only those studies that are positive and ignore lots and lots of negative studies, then I’m “cherry picking” my studies.

“Systematic Review”

In the “bad old days” (before the 1980s) review articles were unsystematic, meaning that people writing the reviews included some studies that were relevant, but not all. This resulted in cherry picking: later it was shown that systematic meta-analyses often had the opposite conclusions of unsystematic reviews. A systematic review looks at all the relevant studies, not just some.

The File Drawer Problem

A second and more difficult problem for literature reviews is that the published literature is a biased sample of all the studies that have been done. As we saw last time, there is publication bias against negative results: people tend to publish positive findings, but to leave negative findings unpublished, sitting in the “file drawers” of their offices (metaphorically).

Authority

That’s impossible: we all need to trust other people for knowledge about lots of things.

I’m not a climate scientist, I don’t have stations that collect and analyze data about global temperatures, so I can’t determine myself whether global warming is happening. I have to trust an authority to tell me if it is.

Appeal to Authority

Sometimes you’ll hear about the “fallacy of appealing to authority.” But basing your views off on appeals to authority is not always fallacious. Your authority needs to be:

• An expert, or better, an agreeing group of experts, on the subject in question.

• A proven truth-teller on the subject.

Credentials

One way society has of distinguishing genuine authorities from other people is via credentials.

A person’s authority can be certified by their having certain degrees, like a PhD or MD (doctor of medicine). A person’s ideas can have their authority certified by being published in a top peer-reviewed scientific journal.

Problems with Newspapers

Editorials are often uninformed opinions.Most reported-on scientific studies are false.Most newspaper stories are not written by reporters. Many are written by private companies for commercial purposes.News creators (government, scientists) often require news sources to only give their side.He-said she-said reporting often results in a false equivalence between opposing views.

DECISION THEORY