Chapter 2 highlightsAssociation and Causation

2

Many interesting examples of the use of statistics involve relationships between pairs of variables.

Two variables measured on the same cases are associated if knowing the value of one of the variables tells you something that you would not otherwise know about the value of the other variable.

2

Associations Between Variables

Height and weight of same individual Smoking habits and life expectancy Age and bone-density of individuals Gender and political affiliation Gender and Smoking

3

Caution: Often there are spurious, other variables lurking in the background Shorter women have lower risk of heart attack Countries with more TV sets have better life expectancy

rates More deaths by drowning occur when ice cream sales

peak

3

Does Association = Causation?

Is your purpose simply to explore the nature of the relationship, or do you wish to show that one of the variables can explain variation in the other?

4

Key Characteristics of a Data Set

Certain characteristics of a data set are key to exploring the relationship between two variables. These should include the following:

Cases: Identify the cases and how many there are in the data set.

Label: Identify what is used as a label variable if one is present.

Categorical or quantitative: Classify each variable as categorical or quantitative.

Values: Identify the possible values for each variable.

Explanatory or response: If appropriate, classify each variable as explanatory or

response.

A response variable measures an outcome of a study. An explanatory variable explains or causes changes in the response variable.

Examples

Different amount of alcohol given to mice, body temperature noted (belief: drop in body temperature with increasing amount of alcohol)◦ Response variable?◦ Explanatory variable?

SAT scores used to predict college GPA◦ Response variable?◦ Explanatory variable?

Association, however strong, does NOT imply causation.

Some possible explanations for an observed association

The dashed lines show an association. The solid arrows show a cause-

and-effect link. x is explanatory, y is response, and z is a lurking variable.

Explaining Association: Causation

6

Causation Association does not imply causation!◦ An association between x and y, even if it is very

strong, is not itself good evidence that changes in x actually cause changes in y.

Causation: Variable X directly causes a change in Variable Y

Example:◦ X = plant food◦ Y = plant’s growth

Common Response Other variables may affect the relationship

between X and Y Beware of lurking variables

Example: for children, ◦ X = height ◦ Y = Math Score◦ Z =

Confounding Other variables may affect the relationship

between X and Y Can’t separate effects of X and Z on Y

Example:◦ X = number years of education◦ Y = income◦ Z =

3.1 Sources of Data

Anecdotal data Available data Sample surveys and experiments Observation vs. experiment

10

Exploratory Data Analysis vs. Statistical Inference

Exploratory Data Analysis◦ reveals interesting features of data but may not be

sufficient for definite conclusions – more graphical What is the credibility of what I see? How trustworthy are my findings?

Statistical Inference ◦ gives answers with a controlled degree of confidence –

more numerical Relies heavily on properly collected data

Obtaining Data Available data are data that were produced in the past for some

other purpose but that may help answer a present question inexpensively. The library and the Internet are sources of available data.

Some questions require data produced specifically to answer them. This leads to designing observational or experimental studies.

Trustworthy data Trustworthy Inference

Anecdotal Evidence

Beware of drawing conclusions from our own experience or hearsay. Anecdotal data represent individual cases that often come to our attention because they are striking in some way. We tend to remember these cases because they are unusual. The fact that they are unusual means that they may not be representative of any larger group of cases.

“Smoking isn't harmful. My grandmother smoked a pack a day and lived to age 92.

“Last week an airplane crashed and 113 people were killed. So, I’m driving home for break instead of flying.”

14

Sample Surveys

Sample surveys are a special type of designed experiment that usually aim to discover the opinions of people on certain topics.

In a sample survey, a sample of individuals is selected from a larger population of individuals.

One can study a small part of the population in order to gain information about the population as a whole. Example: Gallup poll – originally sampled 3000 adult residents of

the U.S. to represent the entire U.S. population

Conclusions drawn from a sample are valid only when the sample is drawn in a well-defined way, to be discussed in Section 3.3.

Observation vs. Experiment

An observational study observes individuals and measures variables of interest but does not attempt to influence the responses. The purpose is to describe some group or situation.

An experiment deliberately imposes some treatment on individuals to measure their responses. The purpose is to study whether the treatment causes a change in the response.

When our goal is to understand cause and effect, experiments are the only source of fully convincing data.

The distinction between observational study and experiment is one of the most important in statistics.

15

Big advantage Observational studies cannot show causation.◦Example: smoking and lung cancer◦Could there be another lurking variable such as genetics,

which causes a person to both want to smoke and develop lung cancer?

However, experiments can give good evidence for causation. Why?◦Designed to answer specific questions◦Can control factors and levels

Then why have observational studies?◦Can we assign people to smoke?

17

Confounding

Observational studies of the effect of one variable on another often fail because of confounding between the explanatory variable and one or more lurking variables.

A lurking variable is a variable that is not among the explanatory or response variables in a study but that may influence the response variable.

Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.

Well-designed experiments take steps to avoid confounding.

3.2 Design of Experiments

Experimental units, subjects, treatments Comparative experiments Bias Principles of experimental design Statistical significance Matched pairs design Block design

18

19

Individuals, Factors, Treatments

An experiment is a study in which we actually do something (a treatment) to people, animals, or objects (the experimental units) to observe the response. Here is the basic vocabulary of experiments.

An experimental unit is the smallest entity to which a treatment is applied. When the units are human beings, they are often called subjects.

The explanatory variables in an experiment are often called factors.

A specific condition applied to the individuals in an experiment is called a treatment. If an experiment has several explanatory variables, a treatment is a combination of specific values of these variables.

20

Principles of Experimental Design

Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the differences we see in the responses.

1. Control for lurking variables that might affect the response, most simply by comparing two or more treatments. (Use a placebo, etc)

2. Randomize: Use chance to assign experimental units to treatments.

3. Replication: Use enough experimental units in each group to reduce chance variation in the results.

Principles of Experimental Design

Other things to consider… Use a placebo if possible◦ “placebo effect” is when a patient believes the treatment will work

and this affects the results I.E. an improvement in health is due to the patient’s belief, not the

treatment itself. Because they believe they will get better they actually do!

◦ Using a placebo in a experiment eliminates the placebo effect Double-blind◦ Neither the subjects nor those who interact with them and measure

the response variable know which treatment a subject received.

Lack of Realism◦ The most serious potential weakness of experiments is lack of

realism. The subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study.

Comparative Experiments We will look at 3 Experimental Designs

◦ Completely Randomized Design◦ Matched-Pairs Design◦ Block Design

23

In a completely randomized design, the treatments are assigned to all the experimental units completely by chance.

Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment. (Control or Placebo, etc)

Experimental units

Random assignment

Group 1

Group 2

Treatment 1

Treatment 2

Compare resul

ts

Randomized Comparative Experiments

Example: Rats’ diets (CR)

Measure weight gain of rats under new diet. Thirty rats are used.

The simplest randomized comparative design: assign rats completely randomly to treatments

Matched pairs designs Only two treatment groups.◦ Sometimes 1 treatment, 1 control

Try to pair up individuals that are as similar as possible◦ Age, sex, income, etc.

Separate each pair, placing one in each group. Done randomly.

Concept: Try to eliminate differences between the two groups since each is made up of relatively the “same” individuals

Ideal: Identical twins◦ Put one twin in group 1 and one twin in group 2, then any

differences between the two twins should be due to the treatment alone!

26

Blocked Designs

Used when you know that different groups (called blocks) will respond differently to the treatments• Hormone studies—blocks are female and male. Why?

A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments.

In a block design, the random assignment of experimental units to treatments is carried out separately within each block.

Form blocks based on the most important unavoidable sources of variability (lurking variables) among the experimental units.

Randomization will average out the effects of the remaining lurking variables and allow an unbiased comparison of the treatments.

Example: Cancer therapy (Block)

The progress of a type of cancer differs in women and men. A clinical experiment to compare three therapies for this cancer therefore treats sex as a blocking variable. Two separate randomizations are done, one assigning the female subjects to the treatments and the other assigning the male subjects.

Review Examples: Aspirin and Beta Carotene (CR) Does regularly taking aspirin help protect people against

heart attacks? The Physicians’ Health Study was a medical experiment that helped answer this question. The study looked at the effects of two drugs, aspirin and beta carotene, on 21,996 male physicians. It looked at all possible combinations of the drugs. An equal number of subjects was assigned to each treatment.

What are the experimental units? What are the factors? What are the treatments? How many are there?

Example: Aspirin and Beta Carotene (CR)

Review Examples: Electricity Use Many utility companies have programs to encourage their

customers to conserve energy. An electric company is considering placing electronic meters in households to show what the cost would be if the electricity use at that moment continued for a month. Will meters reduce electricity use? Would cheaper methods work almost as well? The company decides to design an experiment.

One cheaper approach is to give customers a chart and information about monitoring their electricity use. The experiments compares these two approaches (meter, chart) with each other and also a control group of customers who receive no help in monitoring electricity use. The response variable is total electricity used in a year. The company finds 60 single-family residences in the same city willing to participate.

Why is this an experiment? What type of experiment is this? Outline the experiment.

Statistical Significance We hope to see a difference between groups But just any difference is not enough!

Example: ◦ Vitamin C group has an average of 3 colds/yr◦ Placebo group has an average of 4 colds/yr◦ Statistical significant?

We will (eventually) learn where this cutoff is

An observed effect so large that it would rarely occur by chance is called statistically significant.

A statistically significant association in data from a well-designed experiment does imply causation.

3.3 Sampling Design

Population and sample Voluntary response sample Simple random sample Stratified samples Undercoverage and nonresponse

33

The distinction between population and sample is basic to statistics. To make sense of any sample result, you must know what population the sample represents.

34

Population and Sample

The population in a statistical study is the entire group of individuals about which we want information.

A sample is the part of the population from which we actually collect information. We use information from a sample to draw conclusions about the entire population.

Population

Sample

Collect data from a representative Sample...

Make an Inference about the Population.

35

Simple Random Samples

Random sampling, the use of chance to select a sample, is the central principle of statistical sampling.

A simple random sample (SRS) of size n consists of n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be the sample actually selected.

In practice, people use random numbers generated by a computer or calculator to choose samples. If you don’t have technology handy, you can use a table of random digits. More later…

36

Other Sampling Designs

To select a stratified random sample, first classify the population into groups of similar individuals, called strata. Then choose a separate SRS in each stratum and combine these SRSs to form the full sample.

Sometimes, there are statistical advantages to using more complex sampling methods. One common alternative to an SRS involves sampling important groups (called strata) within the population separately. These “sub-samples” are combined to form one stratified random sample.

Another common means of restricting random selection is to choose the sample in stages. These designs are called multistage designs. They are still usually random at each stage!

Other probability sampling:

37

How to Sample Badly

The design of a sample is biased if it systematically favors certain outcomes.

A voluntary response sample consists of people who choose themselves by responding to a general appeal. Voluntary response samples often show bias because people with strong opinions (often in the same direction) may be more likely to respond.

Choosing individuals simply because they are easy to reach results in a convenience sample.

38

Cautions About Sample Surveys

Good sampling technique includes the art of reducing all sources of error.

Undercoverage occurs when some groups in the population are left out of the process of choosing the sample.

Nonresponse occurs when an individual chosen for the sample can’t be contacted or refuses to participate.

A systematic pattern of incorrect responses in a sample survey leads to response bias.

The wording of questions is the most important influence on the answers given to a sample survey.

39

How to Choose an SRS

A table of random digits is a long string of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 with these properties: Each entry in the table is equally likely to be any of the 10 digits 0–

9. The entries are independent of one another. That is, knowledge of

one part of the table gives no information about any other part.

Step 1: Label. Give each member of the population a numerical label of the same length.

Step 2: Table. Read consecutive groups of digits of the appropriate length from Table B.

Your sample contains the individuals whose labels you find.

How to Choose an SRS Using Table B

SRS Example using line 130

3.4 Toward Statistical Inference

Parameters and statistics Sampling variability Sampling distribution Bias and variability Sampling from large populations

42

As we begin to use sample data to draw conclusions about a wider population, we must be clear about whether a number describes a sample or a population.

43

Parameters and Statistics

A parameter is a number that describes some characteristic of the population. In statistical practice, the value of a parameter is not known because we cannot examine the entire population.

A statistic is a number that describes some characteristic of a sample. The value of a statistic can be computed directly from the sample data. We often use a statistic to estimate an unknown parameter.

Sampling distributions:

Example: In a survey of 2500 adults, 1650 admit shopping is frustrating◦ Parameter: p = true proportion of adults who find

shopping frustrating◦ Statistic: p-hat = 1650/2500 = 0.66

Take another such sample, you will get a different estimate. How variable are they?

Take more such samples Observe outcomes Make histogram to see center and

spread

Good news: very often the distribution of a sample statistic is well concentrated about its center, which may happen to be the population parameter.

(Mental??) experiment:

Example

We take many samples of size n=100 from a population with parameter p=0.60

The distribution of the sample proportion is given

Now take many samples with size n=2500, distribution of the proportion is much tighter

What does this say about sample size and the effects on variation of p-hat?

Normal quantile plot for the distribution of the statistics calculated based samples n=100

What does this say about the distribution of sample statistics?

We can think of the true value of the population parameter as the bull’s-eye on a target and of the sample statistic as an arrow fired at the target. Bias and variability describe what happens when we take many shots at the target.

Bias concerns the center of the sampling distribution. A statistic used to estimate a parameter is unbiased if the mean of its sampling distribution is equal to the true value of the parameter being estimated. It is on target!

The variability of a statistic is described by the spread of its sampling distribution. This spread is determined by the sampling design and the sample size n. Statistics from larger probability samples have smaller spreads.

49

Bias and Variability

Which is which?I. High Bias, High

Variability

II. High Bias, Low Variability

III. Low Bias, High Variability

IV. Low Bias, Low Variability

A good sampling scheme must have both small bias and small variability.

To reduce bias, use random sampling.

To reduce variability of a statistic from an SRS, use a larger sample.

51

Managing Bias and Variability

The variability of a statistic from a random sample does not depend on the size of the population, as long as the population is at least 100 times larger than the sample.

Variability of a statistic does not depend on the size of the population, only on the size of a sample!!!

52

Why Randomize?

The purpose of a sample is to give us information about a larger population. The process of drawing conclusions about a population on the basis of sample data is called inference.

Why should we rely on random sampling?

1. To eliminate bias in selecting samples from the list of available individuals.

2. The laws of probability allow trustworthy inference about the population.

Results from random samples come with a margin of error that sets bounds on the size of the likely error.

Larger random samples give better information about the population than smaller samples.

3.5 Ethics

Basic data ethics Institutional review boards Informed consent Confidentiality Clinical trials Behavioral and social science experiments

53

Basic Data Ethics

The most complex issues of data ethics arise when we collect data from people.

Basic Data Ethics

The organization that carries out the study must have an institutional review board that reviews all planned studies in advance in order to protect the subjects from possible harm.

All individuals who are subjects in a study must give their informed consent before data are collected.

All individual data must be kept confidential. Only statistical summaries for groups of subjects may be made public.

54

Institutional Review Boards The organization that carries out the study must

have an institutional review board that reviews all planned studies in advance in order to protect the subjects from possible harm.

The purpose of an institutional review board is “to protect the rights and welfare of human subjects (including patients) recruited to participate in research activities.”

The institutional review board:Reviews the plan of studyCan require changesReviews the consent formMonitors progress at least once a year

55

Informed Consent All subjects must give their informed consent

before data are collected. Subjects must be informed in advance about the

nature of a study and any risk of harm it might bring.

Subjects must then consent in writing. Who can’t give informed consent?

Prison inmatesVery young childrenPeople with mental disorders

56

Confidentiality All individual data must be kept confidential. Only

statistical summaries may be made public. Confidentiality is not the same as anonymity.

Anonymity prevents follow-ups to decrease non-response or inform subjects of results.

Separate the identity of the subjects from the rest of the data immediately!

Example: Citizens are required to give information to the government (tax returns, social security contributions). Some people feel that individuals should be able to forbid any other use of their data, even with all identification removed.

57

Behavioral and Social Science Experiments

Many behavioral experiments rely on hiding the true purpose of the study.

Subjects would change their behavior if told in advance what investigators were looking for.

The “Ethical Principles” of the American Psychological Association require consent, unless a study merely observes behavior in a public space.

58

Chapter ReviewExamples: page 1

In a study to assess the impact of aerobic exercise on weight loss for Lafayette residents, a physical education student selected the first 20 people who signed up for a University sponsored Wellness program. At the start of the study, each of the study participants was weighed. At the end of the 5 week program, each participant was asked for the average number of hours per week they did aerobic exercises. At this time, the 20 people were weighed again. The amount of weight lost by each person was determined.◦ What is the experimental unit?◦ What is the population?◦ What is the sample?◦ What is the response variable?◦ Explain how you would improve the design of this study.

More Examples: page 2

A study was conducted to compare the overall length of two species of lake trout, labeled A and B. Ten fish from each species were randomly selected from each of three lakes and their respective lengths were recorded.◦ What is the experimental unit?◦ What is the population?◦ What is the sample?◦ What is the response variable?◦ Is this an experiment or an observational study?

More Examples: page 3 Three methods of group-encounter techniques were

compared with respect to the level of group interaction achieved. Twenty-one group leaders from around the country were randomly chosen to participate in the study. The study was conducted at one training camp. Seven group leaders were randomly assigned to method 1, seven to method 2, and seven to method 3. After one session, the group leaders were scored on their respective abilities to achieve meaningful group interaction.◦ What is the response variable?◦ What is the experimental unit?◦ Is this an experiment or observational study?◦ What type of experimental design is this?◦ What kind of sample design did they use?

More Examples: page 4 A ________ is a number describing a sample

whereas a ________ is associated with a population.

“Statistics obtained from relatively large samples must always be closer to the population parameter they estimate than statistics obtained from smaller samples.” True or False? Explain your answer.