Estimation and Hypothesis Testing

Now the real fun begins

Inferential Statistics

Making inferences about the population based on sample measurements

Requires interpretation◦Not merely observing and describing

Sampling

oNeed to recruit a (hopefully representative) sample to get started

oSampling methods:oSimple random samplingoEveryone in population has = chance of being recruited

oSystematic samplingoSelecting participants from a preexisting list using a sampling

fraction (Ex: every 3rd person on a dorm roster)oCluster samplingoGroups of people are recruited as single units (ex: classrooms,

sororities, etc.)oGroup performance evaluated rather than individual scores

oStratified random samplingoFirst define subgroups (strata)

o Ex: college majorsoRecruit so as to ensure all strata represented in sample

Chain of Reasoning in Inferential Statistics

1. To draw inferences about the parameter based on an estimate from a sample, sample must have been created with random selection

2. Sample estimate must be compared to an underlying distribution of estimates (all possible outcomes)from all other samples of the same size that could be selected from the population

3. We can draw reasonable conclusions about the parameters based on such comparisons and the probability of outcomes achieved using ramdon sampling

Sampling Distribution of the Mean

Distribution of all possible sample means for all samples of a given size randomly selected from population

In practice, we deal with theoretical sampling distributions based on central limit theorem◦bigger sample size sampling distribution more

closely resembles normal distributionStandard deviation of sampling distribution

= standard error of the mean

Generalizations about Sampling Distribution of the Mean

1. As sample size gets larger, one observes less variability in the sampling distribution of the mean (standard error decreases)

2. Even in a non-normally distributed population, the sampling distribution still more closely resembles the normal distribution with a larger sample size

Hypotheses

We get to use inferential statistics for hypothesis testing◦Drawing conclusions about the populations based on

observations of a sampleHypothesis = educated guess, reasonable

prediction about some phenomenon at work in the population

Null hypothesis (H0) = what we actually test with our statistical procedures◦States that there is no relationship between the

variables or no difference between the groups

The dreaded P Word

So, despite common language (even in “science” shows or articles), we do not PROVE a hypothesis

Instead, we set out to disprove or refute the null hypothesis

Refuting the null allows us to say we have support for our hypothesis (the alternative hypothesis, Ha)◦It is still hardly “proven”

Coming up with a hypothesis

InterestFeasibilityRelevanceFalsifiability Replicating previous workOperational definitions

Research hypothesis versus statistical hypothesis

Research hypothesis: educated prediction about relationships between study variables based on past research

Statistical hypothesis: expected result of your specific statistical test

Sometimes, much work is needed to get from a research hypothesis to a statistical hypothesis

Need precise operational definitions!

Errors in Hypothesis Testing

Type I Error◦ Rejecting the null hypothesis when it is indeed true◦ Interpreting your results as supporting your

hypothesis when there really is no relationship or difference

◦ False positive Type II Error

◦ Failing to reject the null when it is indeed false◦ Interpreting your results as showing no support for

your hypothesis when it is actually a sound hypothesis

◦ False negative, failing to detect a genuine phenomenon

Level of Significance

When can you reject the null?How different do the scores have to be

before you can say you’ve found some evidence for your hypothesis?

Conventionally, we go by the level of significance or alpha(α) level◦Probability of making a Type I error◦Lower probability = Higher certainty you can

trust findings that support the alternative hypothesis

◦Standard in the field: .05 or .01

Steps in testing H0

1. State the hypothesis2. Set the criterion for rejecting H0

3. Compute the test statistic4. Decide whether to reject H0.

t Distributions

t distributions: A family of distributions that are symmetrical, bell-shaped, and centered on the mean, but changes for each sample of a certain size

Degrees of freedom (df): number of obs minus number of restrictions◦Freedom to vary◦As number df increases, difference between t

distribution and normal distribution decreases

Statistical Significance versus Practical Significance

A finding can be “significant” but not necessarily interesting, important, or meaningful in real-world settings