Copyright c 2001 The McGraw-Hill Companies, Inc. 1

Chapter 7

Sampling, Significance Levels, and Hypothesis Testing

Three scientific traditions critical to experimental research Sampling

Significance levels

Hypothesis testing

Copyright c 2001 The McGraw-Hill Companies, Inc. 2

Population and Sample

Population – all units (people or things) possessing the attributes and characteristics of interestSample -- subset of a populationSampling frame -- subset of units that have a chance to become part of the sampleResearchers study the sample to make generalizations back to the population

Copyright c 2001 The McGraw-Hill Companies, Inc. 3

Defining the Population

Choose the dimensions or characteristics meaningful to the hypothesis or research question

Must be at least one common characteristic among all members of a population

Must develop procedure to ensure representative sampling

Copyright c 2001 The McGraw-Hill Companies, Inc. 4

Addressing Generalizability

Extent to which conclusions developed from data collected from sample can be extended to its populationSample is representative to the degree that all units had same chance for being selectedRepresentative sampling eliminates selection bias Characteristics of population should appear to the same

degree in sample

Representativeness can only be assured through random sampling

Copyright c 2001 The McGraw-Hill Companies, Inc. 5

Probability Sampling

The probability of any unit being included in the sample is known and equal

When probability for selection is equal, selection is random

Also known as random sampling

Sampling error will always occur

Copyright c 2001 The McGraw-Hill Companies, Inc. 6

Types of Probability Sampling

Simple random sampling Simplest and quickest

Systematic sampling If used on a randomly ordered frame, results in truly

random sample

Stratified random sampling Random sampling within all subgroups

Cluster sampling Random sampling within known clusters

Copyright c 2001 The McGraw-Hill Companies, Inc. 7

Nonprobability Sampling

Does not rely on random selectionWeakens sample-to-population representativenessUsed when other techniques will not result in an adequate or appropriate sampleUsed when researchers desire participants with special experiences or abilities – including qualitative research

Copyright c 2001 The McGraw-Hill Companies, Inc. 8

Nonprobability Sampling Techniques

Convenience sample

Volunteer sample

Inclusion/exclusion sample

Snowball sample

Network sample

Purposive sample

Quota sample

Copyright c 2001 The McGraw-Hill Companies, Inc. 9

Sample Size

Number of people/units for whom you need to collect data

Determined prior to selecting sample

Less than the number you ask to participate

The larger the sample relative to the population, the less error or bias

Copyright c 2001 The McGraw-Hill Companies, Inc. 10

Comparisons of Sample Size to Population

Population Size

Sample Size

Population Size

Sample Size

100 80 1,000 278

200 132 5,000 357

500 217 50,000 384

Copyright c 2001 The McGraw-Hill Companies, Inc. 11

Significance Levels

The researcher sets the significance level, or p, for each statistical test

The degree of error the researcher finds acceptable in a statistical test

An estimate of what would happen if the study were actually repeated many times

Generally .05 is accepted level

Copyright c 2001 The McGraw-Hill Companies, Inc. 12

Significance Levels

.05 significance level = 5 out of 100 findings that appear to be valid will be due to chance

Also known as the alpha level or p

If p > .05, the finding is nonsignificant

If p is .05, the finding is significant or real

Copyright c 2001 The McGraw-Hill Companies, Inc. 13

Hypothesis Testing

Hypothesis states the expected relationship or difference between two or more variables

Alternative hypothesis presented in report

Null is statistically tested Act of decision making based on the

significance level Decision based on comparison between p set

before study to p produced by statistical test

Copyright c 2001 The McGraw-Hill Companies, Inc. 14

Hypothesis Testing

Belief in the null hypothesis continues until there is sufficient evidence to the contrary

If p for statistical test exceeds significance level, null is retained (p > .05)

If p for statistical test is .05 then alternative hypothesis is accepted

Copyright c 2001 The McGraw-Hill Companies, Inc. 15

Error in Hypothesis Testing

In reality, the null hypothesis is true

In reality, the null hypothesis is false

Use level of significance to reject null

Type I error – Null is rejected even though it is true

Decision 1 – Null is rejected when it is false

Use level of significance to retain the null

Decision 2 – Null is retained when it is true

Type II error – Null is retained even though it is false