Slide 1

Slide 2

Sampling & External Validity 1 KNR 497 Research Methods Sampling Slide 1 Chapter 2 part 2

Slide 3

2 1 KNR 497 Research Methods: Sampling Slide 2 The 65, 95, 99 Percent Rule

Slide 4

KNR 497 Research Methods: Sampling Slide 3 Estimating the Population Using a Sampling Distribution 2 1

Slide 5

1 The rest of the slides Types of sampling Probability based Non-probability based KNR 497 Research Methods: Sampling Slide 4 2

Slide 6

3 1 KNR 497 Research Methods: Sampling Slide 5 Probability Sampling Utilizes some form of random selection All units in the population have equal probability of being chosen Nomenclature: N = number of cases in the sampling frame n = number of cases in the sample NCn = number of combinations of n from N f = n/N and is the sampling fraction 2 4

Slide 7

KNR 497 Research Methods: Sampling Slide 6 Probability Sampling Simple random sampling Stratified random sampling Systematic random sampling Cluster (area) random sampling Multistage sampling 1

Slide 8

KNR 497 Research Methods: Sampling Slide 7 Probability Sampling: Simple Random Sampling Objective: To select n units out of N such that each NCn has an equal chance of being selected Procedure: Use a table of random numbers or computer random- number generator Example: N = 1000 n (desired) = 100 f = n/N = 100/1000 =.10 or 10% Randomly select 100 units (10%) Generalizable; may not be representative of subgroups 3 1 2

Slide 9

KNR 497 Research Methods: Sampling Slide 8 Probability Sampling: Stratified Random Sampling Objective: To select n units out of N such that key subgroups of n are representative of subgroups of N Procedure: Divide the population into nonoverlapping groups (strata) N 1, N 2, N 3 N i, such that N 1 + N 2 + N 3 + N i = N. Then do simple random sample of f = n/N in each strata Disproportionate stratified random sampling can be used to oversample small groups. 3 1 2 4

Slide 10

KNR 497 Research Methods: Sampling Slide 9 Probability Sampling: Systematic Random Sampling Objective: To systematically select n units out of N such that n is a random sample of N Procedure: Number units in the population from 1 to N (NOTE: Units must be randomly ordered) Decide on the n that you need Calculate k = N/n = the interval size Randomly select an integer between 1 and k Take every kth unit (diagram on next slide illustrates this) 1 2

Slide 11

KNR 497 Research Methods: Sampling Slide 10 Probability Sampling: Systematic Random Sampling 1

Slide 12

KNR 497 Research Methods: Sampling Slide 11 Probability Sampling: Cluster (Area) Random Sampling Objective: To obtain a representative sample from N when N is spread out over a large geographic area Procedure: Divide the population into clusters Randomly sample clusters Measure all units within sampled clusters Clusters are usually divided along geographical boundaries. 3 1 2

Slide 13

KNR 497 Research Methods: Sampling Slide 12 Probability Sampling: Multistage Sampling Objective: To obtain a representative sample from N by combining several sampling techniques to create a more efficient or effective sample than the use of any one sampling type can achieve on its own Example: 1. National sample of school districts stratified by economics 2. Simple random selection of schools within districts 3. Simple random selection of classes within schools 3 1 2

Slide 14

KNR 497 Research Methods: Sampling Slide 13 Nonprobability Sampling Does not involve random selection May be representative but cannot depend upon the rationale of probability theory Used when it is not feasible, practical, or theoretically sensible to use random sampling Accidental versus purposive 3 1 2 4

Slide 15

Nonprobability Sampling: Accidental, Haphazard, or Convenience Sampling One of the most common methods of sampling Man on the street Volunteers or subjects who are conveniently available No evidence that sample is representative KNR 497 Research Methods: Sampling Slide 14 1 2

Slide 16

KNR 497 Research Methods: Sampling Slide 15 Nonprobability Sampling: Purposive Sampling Sampling with a purpose in mind Useful in reaching a targeted sample quickly Target population is reached but with over- representation of subgroups that are more readily accessible Types: Modal instance Expert Quota Heterogeneity Snowball 3 1 2

Slide 17

KNR 497 Research Methods: Sampling Slide 16 Nonprobability Sampling: Purposive Sampling Modal Instance Sampling the most frequent case or typical case Difficult to define what a typical case is Probably only useful for informal sampling contexts (or perhaps even more dangerous for those) 1 2 3

Slide 18

KNR 497 Research Methods: Sampling Slide 17 Nonprobability Sampling: Purposive Sampling Expert Sampling Assembling of a sample of persons with known or demonstrable expertise in some area Panel of experts May be useful for providing evidence as to the validity of another sampling approach you have chosen 1 2

Slide 19

KNR 497 Research Methods: Sampling Slide 18 Nonprobability Sampling: Purposive Sampling Quota Sampling Sample selected nonrandomly according to some fixed quota Proportional quota sampling used to represent the major characteristics of the population of interest by sampling a proportional amount of each Nonproportional quota sampling used to supply a minimum number of units in each category but not concerned with proportions 1 2

Slide 20

KNR 497 Research Methods: Sampling Slide 19 Nonprobability Sampling: Purposive Sampling Heterogeneity Sampling Used to provide a sample that will include all the view or opinions without regard to proportional representation Sampling for diversity Can be thought of as the opposite of modal instance sampling 1 2

Slide 21

KNR 497 Research Methods: Sampling Slide 20 Nonprobability Sampling: Purposive Sampling Snowball Sampling People meeting the criteria for inclusion in the sample are identified and then they recommend others they know who meet the criteria Useful when trying to reach inaccessible or hard to find populations Examples may include the homeless, drug users, etc. 1 2

Slide 22

Threats to External Validity Interaction of selection and treatment Interaction of setting and treatment Interaction of history and treatment Maybe it is just these people. Maybe it is just these places. Maybe it is just these times.

Slide 23

Guiding Questions for Critiquing the External Validity of Research 1. What are the main results of the study (e.g., positive or negative relationship, group differences, effectiveness of the intervention or treatment)? 2. Do the researchers explicitly state or imply that similar results would hold for other: (a) people, (b) places or situations, and/or (c) times? If so, what is the population/place/time they are attempting to generalize to? 3. If the researchers are generalizing their results, how reasonable are these conclusions given the sample, sampling procedures, and settings used? [This is the key External Validity question] 4. What specifically might lead you to question these conclusions? In other words, if they did suggest the results were generalizable, why might you think otherwise? [The more convincing of a rationale you can generate, the more you should question the external validity]

Slide 24

Use the guiding questions to evaluate the external validity of the following study Prior research has found that (a) intercollegiate athletes are especially at-risk for excessive alcohol consumption (e.g., Nelson & Wechsler, 2001), and (b) sport-type differences exist among college athletes in terms of yearly drinking prevalence rates (National Collegiate Athletic Association, 2001). No studies, however, have examined sport-type differences on more specific measures of alcohol consumption (i.e., drinks per week). In the present study, data were analyzed on 298 intercollegiate athletes from two different NCAA Division III universities. Results indicated significant sport type differences on alcohol consumption variables, with athletes from the sports of swimming and diving and wrestling reporting the highest levels of alcohol consumption (M = 5.20, SD = 4.00) and soccer and football reporting the lowest (M = 4.02, SD = 3.25). Results suggest college athletes participating in individual sports are at-risk for future alcohol abuse.