Slide 1

Research Design Part II: Cross-sectional and Quasi-Experimental Designs Frankfort-Nachmias & Nachmias (Chapter 6 Cross-Sectional and Quasi-Experimental Designs) Gerring (Chapter 8) Campbell and Stanley, Experimental and Quasi-Experimental Designs for Research. (remainder) King, Keohane and Verba (Chapter 5, sections 5.2 - 5.6) Applications Stack, S. and Gunlach, J. (1992) The Effect of Country Music on Suicide. Social Forces 71: 21118.The Effect of Country Music on Suicide. Lawrence S. Rothenberg; Mitchell S. Sanders, Severing the Electoral Connection: Shirking in the Contemporary Congress. American Journal of Political Science, Vol. 44, No. 2. (Apr., 2000), pp. 316-325. (Difference-in- difference) Severing the Electoral Connection: Shirking in the Contemporary Congress. Slide 2 Writing a Literature Review Typical format for research article Introduction Literature Review Theory Research Design Empirical Analysis and Results Conclusion Slide 3 Writing a Literature Review Purpose of literature review Inform reader of prior relevant work Persuade reader that your work is important (justify your research) Slide 4 Possible Justifications for Your Research New question, new theory New question, existing theory Old question, new theory Old question, conflicting theories Old question, conflicting findings Old question, new methods Old question, new data Slide 5 Literature Review Donts Dont (just) provide a chronological listing of article summaries Dont provide every detail of every study Slide 6 Literature Review Dos Organize your discussion of the literature in a way that reflects and supports the justification for your research Provide more detail for seminal studies, less detail (or simply a citation) for less cited studies For questions that have been studied extensively, it is not necessary to cite every study End your literature review with a summary and critique that justifies your research Slide 7 Literature Review Assignment Approx. 7-10 pages, double-spaced Due November 2nd Slide 8 How to Identify the Relevant Literature Use electronic databases and keyword searches (Google Scholar, JSTOR) Prioritize: Articles Articles published in highly-ranked journals Recently published articles Seminal articles Number of articles: 10-20? Slide 9 Quasi-experimental and Cross- Sectional Designs What are they? Quasi-experimental study of more than one sample (often over period of time) Cross-sectional Analysis of a single sample (lacks random assignment, temporal variation, and manipulation); but includes comparison groups Pre-experimental Cross-sectional, with no comparison group; causal inference impossible Slide 10 Quasi-experimental and Cross- Sectional Designs Why? Property-disposition relationship vs. Stimulus-response relationship Slide 11 Pre-Experimental Designs One-Shot Case Study One group No variation in independent variable X O Slide 12 Pre-Experimental Designs Example: Dependent variable: Americans support for campaign finance reform Independent variable: Watergate scandal Data: 1976 survey of American adults; examine mean level of support Slide 13 Cross-Sectional Designs Static-Group Comparison Design Two groups observed at one time Allows variation in the independent variable X O 1 O 2 Slide 14 Elaboration of Static Group Comparison Design Correlational / Cross-Sectional Designs X1 O 1 X2 O 2 X3 O 3 X4 O 4 Xi O i Problems with correlational/cross-sectional designs? Slide 15 Example: Wine and Health Hypothesis: Drinking wine causes individuals to be healthier (esp. heart) Existing studies: compared the health of wine drinkers to the health of those who do not drink wine: Research design X 1 (Wine drinkers) O 1 (Health) X 2 (Non-drinkers) O 2 (Health) Slide 16 Spurious Results? Slide 17 Controlling for Affluence Research design: X 1 (Affluent Wine drinkers) O 1 (Health) X 2 (Affluent Non-drinkers) O 2 (Health) X 3 (Poor Wine drinkers) O 3 (Health) X 4 (Poor Non-drinkers) O 4 (Health) Slide 18 Another Example (?) http://www.washingtonpost.com/wp- dyn/content/article/2006/05/25/AR200605250 1729.html http://www.washingtonpost.com/wp- dyn/content/article/2006/05/25/AR200605250 1729.html http://www.sciencedaily.com/releases/2007/0 4/070417193338.htm http://www.sciencedaily.com/releases/2007/0 4/070417193338.htm Slide 19 Quasi-Experimental Designs Contrasted Groups Design Multiple groups, based on some categorical variable Observed at one point in time (similar to cross-sectional) O1 O2 O3 O4 Oi Problems with contrasted groups designs? Slide 20 Quasi-Experimental Designs One-Group Pretest-Posttest Design One group Allows variation in the independent variable O1 X O2 Slide 21 Elaborations of the One-Group Pretest- Posttest Design Time Series Designs simple vs. extended O 1 O k X O k+1 O m k = # of pretest observations m = total # observations Or (equivalent time samples design) X 1 O 1 X 2 O 2 X 3 O 3 X m O m Slide 22 Slide 23 Example: Murrays Poverty-Spending Paradox (Schram 1991) Slide 24 Slide 25 Nonequivalent Control Group Design No random assignment O 1 X O 2 O 3 O 4 or O 1 X 1 O 2 O 3 X 2 O 4 Slide 26 Control Series Designs Addition of second (control) group to time series design (CS: multiple time series design) O1 O2 O3 X1 O4 O5 O6 O7 O8 O9 X2 O10 O11 O12 Slide 27 Panel Designs Repeated observations of the same units over time Also goes by: Pooled time-series design Pooled cross-sectional time-series design Slide 28 The Effect of Sanctioning in the TANF Program Slide 29 Slide 30 Slide 31 Slide 32 Donahue and Levitt Hypothesis: The (legal) availability of abortion in a state is negatively related to the crime rate (many years later). O1 O2 O3 X1(early legalization) O4 O5 O6 O7 O8 O9 X2 (legalize 1973) O10 O11 O12 Slide 33 Slide 34 Gerrings Criteria for Research Design Plentitude (N) Boundedness (relevant cases) Comparability (descriptive/causal) Independence Representativeness Variation (X, Y, X&Y) Analytic Utility (of the sample) Replicability Mechanism Causal Comparison Slide 35 KKV Overcoming Common Problems Omitted Variable Bias Inclusion of Irrelevant Variables Endogeneity Assigning Values of the IVs Controlling the Research Situation Slide 36 Does Country Music Cause Suicide? Slide 37 Stack & Gundlach Hypothesis: There is a positive relationship between exposure to country music and suicide rates Research design: X 1 (no country music) O 1 (suicide rate) X 2 (1 station) O 2 (suicide rate) X 3 (2 stations) O 3 (suicide rate) X 4 (3 stations) O 4 (suicide rate) X i ( etc.) O i (suicide rate) Slide 38 Stack & Gundlach Findings: 51% of the variation in urban white suicide rates can be explained by variation in airtime devoted to country music Internal Validity? Slide 39 Slide 40 Slide 41 Slide 42 Slide 43 Slide 44 Slide 45 O 1A Retired A O 4A O 2B SW-Office B O 5B O 3C Returned C O 6C Slide 46 Slide 47 Regression Discontinuity Designs Example: What is the effect of an award on later achievement? (from CS) Slide 48 Regression Discontinuity Designs Example: What is the effect of an award on later achievement? (from CS) Inferential challenge: Award recipients are likely to do well anyway, even without the award, because criteria for receiving award also predict future success Slide 49 A regression discontinuity design is appropriate for any research design in which the assignment of the treatment is determined by a continuous variable that is also related to the outcome of interest. Slide 50 Ludwig and Miller, 2007 Slide 51 Head Start Head Start is a federal program that provides comprehensive education, health, nutrition, and parent involvement services to low- income children and their families. Head Start began in 1965 Local nonprofits/education agencies must apply to federal government to be a Head Start provider Slide 52 Head Start Research Question: What is the effect of Head Start on health/well-being of former participants? What problems might we encounter if we were to conduct the following study: X O (where X = Head Start participation) O Slide 53 Ludwig and Miller, 2007 OEO and application assistance to poor counties in 1965 Treatment group 300 poorest counties that received head start application assistance Control group 228 poor counties that did not Slide 54 Slide 55 Slide 56