Designing Social Inquiry

Determining What to Observe: Things That We Should Be

Careful about When We Design Researches

Jaechun Kim

PROBLEM OF INDETERMINATE RESEARCH DESIGN

    What is research design?

Design that shows how we are going to use our evidence to evaluate hypotheses…

What is indeterminate research design?

Nothing can be learned about the causal hypotheses…

Two cases of indeterminate research design

More inferences than observations – Problem of too many variables and too few cases! Why does this lead to indeterminate research design?

P. 119 – What explains successful joint collaboration on defense projects?

3 cases 7 IVs … why is it a problem?

Too many variables, but too few cases…!

Mathematical model, see p. 121

Multicollinearity… What is it?? p. 122

Punch Kick

Ability to Fight a Good Fight

What if two IV s are perfectly correlated?

Mathematical model, p. 123

THE LIMITS OF RANDOME SELECITON

Random selection is best, but usually not feasible in social science… Why?

Universe of cases is not known…? P. 125 You may fail to examine important cases…

if you rely on random selection… Problem of “small n” inherent problem of

selection bias… so be careful! Careful non-random selection!! P. 126 good example!

TO AVIOD SELECTION BIAS

Allow variation on your DV of possible! If possible…

The dogs that didn’t bark… !!

EXAMPLES OF SELECTION BIAS  P. 128 American investment Internal

violence

Cases chosen: Countries with major American investments and

good deal of violence Countries with no American investment and no

violence

What’s wrong with this case selection?

What determines presidential involvement in foreign policy decisions in the US? P. 132

Cases chosen?

Policy decisions on which information about the president’s participation in meeting is available

What’s wrong with this?

Selection rule is correlated with low levels of presidential involvement!

Michael Porter’s case: What are the sources of “competitive advantage” for industries and firms?

p. 133-134

Cases selected:

Ten states which compete successfully in a range of such industries

What’s wrong with this?

Selection on DV…

Literature on deterrence theory:

Deterrence: The use of threats to induce the opponents to behave in desirable ways…

Students of deterrence have often examined “acute crises” in which deterrence have failed…

What’s wrong with this research design and case selection?

Dogs that didn’t bark…!

‘The dog that didn’t bark’ is an expression from a Sherlock Holmes mystery. It was an important clue that led to identifying the criminal. It seems that the killer entered and left the estate grounds one night but without the guard dog barking an alarm at the intruder’s presence as expected. From this non-event Holmes reasoned that the dog must have known the killer and that clue led to solving the case.

Arthur Conan Doyle’s story, The Hound of the Baskervilles

P. 140 Snyder’s work… evaluate!! Myth of Empire (1991)

P. 144 Kohli’s work… evaluate!!

Snyder’s puzzle: Why do states tend to over-expand?

Before presenting his own explanation of this puzzle, Snyder summarizes the answers given by other schools of thought (from handout): Snyder's explanation of this puzzle: a different domestic political explanation:

Realist Explanation Cognitive Explanation Domestic Political Explanations

Snyder’s explanation of this puzzle - a different domestic political explanation

Coalition Logrolling and Coalition Ideology:

Snyder claims that the myths of empire arise out of coalitions of interests that logroll (trade) various policy choices and use a nationalist/expansionist ideology to justify expansionist policies.

e.g. authoritarian rules induce high economic growth

Cases selected:

authoritarian rules – high econ growth and non-authoritarian rules – low econ growth

What’s wrong with this?

 

e.g. Inkeles and Rossi p. 146

Industrialization a particular prestige (attached to occupations) hierarchy

Cases: a set of countries that was quite varied except that they all were industrialized!!

What’s wrong with this case selection??

Some research designs are good for generating hypothesis, but not evaluating it…! Admit your limits!!

Sometimes researchers select observations so the DV is constant

e.g. Center for Disease Control (CDC) examining ‘cancer cluster’ p. 148

cf. political cluster

What is use of it?

*Methodological Critiques of Others’ Works…

*Helen Milner’s work…pp. 179~182