Methods of Methods of ObservationObservation

PS 204A, Week 2PS 204A, Week 2

What is Science?What is Science?

Science is: Science is: Based on natural laws/empirical Based on natural laws/empirical

regularities.regularities. Makes predictions.Makes predictions.

Collections of laws that generate predictions Collections of laws that generate predictions that are empirically confirmed constitute that are empirically confirmed constitute “explanations.” “explanations.”

Must be falsifiable.Must be falsifiable. Is always tentative (move from grossly Is always tentative (move from grossly

wrong to more subtly wrong theories).wrong to more subtly wrong theories).

The Scientific EnterpriseThe Scientific Enterprise

Generalization Theory

Observation Hypothesis

TheoryTheory Theories always possess a Theories always possess a

“theoretical notion” or analogy “theoretical notion” or analogy that simplifies reality.that simplifies reality.

This analogy is embodied in This analogy is embodied in the assumptions or premises the assumptions or premises of the theory. Assumptions are of the theory. Assumptions are themselves unobservable – themselves unobservable – and known to be and known to be simplifications (e.g., simplifications (e.g., individuals are rational, states individuals are rational, states are unitary actors). Prefer are unitary actors). Prefer plausible over less plausible plausible over less plausible premises.premises.

Since premises are never Since premises are never “true” or, at least, are “true” or, at least, are unobservable, theories are unobservable, theories are never true, only more or less never true, only more or less useful.useful.

Utility is defined by the Utility is defined by the number of empirically number of empirically supported propositions the supported propositions the theory generates.theory generates.

Generalization Theory

Observation Hypothesis

Analogy

Plausibility of PremisesPlausibility of Premises

If in theories, premises are things we If in theories, premises are things we do not agree on and are unobservable, do not agree on and are unobservable, how do we assess their plausibility?how do we assess their plausibility?

Utility of their predictions (Friedman).Utility of their predictions (Friedman). Accordance with natural laws.Accordance with natural laws. Transform premises into objects of Transform premises into objects of

investigation that are themselves the investigation that are themselves the subjects of theories. subjects of theories. Theory of rationality.Theory of rationality. Theory of unitary states. Theory of unitary states.

Science is a series of “boxes Science is a series of “boxes within boxes”within boxes”

Balance of power theory: Balance of power theory: international system is anarchic and international system is anarchic and composed of unitary states wishing composed of unitary states wishing only to survive. only to survive.

Anarchy: no common authority

Act to check the power of other states.

Unitary states wishing to survive

Internal hierarchy: state “speaks” with a single voice

Within any given “box,” we take the premises as “given.” But any premise may itself become an object of investigation in another “box.”

If testing leads to a revision of a premise, then still need to plug back into original theory and retest.

HypothesesHypotheses Propositions are general Propositions are general

statements that follow statements that follow logically from the logically from the premises.premises.

Hypotheses are Hypotheses are propositions that contain propositions that contain only observable variables only observable variables (i.e., if X, then Y, when (i.e., if X, then Y, when both X and Y can be both X and Y can be observed).observed).

Central issue is Central issue is deductive validity: Does deductive validity: Does the hypothesis follow the hypothesis follow logically and logically and axiomatically from the axiomatically from the premises?premises?

Generalization Theory

Observation Hypothesis

Deductive Validity

TestsTests We test theories by We test theories by

examining whether the examining whether the hypotheses they hypotheses they generate are supported generate are supported by the evidence. We by the evidence. We make the observations make the observations the theories imply.the theories imply. Conclusion validity: is Conclusion validity: is

there a relationship there a relationship between X and Y? between X and Y?

Internal validity: is the Internal validity: is the relationship causal?relationship causal?

Construct validity: do the Construct validity: do the observable measures observable measures capture concepts in the capture concepts in the theory appropriately?theory appropriately?

Generalization Theory

Observation Hypothesis

Test

1. Conclusion Validity

2. Internal Validity

3. Construct Validity

Explanation v. PredictionExplanation v. Prediction Theory offers an explanation for observed Theory offers an explanation for observed

facts and predicts new facts that, once facts and predicts new facts that, once confirmed, are also explained.confirmed, are also explained.

Theories must be potentially falsifiable. Theories must be potentially falsifiable. Popper/Hempel insist that known facts Popper/Hempel insist that known facts cannot falsify a theory. Therefore, cannot falsify a theory. Therefore, prediction is the goal of all science. prediction is the goal of all science.

Alternatively, Snyder argues that if Alternatively, Snyder argues that if scientific evidence is objective, evidence scientific evidence is objective, evidence is evidence independent of the timing of is evidence independent of the timing of it’s discovery relative to the theory.it’s discovery relative to the theory.

Who’s Right?Who’s Right?

All evidence helps corroborate a All evidence helps corroborate a theory, even known facts.theory, even known facts.

Predictions are more “valuable” than Predictions are more “valuable” than explanations in providing evidence explanations in providing evidence for a theory.for a theory.

GeneralizationGeneralization Can we generalize our Can we generalize our

observations to larger observations to larger populations? populations?

Key issue here is Key issue here is external validity (i.e., external validity (i.e., will conclusions hold for will conclusions hold for other people at other other people at other times).times).

Testing may lead us to Testing may lead us to refine our theories refine our theories further, propelling the further, propelling the cycle another round.cycle another round.

Science is interactive. Science is interactive. Tests suggest Tests suggest refinements to theories, refinements to theories, which then generate which then generate new predictions and new predictions and tests. Conversation tests. Conversation between theory and between theory and evidence.evidence.

Generalization Theory

Observation Hypothesis

4. External Validity

Possible Refinements

Deductive v. Inductive Deductive v. Inductive ReasoningReasoning

Hypothetico-Deductive method begins with Hypothetico-Deductive method begins with theory, then generates tests. From theory, then generates tests. From observations, draw inferences about theory, observations, draw inferences about theory, which in turn lead to generalizations about which in turn lead to generalizations about unobserved populations.unobserved populations.

Inductive approach begins with Inductive approach begins with observations and draws draw inferences observations and draws draw inferences about unobserved populations. May or may about unobserved populations. May or may not lead to theory.not lead to theory.

Induction can be science: body of Induction can be science: body of replicated and confirmed laws that are replicated and confirmed laws that are predictive. But, falsifiablility is always an predictive. But, falsifiablility is always an issue.issue.

What can we What can we learn by learn by

“observing”?“observing”?

Observation Observation InferenceInference

Observation central to both deduction Observation central to both deduction and induction.and induction.

How do we draw inferences about How do we draw inferences about unobservable phenomena, premises, or unobservable phenomena, premises, or populations from observable populations from observable phenomena? phenomena?

How do we learn about what we can’t How do we learn about what we can’t see, from what we can?see, from what we can?

Applies equally to inherently Applies equally to inherently unobservable traits (what goes on in unobservable traits (what goes on in people’s heads), future events people’s heads), future events (predictions), and true populations (predictions), and true populations (alternative worlds). (alternative worlds).

Descriptive Inference: Descriptive Inference: Or how do we know what we Or how do we know what we

saw?saw? What is this a case of? What is the What is this a case of? What is the

class of which you observe one or class of which you observe one or more members? more members? Many categories are question or theory Many categories are question or theory

dependent.dependent. If observation is unique, no If observation is unique, no

generalizations are possible.generalizations are possible. If observation is exhaustive (all members If observation is exhaustive (all members

of class), what can you generalize to? of class), what can you generalize to?

Descriptive Inference IIDescriptive Inference II Probabilistic v. Deterministic Events: all Probabilistic v. Deterministic Events: all

events have systematic and non-events have systematic and non-systematic components.systematic components.

Probabilistic events occur with some Probabilistic events occur with some probability < 1.0; if replayed under probability < 1.0; if replayed under identical conditions, the observed result identical conditions, the observed result would differ (more or less). On average, would differ (more or less). On average, would get same result (non-systematic would get same result (non-systematic component is random).component is random).

Deterministic events occur with Deterministic events occur with certainty (p = 1.0). Special case in certainty (p = 1.0). Special case in which non-systematic component is which non-systematic component is zero.zero.

Descriptive Inference IIIDescriptive Inference III With a probabilistic event, how are we to With a probabilistic event, how are we to

classify a particular instance? classify a particular instance? What should we infer from a war in which What should we infer from a war in which

country A loses? We observe the loss, but can country A loses? We observe the loss, but can we generalize to other similar cases? If the we generalize to other similar cases? If the probability of victory was .25, what can we probability of victory was .25, what can we infer from the actual loss? infer from the actual loss?

Inference is harder the smaller the number of Inference is harder the smaller the number of cases.cases.

Problem arises when we mistake a Problem arises when we mistake a probabilistic event for a deterministic probabilistic event for a deterministic event. event. Incorrect to infer any “pathology” or Incorrect to infer any “pathology” or

“mistake” in such instances. “mistake” in such instances.

Induction I: Empirical Induction I: Empirical LawsLaws

Analysis limited to observable phenomena Analysis limited to observable phenomena only.only.

An empirical law is a robust “regularity” An empirical law is a robust “regularity” (e.g., the democratic peace) (e.g., the democratic peace)

By extending empirical laws, we can make By extending empirical laws, we can make predictions (inferences) about future predictions (inferences) about future events.events.

But, But, Concepts do not exist independent of theory.Concepts do not exist independent of theory. Correlations may be spurious.Correlations may be spurious. Correlation does not equal causation. We may Correlation does not equal causation. We may

“explain” events by empirical laws but such laws “explain” events by empirical laws but such laws do not imply cause or constitute a causal test. do not imply cause or constitute a causal test.

Induction II: Thick Induction II: Thick DescriptionDescription

Inferring unobservable from observable Inferring unobservable from observable phenomena.phenomena.

““Thick description” uses observables to Thick description” uses observables to learn about unobservables.learn about unobservables. Geertz in Geertz in NegaraNegara studies puppet theatre in studies puppet theatre in

Bali to gain insight into political culture in Bali to gain insight into political culture in that society.that society.

““Justifications” as means of learning about Justifications” as means of learning about social norms.social norms.

At this level, does not aim to explain At this level, does not aim to explain outcomes, only to describe outcomes, only to describe unobservable traits. unobservable traits.

Induction III: Induction III: InterpretationInterpretation As in thick description, use observables to infer As in thick description, use observables to infer

unobservable phenomena, typically motives, unobservable phenomena, typically motives, intent, purpose. Then explain observed intent, purpose. Then explain observed outcomes in terms of the actor’s self-outcomes in terms of the actor’s self-understanding. understanding. Geertz: “not an experimental science in search of law Geertz: “not an experimental science in search of law

but an interpretive one in search of meaning.”but an interpretive one in search of meaning.” When the choice by the actor is rendered When the choice by the actor is rendered

appropriate or required by the observable traits appropriate or required by the observable traits of the actor and its environment, claim to have of the actor and its environment, claim to have explained its behavior.explained its behavior. Risk of circularity: observables used to infer Risk of circularity: observables used to infer

unobservables, which are then used to explain unobservables, which are then used to explain observables. observables.

Not falsifiable.Not falsifiable. External validity? Geertz: object is “not to generalize External validity? Geertz: object is “not to generalize

across cases but to generalize within them.”across cases but to generalize within them.” Interpretation is not science. Interpretation is not science.

ConclusionConclusion

Observation is central to the Observation is central to the scientific enterprise. There can be scientific enterprise. There can be no science without observation.no science without observation.

Observation by itself can never Observation by itself can never demonstrate cause. To explain a demonstrate cause. To explain a phenomenon requires an empirically phenomenon requires an empirically supported theory.supported theory.

Nonetheless, much of what we do in Nonetheless, much of what we do in political science is observe.political science is observe.