difference-in-differences designsternality on other spanish regions, and foreign investment might...

Difference-in-Differences Designs

Kosuke Imai

Harvard University

STAT 186/GOV 2002 CAUSAL INFERENCE

Fall 2019

Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 1 / 22

Motivation

How should we conduct causal inference when repeatedmeasurements are available?Two types of variations:

1 cross-sectional variation within each time period2 temporal variation within each unit

Before-and-after and cross-sectional designs

0.0

0.2

0.4

0.6

0.8

1.0

Ave

rage

Out

com

e

●

●

●

●

treatment group

control group

time t time t+1

counterfactual

0.0

0.2

0.4

0.6

0.8

1.0

Ave

rage

Out

com

e

●

●

●

●

treatment group

control group

time t time t+1

counterfactual

Can we exploit both variations?Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 2 / 22

Minimum Wage and Unemployment(Card and Krueger. 1994. Am. Econ. Rev)

How does the increase in minimum wage affect employment?Many economists believe the effect is negative

especially for the pooralso for the whole economy

Hard to randomize the minimum wage increase

In 1992, NJ minimum wage increased from $4.25 to $5.05Neighboring PA stays at $4.25Observe employment in both states before and after increase

NJ and (eastern) PA are similarFast food chains in NJ and PA are similar: price, wages, products,etc.They are most likely to be affected by this increase


Difference-in-Differences Design

Parallel trend assumptionVisualizing Difference-in-Differences

0.24

0.26

0.28

0.30

0.32

0.34

0.36

Aver

age

Prop

ortio

n of

Ful

l−tim

e Em

ploy

ees

●

●

●

●

treatment group (New Jersey)

control group (Pennsylvania)

before after

counterfactual (New Jersey)

average causal effect

estimate

POL345/SOC305 (Princeton) Observational Studies Fall 2016 18 / 20


Setup:Two time periods: time 0 (pre-treatment), time 1 (post-treatment)Gi : treatment (Gi = 1) or control (Gi = 0) groupZit = tGi : treatment assignment indicator for t = 0,1Potential outcomes: Yi0(0), Yi0(1), Yi1(0), Yi1(1)Observed outcomes: Yit = Yit (Zit )

Average treatment effect for the treated:

τ = E{Yi1(1)− Yi1(0) | Gi = 1}

Parallel trend assumption:

E{Yi1(0)− Yi0(0) | Gi = 1} = E{Yi1(0)− Yi0(0) | Gi = 0}

DiD estimator:

τDiD = {E(Yi1 | Gi = 1)− E(Yi0 | Gi = 1)}−{E(Yi1 | Gi = 0)− E(Yi0 | Gi = 0)}

Applicable to repeated cross-section data as wellKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 5 / 22

Linear Model for the Difference-in-Differences

Two-way fixed effects model:

Yit (z) = αi + βt + τz + εitE{Yi0(0)} = αiE{Yi1(0)} = αi + βE{Yi1(1)} = αi + β + τE{Yi1(1)− Yi1(0)} = τ

Parallel trend assumption:E{Yi1(0)− Yi0(0) | Gi = g} = βOr equivalently E(εi1 − εi0 | Gi = g) = 0Both Zit and εit can depend on αi or unobserved confounders

Least squares estimator equals the nonparametric DiD estimator,i.e., τFE = τDiD

This equivalence does not hold in general beyond the 2× 2 case


Comparison with the Lagged Outcome Model

Lagged outcome model:

Yi1(z) = α + ρYi0 + τz + εi(z)

Nonparametric identification assumption:

{Yi1(1),Yi1(0)} ⊥⊥ Zit | Yi0

can be made conditional on Xi as well as Yi0neither stronger nor weaker than the parallel trend assumptionsame as parallel trend if E(Yi0 | Gi = 1) = E(Yi0 | Gi = 0)

Where does the imbalance in lagged outcome come from?Difference-in-Differences unobserved time-invariant confounderLagged outcome directly affects treatment assignment


Relationship between the DiD and Lagged OutcomeEstimators

Least squares estimator:

τLD = E(Yi1 | Gi = 1)− E(Yi1 | Gi = 0)

− ρ{( E(Yi0 | Gi = 1)− E(Yi0 | Gi = 0))}If ρ = 1, then τLD = τDiDAssume 0 ≤ ρ < 1 (stationarity)Without loss of generality, assume E(Yi0 | Gi = 1) ≥ E(Yi0 | Gi = 0)(monotonicity)

1 If parallel trend holds, E(τLD) ≥ E(τDiD) = τ2 If ignorability holds, τ = E(τLD) ≥ E(τDiD)

Nonparametric estimator (Ding and Li. 2019. Political Anal.):

µ0 = E{Yi1(0) | Gi = 1} = E{E(Yi1 | Gi = 0,Yi0) | Gi = 1}(stationarity) ∂E(Yi1 | Gi = 0,Yi0 = y)/∂y < 1 for all y(stochastic monotonicity) FY0 (y | Gi = 1) ≤ FY0 (y | Gi = 0) for all y

Then, the bracketing relationship holdsKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 8 / 22

Adjusting for Baseline Covariates

Parallel trend assumption conditional on the baseline covaraites:

E{Yi1(0)− Yi0(0) | Xi = x,Gi = 1}= E{Yi1(0)− Yi0(0) | Xi = x,Gi = 0} for all x

Matching: parallel trend within a pair or a strata

Weighting (Abadie. 2005. Rev. Econ. Stud ):

E{Yi1(1)− Yi1(0) | Gi = 1}

= E[

Yi1 − Yi0

Pr(Gi = 1)· Gi − Pr(Gi = 1 | Xi)

1− Pr(Gi = 1 | Xi)

]

Unconditional parallel trend assumption neither implies nor isimplied by conditional parallel trend assumption


Nonlinear Difference-in-Differences(Athey and Imbens. 2006. Econometrica )

Standard DiD relies upon the linearity assumptionNot invariant to a nonlinear transformation of outcome (e.g., log)

−4 −2 0 2 4

Control

0

r0 (q)

q

1

F00

F01

−4 −2 0 2 4

Treated

0

r1 (q)

q

1

F10 F11

Temporal change in quantile is identical between the two groupsKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 10 / 22

Formalization of Nonlinear DiD

Distribution functions: Fgt (y) = Pr(Yit (0) ≤ y | Gi = g)

Quantile treatment effect for a given q:

τ(q) = F−111 (q)− F−1

11 (q)

where F11(y) = Pr(Yi1(1) ≤ y | Gi = 1)

We wish to identify F11(y)

Identification assumption:

F01(F−100 (q))︸︷︷︸

r0(q)

= F11(F−110 (q))︸︷︷︸

r1(q)

for all q ∈ [0,1]

Under this assumption,

F11(y) = F01[F−100 {F10(y)}]


Synthetic Control Method (Abadie et al. 2010. J. Am. Stat. Assoc.)

One treated unit i = 1 receiving the treatment at time TQuantity of interest: Y1T − Y1T (0)

Create a synthetic control using past outcomesWeighted average:

Y1T (0) =N∑

i=2

wiYiT

where the weights balance past outcomes

w = argminw

T−1∑t=1

(Y1t −

N∑i=2

wiYit

)2

with∑N

i=2 wi = 1 and wi ≥ 0One could include time-invariant covariates Xi


Causal Effect of ETA’s TerrorismTHE AMERICAN ECONOMIC REVIEW MARCH 2003

11--5 / s /a /

210-5

0 9 -

, -Actual with terrorism - -Synthetic without terrorism

1

1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 year

FIGURE1. PERCAPITA GDP FOR THE BASQUECOUNTRI

4 -\

-GDt' gap I -Terrorist activity

year

FIGIJRE2. TERRORIST AND ESTIMATEDACTIVITY GAP

expect terrorism to have a lagged negative ef- of deaths caused by terrorist actions (used as a fect on per capita GDP. In Figure 2, we plotted proxy for overall terrorist activity). As ex-the per capita GDP gap, Y, - YT, as a percent- pected, spikes in terrorist activity seem to be age of Basque per capita GDP, and the number followed by increases in the amplitude of the

(Abadie and Gardeazabal. 2003. Am. Econ. Rev.)


Placebo TestTHE AMERICAN ECONOMIC REVIEW MARCH 2003

Actual without terrorism Synthetic without terrorism

3

)75 1980 1985 1990 1995 2000 year

FIGURE4 A "PLACEBOSTUDY."PER CAPITA GDP FOR CATALONIA

Catalonia is the main contributor to the syn- economic effect of terrorism on the Basque thetic control for the Basque Country, an ab- Country. To the extent that the regions which normally high level of per capita GDP for form the synthetic control might have been eco- Catalonia during the 1990's may artificially nomically hurt by the conflict, our estimated widen the GDP gap for the Basque Country in GDP gap would provide a lower bound on the Figure 1. Therefore, our placebo study suggests economic effect of terrorism on the Basque that, while per capita GDP for Catalonia can be Country economy. On the other hand, the con- reasonably well reproduced by our techniques, flict may have diverted investment from the the catch-up in per capita GDP for the Basque Basque Country to other Spanish regions, arti- Country during the 1990's (relative to the syn- ficially increasing the magnitude of the gap. thetic control region) may have been more pro- However, since the size of the synthetic Basque nounced than what Figure 1 indicates. Country is much larger than the actual Basque

Country, this type of bias is arguably small.12 In C. Discussion the next section we show evidence that support

the view that the effect of the conflict was small As noted earlier, the Basque Country has outside the Basque Country.

been the main scenario of the terrorist conflict. A more important criticism of the analysis in However, ETA has also operated in other Span- this section is that, as long as the synthetic ish regions. Even though there is no indication control cannot reproduce exactly the character- that entrepreneurs have abandoned Spain as a istics of the Basque Country before terrorism, result of the terrorist threat, Basque terrorism the GDP gap may have been created by differ- might have imposed a negative reputational ex- ternality on other Spanish regions, and foreign investment might have chosen alternative des- "For the 1964-1975 period, GDP for the synthetic

tinations with no terrorist conflicts. If it is in fact region was 2.5 times larger than GDP for the Basque Coun- try: this figure increased to more than 3 during the terrorism

the case that the Basque terrorist conflict has era. Furthermore, investment diverted to regions other than had a negative economic effect on other Spanish those in the synthetic Basque Country does not affect the regions, this effect is arguably weaker than the validity of our analysis.

can do this for all control units and compare them with the treated unitKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 14 / 22

502 Journal of the American Statistical Association, June 2010

Figure 4. Per-capita cigarette sales gaps in California and placebogaps in all 38 control states.

provide a good fit for per capita cigarette consumption priorto Proposition 99 for the majority of the states in the donorpool. However, Figure 4 indicates also that per capita cigarettesales during the 1970–1988 period cannot be well reproducedfor some states by a convex combination of per capita ciga-rette sales in other states. The state with worst fit in the pre-Proposition 99 period is New Hampshire, with a MSPE of 3437.The large MSPE for New Hampshire does not come as a sur-prise. Among all the states in the donor pool, New Hampshireis the state with the highest per capita cigarette sales for everyyear prior to the passage of Proposition 99. Therefore, there isno combination of states in our sample that can reproduce thetime series of per capita cigarette sales in New Hampshire priorto 1988. Similar problems arise for other states with extremevalues of per capita cigarette sales during the pre-Proposition 99period.

If the synthetic California had failed to fit per capita ciga-rette sales for the real California in the years before the pas-sage of Proposition 99, we would have interpreted that muchof the post-1988 gap between the real and the synthetic Cal-ifornia was also artificially created by lack of fit, rather thanby the effect of Proposition 99. Similarly, placebo runs withpoor fit prior to the passage of Proposition 99 do not provideinformation to measure the relative rarity of estimating a largepost-Proposition 99 gap for a state that was well fitted priorto Proposition 99. For this reason, we provide several differentversions of Figure 4, each version excluding states beyond acertain level of pre-Proposition 99 MSPE.

Figure 5 excludes states that had a pre-Proposition 99 MSPEof more than 20 times the MSPE of California. This is a verylenient cutoff, discarding only four states with extreme valuesof pre-Proposition 99 MSPE for which the synthetic methodwould be clearly ill-advised. In this figure there remain a fewlines that still deviate substantially from the zero gap line in thepre-Proposition 99 period. Among the 35 states remaining inthe figure, the California gap line is now about the most unusualline, especially from the mid-1990s onward.

Figure 5. Per-capita cigarette sales gaps in California and placebogaps in 34 control states (discards states with pre-Proposition 99MSPE twenty times higher than California’s).

Figure 6 is based on a lower cutoff, excluding all states thathad a pre-Proposition 99 MSPE of more than five times theMSPE of California. Twenty-nine control states plus Californiaremain in the figure. The California gap line is now clearly themost unusual line for almost the entire post-treatment period.

In Figure 7 we lower the cutoff even further and focusexclusively on those states that we can fit almost as wellas California in the period 1970–1988, that is, those stateswith pre-Proposition 99 MSPE not higher than twice the pre-Proposition 99 MSPE for California. Evaluated against the dis-tribution of the gaps for the 19 remaining control states in Fig-ure 7, the gap for California appears highly unusual. The nega-tive effect in California is now by far the lowest of all. Becausethis figure includes 19 control states, the probability of estimat-

Figure 6. Per-capita cigarette sales gaps in California and placebogaps in 29 control states (discards states with pre-Proposition 99MSPE five times higher than California’s).


Model-based Justification

The main motivating factor analytic model:

Yit (0) = γt + δ>t Xi + ξ>t Ui + εit

Generalization of the linear two-way fixed effects modelKey assumption: there exist weights such that

N∑i=2

wiXi = X1 andN∑

i=2

wiUi = U1

Another motivating autoregressive model with time-varyingcovariates:

Yit (0) = ρtYi,t−1(0) + δ>t Xit + εit

Xit = λt−1Yi,t−1(0) + ∆t−1Xi,t−1 + νit

Past outcomes can affect current treatmentNo unobserved time-invariant confounders


Generalizing the Difference-in-Differencesstaggered treatment What if units go in and out of treatment?

1960 1970 1980 1990 2000 2010

Year

Cou

ntrie

s

treatment Autocracy (Control) Democracy (Treatment)

Democracy as the Treatment

ireland

netherlands

switzerland

sweden

norway

denmark

belgium

japan

usa

uk

france

austria

finland

new zealand

italy

canada

korea

australia

germany

1850 1900 1950 2000

Year

Cou

ntrie

s

treatment Peace (Control) War (Treatment)

War as the Treatment

Acemoglu et al. 2019. J. Political Econ. Scheve and Stasavage. 2012.

Am. Political Sci. Rev.Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 17 / 22

Matching Methods for Panel Data (Imai et al. 2019. Working Paper)

Choose the number of lags L and leads FATE of Policy Change for the Treated:

E{

Yi,t+F

(Zit = 1,Zi,t−1 = 0, {Zi,t−`}L`=2

)−

Yi,t+F

(Zit = 0,Zi,t−1 = 0, {Zi,t−`}L`=2

)| Zit = 1,Zi,t−1 = 0

}Estimation procedure:

1 Construct a matched set for each treated unit that consists ofcontrol units with the identical treatment history up to L time periods

2 Refine covariate balance with a matching/weighting method withina matched set

3 Use the multi-period difference-in-differences estimator:

1∑Ni=1∑T−F

t=L+1 Zit

N∑i=1

T−F∑t=L+1

Zit

(Yi,t+F − Yi,t−1)−∑

i′∈Mit

w i′it (Yi′,t+F − Yi′,t−1)


Empirical Application (1)

ATT with L = 4 and F = 1,2,3,4We consider democratization and authoritarian reversalExamine the number of matched control units18 (13) treated observations have no matched control

Democratization

0 20 40 60 80 100 120

05

1015

2025

30

Four Year Lags 9 Emtpy SetsOne Year Lag 0 Emtpy Sets

Fre

quen

cy

Authoritarian Reversal

0 20 40 60 80 100 120

05

1015

2025

30


Fre

quen

cy

Starting War

0 5 10 15 20

05

1015

2025

30


Fre

quen

cy

Ending War

0 5 10 15 20

05

1015

2025

30


Fre

quen

cy

Ace

mog

lu e

t al.

(201

8)S

chev

e &

Sta

sava

ge (

2012

)

Number of matched control units


Improved Covariate Balance−

2−

10

12

−4 −3 −2 −1

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Dem

ocra

tizat

ion

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Aut

horit

aria

n R

ever

sal

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Sta

rtin

g W

ar

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Sta

rtin

g W

ar

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Sta

rtin

g W

ar

Sta

ndar

dize

d M

ean

Diff

eren

ces

for

Sta

rtin

g W

ar

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

−2

−1

01

2

−4 −3 −2 −1

Mahalanobis Distance Matching

Propensity Score Matching

Propensity Score Weighting

Before Matching

Before Refinement

Ace

mog

lu e

t al.

(201

8)S

chev

e &

Sta

sava

ge (

2012

)

Years relative to the administration of treatmentKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 20 / 22

Estimated Causal Effects

● ●●

● ●

Up to 5 matches

−0.

2−

0.1

0.05

0.15

0 1 2 3 4

● ● ● ● ●

Up to 10 matches

0 1 2 3 4

● ● ● ●●

Up to 5 matches

0 1 2 3 4

● ● ● ● ●

Up to 10 matches

0 1 2 3 4

● ● ● ● ●

0 1 2 3 4

●

● ● ●●

−0.

2−

0.1

0.05

0.15

0 1 2 3 4

●

● ● ●●

0 1 2 3 4

●

● ● ●●

0 1 2 3 4

●

● ● ●●

0 1 2 3 4

●

● ● ● ●

0 1 2 3 4

Mahalanobis Matching Propensity Score Matching Propensity Score Weighting

Est

imat

ed E

ffect

of

Dem

ocra

tizat

ion

Est

imat

ed E

ffect

of

Aut

horit

aria

n R

ever

sal

Years relative to the administration of treatment


Concluding Remarks

Difference-in-differences design:fully exploit the panel data structurecross sectional and before-and-after designs do notparallel trend assumptionadjusts for time-invariant unobserved confounderstradeoff between dynamics and unobservables

Extensions:adjusting for baseline covariatesnonlinear difference-in-differencessysthetic controlsmultiperiod difference-in-differences estimator

Readings: ANGRIST AND PISKE. CHAPTER 5


difference-in-differences designsternality on other spanish regions, and foreign investment might...

Documents