difference-in-differences designsternality on other spanish regions, and foreign investment might...
Post on 21-Jan-2021
1 Views
Preview:
TRANSCRIPT
Difference-in-Differences Designs
Kosuke Imai
Harvard University
STAT 186/GOV 2002 CAUSAL INFERENCE
Fall 2019
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 1 / 22
Motivation
How should we conduct causal inference when repeatedmeasurements are available?Two types of variations:
1 cross-sectional variation within each time period2 temporal variation within each unit
Before-and-after and cross-sectional designs
0.0
0.2
0.4
0.6
0.8
1.0
Ave
rage
Out
com
e
●
●
●
●
treatment group
control group
time t time t+1
counterfactual
0.0
0.2
0.4
0.6
0.8
1.0
Ave
rage
Out
com
e
●
●
●
●
treatment group
control group
time t time t+1
counterfactual
Can we exploit both variations?Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 2 / 22
Minimum Wage and Unemployment(Card and Krueger. 1994. Am. Econ. Rev)
How does the increase in minimum wage affect employment?Many economists believe the effect is negative
especially for the pooralso for the whole economy
Hard to randomize the minimum wage increase
In 1992, NJ minimum wage increased from $4.25 to $5.05Neighboring PA stays at $4.25Observe employment in both states before and after increase
NJ and (eastern) PA are similarFast food chains in NJ and PA are similar: price, wages, products,etc.They are most likely to be affected by this increase
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 3 / 22
Difference-in-Differences Design
Parallel trend assumptionVisualizing Difference-in-Differences
0.24
0.26
0.28
0.30
0.32
0.34
0.36
Aver
age
Prop
ortio
n of
Ful
l−tim
e Em
ploy
ees
●
●
●
●
treatment group (New Jersey)
control group (Pennsylvania)
before after
counterfactual (New Jersey)
average causal effect
estimate
POL345/SOC305 (Princeton) Observational Studies Fall 2016 18 / 20
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 4 / 22
Setup:Two time periods: time 0 (pre-treatment), time 1 (post-treatment)Gi : treatment (Gi = 1) or control (Gi = 0) groupZit = tGi : treatment assignment indicator for t = 0,1Potential outcomes: Yi0(0), Yi0(1), Yi1(0), Yi1(1)Observed outcomes: Yit = Yit (Zit )
Average treatment effect for the treated:
τ = E{Yi1(1)− Yi1(0) | Gi = 1}
Parallel trend assumption:
E{Yi1(0)− Yi0(0) | Gi = 1} = E{Yi1(0)− Yi0(0) | Gi = 0}
DiD estimator:
τDiD = {E(Yi1 | Gi = 1)− E(Yi0 | Gi = 1)}−{E(Yi1 | Gi = 0)− E(Yi0 | Gi = 0)}
Applicable to repeated cross-section data as wellKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 5 / 22
Linear Model for the Difference-in-Differences
Two-way fixed effects model:
Yit (z) = αi + βt + τz + εitE{Yi0(0)} = αiE{Yi1(0)} = αi + βE{Yi1(1)} = αi + β + τE{Yi1(1)− Yi1(0)} = τ
Parallel trend assumption:E{Yi1(0)− Yi0(0) | Gi = g} = βOr equivalently E(εi1 − εi0 | Gi = g) = 0Both Zit and εit can depend on αi or unobserved confounders
Least squares estimator equals the nonparametric DiD estimator,i.e., τFE = τDiD
This equivalence does not hold in general beyond the 2× 2 case
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 6 / 22
Comparison with the Lagged Outcome Model
Lagged outcome model:
Yi1(z) = α + ρYi0 + τz + εi(z)
Nonparametric identification assumption:
{Yi1(1),Yi1(0)} ⊥⊥ Zit | Yi0
can be made conditional on Xi as well as Yi0neither stronger nor weaker than the parallel trend assumptionsame as parallel trend if E(Yi0 | Gi = 1) = E(Yi0 | Gi = 0)
Where does the imbalance in lagged outcome come from?Difference-in-Differences unobserved time-invariant confounderLagged outcome directly affects treatment assignment
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 7 / 22
Relationship between the DiD and Lagged OutcomeEstimators
Least squares estimator:
τLD = E(Yi1 | Gi = 1)− E(Yi1 | Gi = 0)
− ρ{( E(Yi0 | Gi = 1)− E(Yi0 | Gi = 0))}If ρ = 1, then τLD = τDiDAssume 0 ≤ ρ < 1 (stationarity)Without loss of generality, assume E(Yi0 | Gi = 1) ≥ E(Yi0 | Gi = 0)(monotonicity)
1 If parallel trend holds, E(τLD) ≥ E(τDiD) = τ2 If ignorability holds, τ = E(τLD) ≥ E(τDiD)
Nonparametric estimator (Ding and Li. 2019. Political Anal.):
µ0 = E{Yi1(0) | Gi = 1} = E{E(Yi1 | Gi = 0,Yi0) | Gi = 1}(stationarity) ∂E(Yi1 | Gi = 0,Yi0 = y)/∂y < 1 for all y(stochastic monotonicity) FY0 (y | Gi = 1) ≤ FY0 (y | Gi = 0) for all y
Then, the bracketing relationship holdsKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 8 / 22
Adjusting for Baseline Covariates
Parallel trend assumption conditional on the baseline covaraites:
E{Yi1(0)− Yi0(0) | Xi = x,Gi = 1}= E{Yi1(0)− Yi0(0) | Xi = x,Gi = 0} for all x
Matching: parallel trend within a pair or a strata
Weighting (Abadie. 2005. Rev. Econ. Stud ):
E{Yi1(1)− Yi1(0) | Gi = 1}
= E[
Yi1 − Yi0
Pr(Gi = 1)· Gi − Pr(Gi = 1 | Xi)
1− Pr(Gi = 1 | Xi)
]
Unconditional parallel trend assumption neither implies nor isimplied by conditional parallel trend assumption
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 9 / 22
Nonlinear Difference-in-Differences(Athey and Imbens. 2006. Econometrica )
Standard DiD relies upon the linearity assumptionNot invariant to a nonlinear transformation of outcome (e.g., log)
−4 −2 0 2 4
Control
0
r0 (q)
q
1
F00
F01
−4 −2 0 2 4
Treated
0
r1 (q)
q
1
F10 F11
Temporal change in quantile is identical between the two groupsKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 10 / 22
Formalization of Nonlinear DiD
Distribution functions: Fgt (y) = Pr(Yit (0) ≤ y | Gi = g)
Quantile treatment effect for a given q:
τ(q) = F−111 (q)− F−1
11 (q)
where F11(y) = Pr(Yi1(1) ≤ y | Gi = 1)
We wish to identify F11(y)
Identification assumption:
F01(F−100 (q))︸ ︷︷ ︸
r0(q)
= F11(F−110 (q))︸ ︷︷ ︸
r1(q)
for all q ∈ [0,1]
Under this assumption,
F11(y) = F01[F−100 {F10(y)}]
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 11 / 22
Synthetic Control Method (Abadie et al. 2010. J. Am. Stat. Assoc.)
One treated unit i = 1 receiving the treatment at time TQuantity of interest: Y1T − Y1T (0)
Create a synthetic control using past outcomesWeighted average:
Y1T (0) =N∑
i=2
wiYiT
where the weights balance past outcomes
w = argminw
T−1∑t=1
(Y1t −
N∑i=2
wiYit
)2
with∑N
i=2 wi = 1 and wi ≥ 0One could include time-invariant covariates Xi
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 12 / 22
Causal Effect of ETA’s TerrorismTHE AMERICAN ECONOMIC REVIEW MARCH 2003
11--5 / s /a /
210-5
0 9 -
, -Actual with terrorism - -Synthetic without terrorism
1
1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 year
FIGURE1. PERCAPITA GDP FOR THE BASQUECOUNTRI
4 -\
-GDt' gap I -Terrorist activity
year
FIGIJRE2. TERRORIST AND ESTIMATEDACTIVITY GAP
expect terrorism to have a lagged negative ef- of deaths caused by terrorist actions (used as a fect on per capita GDP. In Figure 2, we plotted proxy for overall terrorist activity). As ex-the per capita GDP gap, Y, - YT, as a percent- pected, spikes in terrorist activity seem to be age of Basque per capita GDP, and the number followed by increases in the amplitude of the
(Abadie and Gardeazabal. 2003. Am. Econ. Rev.)
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 13 / 22
Placebo TestTHE AMERICAN ECONOMIC REVIEW MARCH 2003
Actual without terrorism Synthetic without terrorism
3
)75 1980 1985 1990 1995 2000 year
FIGURE4 A "PLACEBOSTUDY."PER CAPITA GDP FOR CATALONIA
Catalonia is the main contributor to the syn- economic effect of terrorism on the Basque thetic control for the Basque Country, an ab- Country. To the extent that the regions which normally high level of per capita GDP for form the synthetic control might have been eco- Catalonia during the 1990's may artificially nomically hurt by the conflict, our estimated widen the GDP gap for the Basque Country in GDP gap would provide a lower bound on the Figure 1. Therefore, our placebo study suggests economic effect of terrorism on the Basque that, while per capita GDP for Catalonia can be Country economy. On the other hand, the con- reasonably well reproduced by our techniques, flict may have diverted investment from the the catch-up in per capita GDP for the Basque Basque Country to other Spanish regions, arti- Country during the 1990's (relative to the syn- ficially increasing the magnitude of the gap. thetic control region) may have been more pro- However, since the size of the synthetic Basque nounced than what Figure 1 indicates. Country is much larger than the actual Basque
Country, this type of bias is arguably small.12 In C. Discussion the next section we show evidence that support
the view that the effect of the conflict was small As noted earlier, the Basque Country has outside the Basque Country.
been the main scenario of the terrorist conflict. A more important criticism of the analysis in However, ETA has also operated in other Span- this section is that, as long as the synthetic ish regions. Even though there is no indication control cannot reproduce exactly the character- that entrepreneurs have abandoned Spain as a istics of the Basque Country before terrorism, result of the terrorist threat, Basque terrorism the GDP gap may have been created by differ- might have imposed a negative reputational ex- ternality on other Spanish regions, and foreign investment might have chosen alternative des- "For the 1964-1975 period, GDP for the synthetic
tinations with no terrorist conflicts. If it is in fact region was 2.5 times larger than GDP for the Basque Coun- try: this figure increased to more than 3 during the terrorism
the case that the Basque terrorist conflict has era. Furthermore, investment diverted to regions other than had a negative economic effect on other Spanish those in the synthetic Basque Country does not affect the regions, this effect is arguably weaker than the validity of our analysis.
can do this for all control units and compare them with the treated unitKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 14 / 22
502 Journal of the American Statistical Association, June 2010
Figure 4. Per-capita cigarette sales gaps in California and placebogaps in all 38 control states.
provide a good fit for per capita cigarette consumption priorto Proposition 99 for the majority of the states in the donorpool. However, Figure 4 indicates also that per capita cigarettesales during the 1970–1988 period cannot be well reproducedfor some states by a convex combination of per capita ciga-rette sales in other states. The state with worst fit in the pre-Proposition 99 period is New Hampshire, with a MSPE of 3437.The large MSPE for New Hampshire does not come as a sur-prise. Among all the states in the donor pool, New Hampshireis the state with the highest per capita cigarette sales for everyyear prior to the passage of Proposition 99. Therefore, there isno combination of states in our sample that can reproduce thetime series of per capita cigarette sales in New Hampshire priorto 1988. Similar problems arise for other states with extremevalues of per capita cigarette sales during the pre-Proposition 99period.
If the synthetic California had failed to fit per capita ciga-rette sales for the real California in the years before the pas-sage of Proposition 99, we would have interpreted that muchof the post-1988 gap between the real and the synthetic Cal-ifornia was also artificially created by lack of fit, rather thanby the effect of Proposition 99. Similarly, placebo runs withpoor fit prior to the passage of Proposition 99 do not provideinformation to measure the relative rarity of estimating a largepost-Proposition 99 gap for a state that was well fitted priorto Proposition 99. For this reason, we provide several differentversions of Figure 4, each version excluding states beyond acertain level of pre-Proposition 99 MSPE.
Figure 5 excludes states that had a pre-Proposition 99 MSPEof more than 20 times the MSPE of California. This is a verylenient cutoff, discarding only four states with extreme valuesof pre-Proposition 99 MSPE for which the synthetic methodwould be clearly ill-advised. In this figure there remain a fewlines that still deviate substantially from the zero gap line in thepre-Proposition 99 period. Among the 35 states remaining inthe figure, the California gap line is now about the most unusualline, especially from the mid-1990s onward.
Figure 5. Per-capita cigarette sales gaps in California and placebogaps in 34 control states (discards states with pre-Proposition 99MSPE twenty times higher than California’s).
Figure 6 is based on a lower cutoff, excluding all states thathad a pre-Proposition 99 MSPE of more than five times theMSPE of California. Twenty-nine control states plus Californiaremain in the figure. The California gap line is now clearly themost unusual line for almost the entire post-treatment period.
In Figure 7 we lower the cutoff even further and focusexclusively on those states that we can fit almost as wellas California in the period 1970–1988, that is, those stateswith pre-Proposition 99 MSPE not higher than twice the pre-Proposition 99 MSPE for California. Evaluated against the dis-tribution of the gaps for the 19 remaining control states in Fig-ure 7, the gap for California appears highly unusual. The nega-tive effect in California is now by far the lowest of all. Becausethis figure includes 19 control states, the probability of estimat-
Figure 6. Per-capita cigarette sales gaps in California and placebogaps in 29 control states (discards states with pre-Proposition 99MSPE five times higher than California’s).
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 15 / 22
Model-based Justification
The main motivating factor analytic model:
Yit (0) = γt + δ>t Xi + ξ>t Ui + εit
Generalization of the linear two-way fixed effects modelKey assumption: there exist weights such that
N∑i=2
wiXi = X1 andN∑
i=2
wiUi = U1
Another motivating autoregressive model with time-varyingcovariates:
Yit (0) = ρtYi,t−1(0) + δ>t Xit + εit
Xit = λt−1Yi,t−1(0) + ∆t−1Xi,t−1 + νit
Past outcomes can affect current treatmentNo unobserved time-invariant confounders
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 16 / 22
Generalizing the Difference-in-Differencesstaggered treatment What if units go in and out of treatment?
1960 1970 1980 1990 2000 2010
Year
Cou
ntrie
s
treatment Autocracy (Control) Democracy (Treatment)
Democracy as the Treatment
ireland
netherlands
switzerland
sweden
norway
denmark
belgium
japan
usa
uk
france
austria
finland
new zealand
italy
canada
korea
australia
germany
1850 1900 1950 2000
Year
Cou
ntrie
s
treatment Peace (Control) War (Treatment)
War as the Treatment
Acemoglu et al. 2019. J. Political Econ. Scheve and Stasavage. 2012.
Am. Political Sci. Rev.Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 17 / 22
Matching Methods for Panel Data (Imai et al. 2019. Working Paper)
Choose the number of lags L and leads FATE of Policy Change for the Treated:
E{
Yi,t+F
(Zit = 1,Zi,t−1 = 0, {Zi,t−`}L`=2
)−
Yi,t+F
(Zit = 0,Zi,t−1 = 0, {Zi,t−`}L`=2
)| Zit = 1,Zi,t−1 = 0
}Estimation procedure:
1 Construct a matched set for each treated unit that consists ofcontrol units with the identical treatment history up to L time periods
2 Refine covariate balance with a matching/weighting method withina matched set
3 Use the multi-period difference-in-differences estimator:
1∑Ni=1∑T−F
t=L+1 Zit
N∑i=1
T−F∑t=L+1
Zit
(Yi,t+F − Yi,t−1)−∑
i′∈Mit
w i′it (Yi′,t+F − Yi′,t−1)
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 18 / 22
Empirical Application (1)
ATT with L = 4 and F = 1,2,3,4We consider democratization and authoritarian reversalExamine the number of matched control units18 (13) treated observations have no matched control
Democratization
0 20 40 60 80 100 120
05
1015
2025
30
Four Year Lags 9 Emtpy SetsOne Year Lag 0 Emtpy Sets
Fre
quen
cy
Authoritarian Reversal
0 20 40 60 80 100 120
05
1015
2025
30
Four Year Lags 5 Emtpy SetsOne Year Lag 0 Emtpy Sets
Fre
quen
cy
Starting War
0 5 10 15 20
05
1015
2025
30
Four Year Lags 2 Emtpy SetsOne Year Lag 0 Emtpy Sets
Fre
quen
cy
Ending War
0 5 10 15 20
05
1015
2025
30
Four Year Lags 18 Emtpy SetsOne Year Lag 17 Emtpy Sets
Fre
quen
cy
Ace
mog
lu e
t al.
(201
8)S
chev
e &
Sta
sava
ge (
2012
)
Number of matched control units
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 19 / 22
Improved Covariate Balance−
2−
10
12
−4 −3 −2 −1
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Dem
ocra
tizat
ion
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Aut
horit
aria
n R
ever
sal
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Sta
rtin
g W
ar
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Sta
rtin
g W
ar
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Sta
rtin
g W
ar
Sta
ndar
dize
d M
ean
Diff
eren
ces
for
Sta
rtin
g W
ar
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
−2
−1
01
2
−4 −3 −2 −1
Mahalanobis Distance Matching
Propensity Score Matching
Propensity Score Weighting
Before Matching
Before Refinement
Ace
mog
lu e
t al.
(201
8)S
chev
e &
Sta
sava
ge (
2012
)
Years relative to the administration of treatmentKosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 20 / 22
Estimated Causal Effects
● ●●
● ●
Up to 5 matches
−0.
2−
0.1
0.05
0.15
0 1 2 3 4
● ● ● ● ●
Up to 10 matches
0 1 2 3 4
● ● ● ●●
Up to 5 matches
0 1 2 3 4
● ● ● ● ●
Up to 10 matches
0 1 2 3 4
● ● ● ● ●
0 1 2 3 4
●
● ● ●●
−0.
2−
0.1
0.05
0.15
0 1 2 3 4
●
● ● ●●
0 1 2 3 4
●
● ● ●●
0 1 2 3 4
●
● ● ●●
0 1 2 3 4
●
● ● ● ●
0 1 2 3 4
Mahalanobis Matching Propensity Score Matching Propensity Score Weighting
Est
imat
ed E
ffect
of
Dem
ocra
tizat
ion
Est
imat
ed E
ffect
of
Aut
horit
aria
n R
ever
sal
Years relative to the administration of treatment
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 21 / 22
Concluding Remarks
Difference-in-differences design:fully exploit the panel data structurecross sectional and before-and-after designs do notparallel trend assumptionadjusts for time-invariant unobserved confounderstradeoff between dynamics and unobservables
Extensions:adjusting for baseline covariatesnonlinear difference-in-differencessysthetic controlsmultiperiod difference-in-differences estimator
Readings: ANGRIST AND PISKE. CHAPTER 5
Kosuke Imai (Harvard University) Difference-in-Differences Designs Causal Inference (Fall 2019) 22 / 22
top related