simplifying causal inference - harvard university
TRANSCRIPT
![Page 1: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/1.jpg)
Simplifying Causal Inference
Gary King
Institute for Quantitative Social ScienceHarvard University
(Talk at the Center for Population and Development Studies,Harvard University, 11/8/2012)
Gary King (Harvard, IQSS) Matching Methods 1 / 58
![Page 2: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/2.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 3: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/3.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 4: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/4.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 5: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/5.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 6: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/6.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 7: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/7.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 8: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/8.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 9: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/9.jpg)
Overview
Problem: Model dependence (review)
Solution: Matching to preprocess data (review)
Problem: Many matching methods & specifications
Solution: The Space Graph helps us choose
Problem: The most commonly used method can increase imbalance!
Solution: Other methods do not share this problem
(Coarsened Exact Matching is simple, easy, and powerful)
Lots of insights revealed in the process
Gary King (Harvard, IQSS) Matching Methods 2 / 58
![Page 10: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/10.jpg)
Model Dependence Example
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 11: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/11.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 12: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/12.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 13: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/13.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 14: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/14.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 15: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/15.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 16: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/16.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 17: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/17.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 18: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/18.jpg)
Model Dependence ExampleReplication: Doyle and Sambanis, APSR 2000
Data: 124 Post-World War II civil wars
Dependent variable: peacebuilding success
Treatment variable: multilateral UN peacekeeping intervention (0/1)
Control vars: war type, severity, duration; development status; etc.
Counterfactual question: UN intervention switched for each war
Data analysis: Logit model
The question: How model dependent are the results?
Gary King (Harvard, IQSS) Matching Methods 3 / 58
![Page 19: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/19.jpg)
Two Logit Models, Apparently Similar Results
Original “Interactive” Model Modified ModelVariables Coeff SE P-val Coeff SE P-valWartype −1.742 .609 .004 −1.666 .606 .006Logdead −.445 .126 .000 −.437 .125 .000Wardur .006 .006 .258 .006 .006 .342Factnum −1.259 .703 .073 −1.045 .899 .245Factnum2 .062 .065 .346 .032 .104 .756Trnsfcap .004 .002 .010 .004 .002 .017Develop .001 .000 .065 .001 .000 .068Exp −6.016 3.071 .050 −6.215 3.065 .043Decade −.299 .169 .077 −0.284 .169 .093Treaty 2.124 .821 .010 2.126 .802 .008UNOP4 3.135 1.091 .004 .262 1.392 .851Wardur*UNOP4 — — — .037 .011 .001Constant 8.609 2.157 0.000 7.978 2.350 .000N 122 122Log-likelihood -45.649 -44.902Pseudo R2 .423 .433
Gary King (Harvard, IQSS) Matching Methods 4 / 58
![Page 20: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/20.jpg)
Doyle and Sambanis: Model Dependence
Gary King (Harvard, IQSS) Matching Methods 5 / 58
![Page 21: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/21.jpg)
Model Dependence: A Simpler Example
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 22: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/22.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 23: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/23.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 24: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/24.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 25: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/25.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 26: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/26.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 27: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/27.jpg)
Model Dependence: A Simpler Example(King and Zeng, 2006: fig.4 Political Analysis)
What to do?
Preprocess I: Eliminate extrapolation region
Preprocess II: Match (prune bad matches) within interpolation region
Model remaining imbalance
Gary King (Harvard, IQSS) Matching Methods 6 / 58
![Page 28: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/28.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
Gary King (Harvard, IQSS) Matching Methods 7 / 58
![Page 29: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/29.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 8 / 58
![Page 30: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/30.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
T
CC
C
CC
C
C
C
C
C
C
C
C
C
C
C
C
CC C
C
C
C
C
C
C
C
C
C
C
C
CCC
CC
CC
C
C
Gary King (Harvard, IQSS) Matching Methods 9 / 58
![Page 31: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/31.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
T
CC
C
CC
C
C
C
C
C
C
C
C
C
C
C
C
CC C
C
C
C
C
C
C
C
C
C
C
C
CCC
CC
CC
C
C
Gary King (Harvard, IQSS) Matching Methods 10 / 58
![Page 32: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/32.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
T
CC
C
CC
C
C
C
C
C
C
C
C
C
C
C
C
CC C
C
C
C
C
C
C
C
C
C
C
C
CCC
CC
CC
C
C
Gary King (Harvard, IQSS) Matching Methods 11 / 58
![Page 33: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/33.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
T
CC
C
CC
C
C
C
C
C
C
C
C
C
C
C
C
CC C
C
C
C
C
C
C
C
C
C
C
C
CCC
CC
CC
C
C
Gary King (Harvard, IQSS) Matching Methods 12 / 58
![Page 34: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/34.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
TC
C
C
C
C
CC
C
C
CC
C CC
C
C
CCCC
C
CC
C
CC
CC
CC
C
C
C
C
CC
CCCC
Gary King (Harvard, IQSS) Matching Methods 13 / 58
![Page 35: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/35.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Education (years)
Out
com
e
12 14 16 18 20 22 24 26 28
0
2
4
6
8
10
12
T
T
T
T T
T
T
TTT
TT
T TT T
T
T
T
TC
C
C
C
C
CC
C
C
CC
C CC
C
C
CCCC
C
CC
C
CC
CC
CC
C
C
C
C
CC
CCCC
Gary King (Harvard, IQSS) Matching Methods 14 / 58
![Page 36: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/36.jpg)
Matching within the Interpolation Region(Ho, Imai, King, Stuart, 2007: fig.1, Political Analysis)
Matching reduces model dependence, bias, and variance
Gary King (Harvard, IQSS) Matching Methods 15 / 58
![Page 37: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/37.jpg)
How Matching Works
Notation:
Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 38: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/38.jpg)
How Matching Works
Notation:
Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 39: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/39.jpg)
How Matching Works
Notation:Yi Dependent variable
Ti Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 40: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/40.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)
Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 41: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/41.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 42: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/42.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 43: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/43.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 44: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/44.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 45: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/45.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 46: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/46.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 47: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/47.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 48: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/48.jpg)
How Matching Works
Notation:Yi Dependent variableTi Treatment variable (0/1, or more general)Xi Pre-treatment covariates
Treatment Effect for treated (Ti = 1) observation i :
TEi = Yi (Ti = 1)−Yi (Ti = 0)
= observed −unobserved
Estimate Yi (Ti = 0) with Yj from matched (Xi ≈ Xj) controls
Yi (Ti = 0) = Yj(Ti = 0) or a model Yi (Ti = 0) = g0(Xj)
Prune unmatched units to improve balance (so X is unimportant)
QoI: Sample Average Treatment effect on the Treated:
SATT =1
nT
∑i∈{Ti=1}
TEi
or Feasible Average Treatment effect on the Treated: FSATT
Gary King (Harvard, IQSS) Matching Methods 16 / 58
![Page 49: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/49.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 50: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/50.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 51: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/51.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)
Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 52: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/52.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)Match each treated unit to the nearest control unit
Control units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 53: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/53.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)Match each treated unit to the nearest control unitControl units: not reused; pruned if unused
Prune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 54: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/54.jpg)
Method 1: Mahalanobis Distance Matching
1 Preprocess (Matching)
Distance(Xi ,Xj) =√
(Xi − Xj)′S−1(Xi − Xj)Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 17 / 58
![Page 55: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/55.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
Gary King (Harvard, IQSS) Matching Methods 18 / 58
![Page 56: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/56.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 19 / 58
![Page 57: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/57.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
C
C
CC
C
C
C
C
C
CC
C
CCC
CC
C
C
C
CC CC
C
C
CC
C
CC
CC
C
C C
CC
C
C
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 20 / 58
![Page 58: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/58.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
C
C
CC
C
C
C
C
C
CC
C
CCC
CC
C
C
C
CC CC
C
C
CC
C
CC
CC
C
C C
CC
C
C
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 21 / 58
![Page 59: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/59.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
T TT T
TT TT T TTTTT TT
TTTT
CCC C
CC
C
C
C CCC
CC
CC C CC
C
C
CCCCC
CCC CCCCC
C CCCC
C
Gary King (Harvard, IQSS) Matching Methods 22 / 58
![Page 60: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/60.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
T TT T
TT TT T TTTTT TT
TTTT
CCC C
CC
C
C
C CCC
CC
CC C CC
C
Gary King (Harvard, IQSS) Matching Methods 23 / 58
![Page 61: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/61.jpg)
Mahalanobis Distance Matching
Education (years)
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
T TT T
TT TT T TTTTT TT
TTTT
CCC C
CC
C
C
C CCC
CC
CC C CC
C
Gary King (Harvard, IQSS) Matching Methods 24 / 58
![Page 62: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/62.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 63: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/63.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 64: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/64.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 65: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/65.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |
Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 66: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/66.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unit
Control units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 67: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/67.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unitControl units: not reused; pruned if unused
Prune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 68: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/68.jpg)
Method 2: Propensity Score Matching
1 Preprocess (Matching)
Reduce k elements of X to scalar πi ≡ Pr(Ti = 1|X ) = 11+e−Xi β
Distance(Xi ,Xj) = |πi − πj |Match each treated unit to the nearest control unitControl units: not reused; pruned if unusedPrune matches if Distance>caliper
2 Estimation Difference in means or a model
Gary King (Harvard, IQSS) Matching Methods 25 / 58
![Page 69: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/69.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
C
C
CC
C
C
C
C
C
CC
C
CCC
CC
C
C
C
CCCC
C
C
CC
C
CC
CC
C
C C
CC
C
C
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 26 / 58
![Page 70: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/70.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
C
C
CC
C
C
C
C
C
CC
C
CCC
CC
C
C
C
CCCC
C
C
CC
C
CC
CC
C
C C
CC
C
C
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
1
0
PropensityScore
Gary King (Harvard, IQSS) Matching Methods 27 / 58
![Page 71: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/71.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
C
C
CC
C
C
C
C
C
CC
C
CCC
CC
C
C
C
CCCC
C
C
CC
C
CC
CC
C
C C
CC
C
C
TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
1
0
PropensityScore
C
C
CC
CCC
C
C
C
CC
C
C
C
C
C
C
C
CCCCC
C
CCCCCCCCC
C
C
C
C
CC
T
TTT
T
TT
T
T
T
T
TT
T
T
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 28 / 58
![Page 72: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/72.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
1
0
PropensityScore
C
C
CC
CCC
C
C
C
CC
C
C
C
C
C
C
C
CCCCC
C
CCCCCCCCC
C
C
C
C
CC
T
TTT
T
TT
T
T
T
T
TT
T
T
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 29 / 58
![Page 73: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/73.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
1
0
PropensityScore
C
C
CC
CCC
C
C
C
CC
C
C
C
C
C
C
C
CCCCC
C
CCCCCCCCC
C
C
C
C
CC
CCC
C
CCC
C
CCC
CCCC
T
TTT
T
TT
T
T
T
T
TT
T
T
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 30 / 58
![Page 74: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/74.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
1
0
PropensityScore
CCC
C
CCC
C
CCC
CCCC
T
TTT
T
TT
T
T
T
T
TT
T
T
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 31 / 58
![Page 75: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/75.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
C
C
CC
C
CC
C
C
C
C
C
C
C
C
C
C
CC
C TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
1
0
PropensityScore
CCC
C
CCC
C
CCC
CCCC
T
TTT
T
TT
T
T
T
T
TT
T
T
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 32 / 58
![Page 76: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/76.jpg)
Propensity Score Matching
Education (years)
Age
12 16 20 24 28
20
30
40
50
60
70
80
C
C
CC
C
CC
C
C
C
C
C
C
C
C
C
C
CC
C TTTT
T
T
T
T
T
T
T
T
T
TT
T
T
T
T
T
Gary King (Harvard, IQSS) Matching Methods 33 / 58
![Page 77: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/77.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)
Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 78: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/78.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)
Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 79: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/79.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 80: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/80.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)
Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 81: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/81.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 82: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/82.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 83: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/83.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )
Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 84: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/84.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 85: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/85.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 86: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/86.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treateds
Can apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 87: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/87.jpg)
Method 3: Coarsened Exact Matching
1 Preprocess (Matching)Temporarily coarsen X as much as you’re willing
e.g., Education (grade school, high school, college, graduate)Easy to understand, or can be automated as for a histogram
Apply exact matching to the coarsened X , C (X )
Sort observations into strata, each with unique values of C(X )Prune any stratum with 0 treated or 0 control units
Pass on original (uncoarsened) units except those pruned
2 Estimation Difference in means or a model
Need to weight controls in each stratum to equal treatedsCan apply other matching methods within CEM strata (inherit CEM’sproperties)
Gary King (Harvard, IQSS) Matching Methods 34 / 58
![Page 88: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/88.jpg)
Coarsened Exact Matching
Gary King (Harvard, IQSS) Matching Methods 35 / 58
![Page 89: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/89.jpg)
Coarsened Exact Matching
Education
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
CCC CCC CC
C CC C CCC CCCCCCC CC CCC CCCCCC
C CCC CC C
T TT T
TT TT T TTTTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 36 / 58
![Page 90: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/90.jpg)
Coarsened Exact Matching
Education
HS BA MA PhD 2nd PhD
Drinking age
Don't trust anyoneover 30
The Big 40
Senior Discounts
Retirement
Old
CCC CCC CC
C CC C CCC CCCCCCC CC CCC CCCCCC
C CCC CC C
T TT T
TT TT T TTTTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 37 / 58
![Page 91: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/91.jpg)
Coarsened Exact Matching
Education
HS BA MA PhD 2nd PhD
Drinking age
Don't trust anyoneover 30
The Big 40
Senior Discounts
Retirement
Old
CCC CCC CC
C CC C CCC CCCCCCC CC CCC CCCCCC
C CCC CC C
T TT T
TT TT T TTTTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 38 / 58
![Page 92: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/92.jpg)
Coarsened Exact Matching
Education
HS BA MA PhD 2nd PhD
Drinking age
Don't trust anyoneover 30
The Big 40
Senior Discounts
Retirement
Old
CC C
CC
CC C CCC CC CCCC
C
TTT T TT
TTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 39 / 58
![Page 93: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/93.jpg)
Coarsened Exact Matching
Education
HS BA MA PhD 2nd PhD
Drinking age
Don't trust anyoneover 30
The Big 40
Senior Discounts
Retirement
Old
CC C
CCCC C CC
C CC CCCC
C
TTT T TT
TTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 40 / 58
![Page 94: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/94.jpg)
Coarsened Exact Matching
Education
Age
12 14 16 18 20 22 24 26 28
20
30
40
50
60
70
80
CC C
CCCC C CC
C CC CCCC
C
TTT T TT
TTT TT
TTTT
Gary King (Harvard, IQSS) Matching Methods 41 / 58
![Page 95: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/95.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 96: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/96.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 97: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/97.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 98: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/98.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 99: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/99.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 100: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/100.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)
Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 101: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/101.jpg)
The Bias-Variance Trade Off in Matching
Bias (& model dependence) = f (imbalance, importance, estimator) we measure imbalance instead
Variance = f (matched sample size, estimator) we measure matched sample size instead
Bias-Variance trade off Imbalance-n Trade Off
Measuring Imbalance
Classic measure: Difference of means (for each variable)Better measure (difference of multivariate histograms):
L1(f , g ;H) =1
2
∑`1···`k∈H(X)
|f`1···`k− g`1···`k
|
Gary King (Harvard, IQSS) Matching Methods 42 / 58
![Page 102: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/102.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 103: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/103.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 104: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/104.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 105: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/105.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 106: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/106.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 107: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/107.jpg)
Comparing Matching Methods
MDM & PSM: Choose matched n, match, check imbalance
CEM: Choose imbalance, match, check matched n
Best practice: iterate
But given the matched solution matching method is irrelevant
Our idea: Identify the frontier of lowest imbalance for each given n,and choose a matching solution
Gary King (Harvard, IQSS) Matching Methods 43 / 58
![Page 108: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/108.jpg)
A Space Graph: Foreign Aid Shocks & ConflictKing, Nielsen, Coberley, Pope, and Wells (2012)
Mahalanobis Discrepancy L1
Imbalance Metric
Difference in Means
N of Matched Sample
010
2030
40
2500 1500 500 0
●
●
published PSM
published PSM with 1/4 sd caliper
N of Matched Sample
0.0
0.4
0.8
2500 1500 500 0
●●
published PSM
published PSM with1/4 sd caliper
N of Matched Sample
0.00
0.10
0.20
0.30
2500 1500 500 0
●
●
published PSM
published PSM with 1/4 sd caliper
● Raw DataRandom Pruning
"Best Practices" PSMPSM
MDMCEM
Gary King (Harvard, IQSS) Matching Methods 44 / 58
![Page 109: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/109.jpg)
A Space Graph: Healthways DataKing, Nielsen, Coberley, Pope, and Wells (2012)
Gary King (Harvard, IQSS) Matching Methods 45 / 58
![Page 110: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/110.jpg)
A Space Graph: Called/Not Called DataKing, Nielsen, Coberley, Pope, and Wells (2012)
Gary King (Harvard, IQSS) Matching Methods 46 / 58
![Page 111: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/111.jpg)
A Space Graph: FDA Drug Approval TimesKing, Nielsen, Coberley, Pope, and Wells (2012)
Gary King (Harvard, IQSS) Matching Methods 47 / 58
![Page 112: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/112.jpg)
A Space Graph: Job Training (Lelonde Data)King, Nielsen, Coberley, Pope, and Wells (2012)
Gary King (Harvard, IQSS) Matching Methods 48 / 58
![Page 113: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/113.jpg)
A Space Graph: Simulated Data — Mahalanobis
MDM: 1 Covariate
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
MDM: 2 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
MDM: 3 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
Gary King (Harvard, IQSS) Matching Methods 49 / 58
![Page 114: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/114.jpg)
A Space Graph: Simulated Data — CEM
CEM: 1 Covariate
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
CEM: 2 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
CEM: 3 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
Gary King (Harvard, IQSS) Matching Methods 50 / 58
![Page 115: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/115.jpg)
A Space Graph: Simulated Data — Propensity Score
PSM: 1 Covariate
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
PSM: 2 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
PSM: 3 Covariates
N of matched sample
L1
500 250 0
0.0
0.5
1.0
HighMedLow
Imbalance:
Gary King (Harvard, IQSS) Matching Methods 51 / 58
![Page 116: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/116.jpg)
PSM Approximates Random Matching in Balanced Data
Covariate 1
Cov
aria
te 2
−2 −1 0 1 2
−2
−1
0
1
2
−2 −1 0 1 2
−2
−1
0
1
2
●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●
PSM MatchesCEM and MDM Matches
Gary King (Harvard, IQSS) Matching Methods 52 / 58
![Page 117: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/117.jpg)
CEM Weights and Nonparametric Propensity Score
CEM Weight: wi =mT
i
mCi
(+ normalization)
CEM Pscore: Pr(Ti = 1|Xi ) =mT
i
mTi + mC
i
CEM:
Gives a better pscore than PSM
Doesn’t match based on crippled information
Gary King (Harvard, IQSS) Matching Methods 53 / 58
![Page 118: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/118.jpg)
CEM Weights and Nonparametric Propensity Score
CEM Weight: wi =mT
i
mCi
(+ normalization)
CEM Pscore: Pr(Ti = 1|Xi ) =mT
i
mTi + mC
i
CEM:
Gives a better pscore than PSM
Doesn’t match based on crippled information
Gary King (Harvard, IQSS) Matching Methods 53 / 58
![Page 119: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/119.jpg)
CEM Weights and Nonparametric Propensity Score
CEM Weight: wi =mT
i
mCi
(+ normalization)
CEM Pscore: Pr(Ti = 1|Xi ) =mT
i
mTi + mC
i
CEM:
Gives a better pscore than PSM
Doesn’t match based on crippled information
Gary King (Harvard, IQSS) Matching Methods 53 / 58
![Page 120: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/120.jpg)
CEM Weights and Nonparametric Propensity Score
CEM Weight: wi =mT
i
mCi
(+ normalization)
CEM Pscore: Pr(Ti = 1|Xi ) =mT
i
mTi + mC
i
CEM:
Gives a better pscore than PSM
Doesn’t match based on crippled information
Gary King (Harvard, IQSS) Matching Methods 53 / 58
![Page 121: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/121.jpg)
CEM Weights and Nonparametric Propensity Score
CEM Weight: wi =mT
i
mCi
(+ normalization)
CEM Pscore: Pr(Ti = 1|Xi ) =mT
i
mTi + mC
i
CEM:
Gives a better pscore than PSM
Doesn’t match based on crippled information
Gary King (Harvard, IQSS) Matching Methods 53 / 58
![Page 122: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/122.jpg)
Destroying CEM with PSM’s Two Step Approach
Covariate 1
Cov
aria
te 2
−2 −1 0 1 2
−2
−1
0
1
2
−2 −1 0 1 2
−2
−1
0
1
2
●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
● ●
CEM MatchesCEM−generated PSM Matches
Gary King (Harvard, IQSS) Matching Methods 54 / 58
![Page 123: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/123.jpg)
Conclusions
Propensity score matching:
The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 124: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/124.jpg)
Conclusions
Propensity score matching:
The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 125: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/125.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 126: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/126.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original data
Can increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 127: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/127.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matches
Approximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 128: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/128.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 129: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/129.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reduction
Implications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 130: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/130.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 131: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/131.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking required
Adjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 132: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/132.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistake
Adjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 133: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/133.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistake
Reestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 134: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/134.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake
1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 135: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/135.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 136: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/136.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score
(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 137: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/137.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)
CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 138: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/138.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problems
You can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 139: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/139.jpg)
Conclusions
Propensity score matching:The problem:
Imbalance can be worse than original dataCan increase imbalance when removing the worst matchesApproximates random matching in well-balanced data(Random matching increases imbalance)
The Cause: unnecessary 1st stage dimension reductionImplications:
Balance checking requiredAdjusting for potentially irrelevant covariates with PSM: mistakeAdjusting experimental data with PSM: mistakeReestimating the propensity score after eliminating noncommonsupport: mistake1/4 caliper on propensity score: mistake
In four data sets and many simulations:CEM > Mahalanobis > Propensity Score(Your performance may vary)CEM and Mahalanobis do not have PSM’s problemsYou can easily check with the Space Graph
Gary King (Harvard, IQSS) Matching Methods 55 / 58
![Page 140: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/140.jpg)
For papers, software (for R, Stata, & SPSS), tutorials, etc.
http://GKing.Harvard.edu/cem
Gary King (Harvard, IQSS) Matching Methods 56 / 58
![Page 141: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/141.jpg)
Data where PSM Works Reasonably Well — PSM & MDM
−4 −2 0 2 4
−4
−2
02
4
Unmatched Data: L1 = 0.685
Covariate 1
Cov
aria
te 2
−4 −2 0 2 4
−4
−2
02
4
PSM: L1 = 0.452
Covariate 1
Cov
aria
te 2
−4 −2 0 2 4
−4
−2
02
4
MDM: L1 = 0.448
Covariate 1
Cov
aria
te 2
Gary King (Harvard, IQSS) Matching Methods 57 / 58
![Page 142: Simplifying Causal Inference - Harvard University](https://reader036.vdocuments.us/reader036/viewer/2022071003/62c83380a0837b213813721f/html5/thumbnails/142.jpg)
Data where PSM Works Reasonably Well — CEM
−4 −2 0 2 4
−4
−2
02
4
Bad CEM: L1 = 0.661
Covariate 1
Cov
aria
te 2
100% of the treated units
−4 −2 0 2 4
−4
−2
02
4
Better CEM: L1 = 0.188
Covariate 1
Cov
aria
te 2
100% of the treated units
−4 −2 0 2 4
−4
−2
02
4
Even Better CEM: L1 = 0.095
Covariate 1
Cov
aria
te 2
72% of the treated units
Gary King (Harvard, IQSS) Matching Methods 58 / 58