slides for* · 2020. 8. 14. · too many slides here for 30 minutes but hopefully the extra slides...

1

SLIDES FOR*

Comparing the Conley-Taber and the Standard Approaches to Inference in Difference-in-Difference Models Based on Small Policy Variation: The Case

of TennCare

John C. Ham

NYU Abu Dhabi and Wagner School of Public Service, NYU

IFAU, IRP and IZA

Ken Ueda The Office of the Comptroller

Of the Currency

Revised September 2018

* We thank Corina Mommaerts for her comments on these slides without implicating her.

2

Introduction and Motivation

Many empirical studies focus on a small

number of policy changes in a few locations.

The extreme version of this is one policy

change in one year in one state. In this case,

Conley and Taber (2011) show under

reasonable assumptions about how one does

the asymptotics, one cannot estimate the

treatment effect consistently, but can

estimate consistently a confidence interval

for the treatment effect.

3

Garthwaite, Gross and Notowidigdo (2014,

hereafter r GGN), using data from the March

CPS, analyzed the effect of a major

contraction in Medicaid coverage in

Tennessee’s TennCare program in 2000’s.

They aggregated the micro data up to the

state-year level. GGN got very large

estimated effects, and on the basis of these,

predicted the Affordable Care Act (ACA),

also known as Obamacare, would have

substantial labor market effects.

4

However several papers have investigated

this issue and found no labor market impact

of the ACA. But perhaps their analysis did

not offer enough power to reject the null

hypothesis of the ACA having no effect even

if the GGN estimates are ‘correct’.

Their regression results seemed very high

compared to other studies, see e.g.,

The survey, Buchmueller, Ham and Shore-

Sheppard (2016).

5

To investigate this, we estimated their models

for two much larger data sets i) all months

of the CPS (ALLCPS) and iii) American

community survey (ACS).

We find quite big differences from those

obtained from the March CPS (MCPS) data.

These do not go away even when we keep

only the estimates that pass diagnostic tests.

6

In terms of the latter, Diff-in-Diff (DD)

estimation using the March CPS produces a

big significantly positive impact on

employment, while the ACS produces

significantly negative impact.

But for Triple Difference (DDD) estimation

the DDD March CPS estimate doubles, while

the ACS estimate becomes significantly

positive.

We also show below that the GGN point

estimates should have enough power to reject

the null hypothesis of no labor market effect

of the ACA, although this is not true of all of

the estimates in their CI.

7

We hypothesize that the above problems of

the regression occur because the model is not

identified in the sense of Conley and Taber

(ReStat 2011, CT hereafter) – everything is

coming from one policy change in one state

in one year. This is a widely cited paper in

empirical papers and theoretical papers, but

seems to be not widely implemented to this

point.

We then estimate TC estimate consistent

Confidence Intervals (CIs) for the TennCare

effect.

8

Again we apply specification tests based on

CT’s consistent CIs for placebo coefficients

and find that we can reject the null of no

placebo effect for several specifications.

Our results for the CT CIs, using the

specifications that survive the tests, produce

a number of fairly wide confidence intervals

that are consistent with the null hypothesis of

no TennCare effect, and hence more in

keeping with the previous literature.

9

But we do not believe that one can claim that

this result is just a case of CT generally

producing noisy CIs, since their CIs were

precise enough in the placebo tests to reject

several specifications.

Too many slides here for 30 minutes but

hopefully the extra slides will help those

interested in the absence of a paper.

10

Paper Organization

1. Review of TennCare.

2. Discuss the GGN regression approach for

the MCPS – specification and results.

3. Review of i) previous research on the

impact of being offered public health

insurance on labor market outcomes ii)

research on measuring the labor market

effects of the ACA.

4. Use the regression approach on the

ALLCPS and ACS.

11

5. Conduct a power analysis for the ACA

impact using the MCPS, ALLCPS, and

ACS regression estimates. Evaluate power

at the parameter estimates and obtain a

consistent CI for the power function

evaluated at the regression estimates using

the approach in Woutersen and Ham

(2018).

6. Apply specification (placebo) tests to the

results from the MCPS, ALLCPS and ACS.

The surviving estimates are very different

across data sets.

7. Review the CT approach to estimating

consistent estimates of the CI for the

treatment effect.

12

8. Use the CT approach on the MCPS,

ALLCPS and ACS. We also conduct power

analysis using their results. Here we can

only provide CIs for the power function,

since CT does not produce consistent

parameter estimates.

9. Apply specification (placebo) tests to the

CT CIs from the MCPS, ALLCPS and

ACS. Surviving estimates are quite similar

across data sets. All CIs include zero and

are fairly wide.

10. Conclude.

13

TennCare Basics

• Started in 1994 to cover

“uninsurable”/“uninsured” individuals.

• Prevent loss of federal funds/Emergency

Room utilization.

• Enrolled all Medicaid recipients into

Managed Care plans, used savings to cover

“uninsurables” – i.e. people who were

rejected by private insurance plans; no

income restrictions. Examples: Displaced

workers, children whose parents did not have

access to workplace insurance.

14

• TennCare faced a $342 million shortfall for

2001.

• BCBST (Blue Cross Blue Shield Tennessee),

which covered almost half of all TennCare

patients, threatens to pull out of TennCare

due to rising costs.

• 2002 – changed definition of eligibility

• Enrollee must have gone through medical

review of “insurability”

• “Reverification” - enrollees must

schedule appointments to determine if

they remained eligible for benefits.

15

Enrollee income distribution (1995)

• 40% had incomes above 100% of the

poverty line.

• 6.3% had incomes between 200% to

400% of poverty line

• 1.3% had incomes above 400% of

poverty line

We don’t have later breakdown since in our

data sets we don’t see income and public

insurance in the same year.

This would seem to contradict the argument

that the big effects in GGN occur because

TennCare covered a lot of people over 200%

of the poverty line.

16

In November 2004, Governor Philip

Bredesen announced that childless

individuals would not be covered as of July 1,

2005, and disenrollment started then.

17

Regression Estimates

The fact that the contraction started in July

2005 is important since we are going to look

at data from 2000-2007, where 2000-2005

are the control years and 2006-2007 in

Tennessee are the treatment years.

In the MCPS, 2005 is actually April 2004-

March 2005. For the ALLCPS and ACS, the

2005 year covers January-December 2005.

18

But there is the question of whether we

should include 2005 for all the data sets.

For the MCPS, the 2005 comparison year

included January-March 2005 period before

the TennCare contraction. But it was

announced in November 2004 that is would

be implemented in July 1, 2005. Hence the

announcement might have lead to an

anticipation effect.

For the ALLCPS and ACS, the 2005 control

year included January-June 2005 before the

TennCare contraction and June-December

2005 after the contraction.

19

We drop 2005, but surprisingly this has no

effect on the results from any data sets.

One could also worry about 2006 treatment

year for the MCPS since it covers April 2005-

March 2006, but in the absence of

anticipation effects, was not in place April –

June 2005.

20

GGN’s Approach

To evaluate the impact of TennCare GGN

use Difference in Difference Estimation

�� = �� + �� + �� + ��[��]

�[�� ≥ 2006]+ �� (1)

• where

��: employment rate within state �, year �,

�: Comparison of change in Tennessee

employment rate after Tenn Care disenrollment

vs change in employment rate in other southern

states or Tennessee before 2006.

21

• Comparison States: Alabama, Arkansas,

Delaware, DC, Florida, Georgia, Kentucky,

Louisiana, Maryland, Mississippi, North

Carolina, Oklahoma, Texas, Virginia, South

Carolina, West Virginia.

• Sample MCPS 2000-2007 for GGN.

• The treatment effect is identified by the

comparison of 2005 and 2006 employment in

their analysis, or 2004 and 2006 employment

after we drop 2005.

• Crucial Assumption – trends constant across

states, and we will test this. To relax this

assumption use a triple difference

specification.

22

• Since the TennCare contraction affected only

adults without children under 17 years old, so

GGN can eliminate state specific trends by

taking the difference between the two groups

in each state.

For childless adults they hypothesize that

1 1 1

1 1

{ [ ] * [ 2006]}

, (2)

st s t

st st

L I s Tenn I t

v

For adults with children

2 2 2 2 2 . (3)st s t st stL v

23

Assume that the state specific trends 1st and

2st are equal (i.e. 1 2st st ). Then subtracting

(3) from (2) yields

1 2 1 2 1 2 1 2

1 12

( ) ( ) ( )

{ [ ]* [ 2006]} ( - )

{ [ ]* [ 2006]} .

st st s s t t st st

st t

s t st

L L

I s Tenn I t v v

I s Tenn I t v

We will test the null hypothesis that 1 2st st

using placebo tests.

24

• Sample: 21-64 years old (inclusive), no

Military, at most college degree.

• We can replicate their results exactly.

25

Table 1 Regression Estimates of the TennCare Effect on Employment (Hours) Using the

March CPS from GGN

Difference-in-Difference

Triple Differences

(1) (2)

Working > 0 hours Point Estimate 0.025** 0.046** Standard Error (0.011) (0.020)

0<Working < 20 hours Point Estimate -0.001 0.002 Standard Error (0.004) (0.009)

Working ≥ 20 hours

Point Estimate 0.026*** 0.044**

Standard Error (0.010) (0.020)

Working ≥ 20 hours, < 35 hours Point Estimate 0.001 0.018 Standard Error (0.007) (0.013)

Working ≥ 35 hours Point Estimate 0.025** 0.026 Standard Error (0.011) (0.021)

Notes: *, **, and *** denote significance at the 10%, 5% and 1% level respectively.

26

There is a question which estimates/hours

group we should focus on.

We would argue that if the we are thinking

people are getting jobs to get Employer

Sponsored Insurance (ESI), the hours > 35 is

the most plausible.

GGN argue that hours > 20 is the most

relevant because part-time workers also can

get ESI.

27

But Carroll and Miller use the MEPS data to

study the extend of ESI for full-time and part-

time workers, where part-time is < 30 hours

per week.

They show that in 2005, 85.6% of full-time

workers were eligible for ESI and 70.6% had

it.

However they show that in 2005, 39.7% of

part-time workers were eligible for ESI and

15.1% had it.

Full-Time defined as hours >=30, Part-time defined as hours <=30.

From Carroll and Miller (2018).

28

Hence we would argue that hours > 35 is the

most appropriate category. Below we also

consider hours > 0. In the next round we will

add hours > 20 to the presentation. Our

results do not change when we include them.

29

How do the MCPS estimates compare to

previous results:

There is a large body of literature on the

employment effects of Medicaid eligibility,

and most studies find small or non-existent

effects.

For example, Yelowitz (1995), from the

March CPS, found large employment effects

of Medicaid eligibility among single

mothers.

30

But Ham and Shore-Sheppard (2005), using

data from the March CPS and from SIPP, that

his results were an artifact of constraining

welfare benefits and Medicaid availability to

have the same coefficient. Once this

constraint was relaxed, welfare benefits, but

not Medicaid eligibility, continued to affect

employment.

31

Further, Meyer and Rosenbaum (2001), using

data from the Current Population Survey

(CPS) Outgoing Rotation Group Files and

from the March CPS, found an important role

for welfare benefits, but not Medicaid

provisions, in a static model of labor force

participation.

Recently Finkelstein et al. (2014) found that

offering individuals Medicaid coverage in the

Oregon Health Experiment had essentially no

effect on employment; since their result is

based on a randomized trial, this evidence is

perhaps the strongest to date.

32

What happens if we estimate the treatment

effect using the ALLCPS or ACS?

Below we drop 2005 but this has very little

effect on any of the estimates.

33

Table 2 Regression Estimates of the TennCare Effect on

Employment (Hours) by Database, 2000-2007, Omit 2005 GGN MCPS ALLCPS ACS (1) (2) (3) (4)

Panel A: Difference in Differences

Working > 0 hours Point

Estimate 0.025** 0.021** 0.015*** -0.013***

(0.011) (0.011) (0.004) (0.004) 95 Percent

CI [0.003, 0.047] [0.000, 0.042] [0.007, 0.023] [-0.021, -0.005]


Point Estimate

0.025** 0.022* 0.016*** -0.015***

(0.011) (0.012) (0.004) (0.004) 95 Percent

CI [0.003, 0.047] [-0.001, 0.045] [0.008, 0.024] [-0.024, -0.006]

Panel B: Triple Difference

Working > 0 hours

Point Estimate

0.046** 0.043** 0.005 0.002

(0.020) (0.022) (0.008) (0.006) 95 Percent

CI [0.007, 0.085] [0.000, 0.086] [-0.010, 0.020] [-0.010, 0.014]


Point Estimate

0.026 0.017 -0.006 0.013*

(0.021) (0.023) (0.009) (0.007) 95 Percent

CI [-0.015, 0.067] [-0.028, 0.062] [-0.023, 0.012] [0.000, 0.026]

Microdata

216,751 1,789,894 2,491,229

Notes: All Standard Errors calculated as in GGN.

34

If we include 2005 we get

Table 3 Regression Estimates of the TennCare Effect on Employment (Hours)

by Database, 2000-2007 GGN MCPS ALLCPS ACS (1) (2) (3) (4)


Working > 0 hours Point Estimate 0.025** 0.025** 0.016*** -0.011***

(0.011) (0.011) (0.004) (0.004) 95 Percent CI [0.003, 0.047] [0.003, 0.047] [0.008, 0.024] [-0.019, -0.003]


Point Estimate 0.025** 0.025** 0.016*** -0.013*** (0.011) (0.011) (0.004) (0.004)

95 Percent CI [0.003, 0.047] [0.003, 0.047] [0.008, 0.024] [-0.021, -0.005]


Working > 0 hours Point Estimate 0.046** 0.046** 0.007 0.004

(0.020) (0.020) (0.008) (0.006) 95 Percent CI [0.007, 0.085] [0.007, 0.085] [-0.009, 0.023] [-0.008, 0.016]


Point Estimate 0.026 0.026 -0.002 0.015 ** (0.021) (0.021) (0.008) (0.006)

95 Percent CI [-0.015, 0.067] [-0.015, 0.067] [-0.018, 0.014] [0.003, 0.027]

Microdata 249,559 2,057,701 3,036,337

35

Summary of DD Estimates

• For both h > 0 and h > 35 specifications

get a significant positive treatment effect from the

MCPS and CPS, but a significant negative

treatment effect from the ACS.

Summary of DDD Estimates

• For h > 0 specification

MCPS estimates double when compared to the DD

estimates and significantly positive, but ALLCPS

and ACS are insignificant and have quite small CIs.

• For h > 35 specification

MCPS estimates are insignificant with a big CI.

ALLCPS estimates are insignificant with a small

CI. ACS are insignificant and have quite small CIs.

Hence the results are unstable across data sets.

36

Power calculations

If the TennCare effects are valid, should the

papers investigating the introduction of the

ACA have found significant effect?

Given a treatment effect, we simulate how

often we expect to be able to reject the null

hypothesis that there was no ACA effect

given above coefficients regression. For now

we consider the DD estimates.

The idea here is that others may have found

no ACA effect even if the above estimates are

valid because of a lack of power.

37

Let be our estimated TennCare Treatment

effect. Our power calculation is ˆ( ),POW

i.e. the fraction of time we simulate getting a

significant effect for the ACA introduction

and the TennCare parameters.

To get a consistent confidence interval for

ˆ( ),POW we need to take into account that

it is a random variable because of are two

sources of randomness here – from the

simulations and from the fact that is a

random variable.

38

If we just use the simulation at the point

estimates we not are considering randomness

from the fact that is estimated. Note that

we cannot use the delta method to calculate a

consistent confidence interval for ˆ( )POW

because ˆ( )POW is a non-differentiable

function.

39

To incorporate randomness from the fact that

is estimated, we might be tempted to a

large number of draws from the asymptotic

distribution of , call them ˆs , and then

calculate ˆ( )sPOW for each s. Then we

could take the top 2.5% of the ˆ( )sPOW

values and the bottom 2.5% of the ˆ( )sPOW

values, and drop them both to get a 95% CI

for ˆ( ).POW

Unfortunately, there is no consistency proof

for this approach, and Woutersen and Ham

(2018) construct counter-examples where it

does not produce consistent CIs.

40

• However, Woutersen and Ham (2018) show

that we can get a consistent confidence

interval for ˆ( )POW by taking a large

number of draws from the 95% confidence

interval for . Call them ˆs , and calculate

ˆ( )sPOW for each s. Then we take minimum

value of ˆ( )sPOW over these draws ˆs as the

lower limit of the CI for ˆ( )sPOW , and take

maximum value of ˆ( )sPOW over these

draws ˆs for the upper limit of the CI for

ˆ( )sPOW .

41

Table 4 Power Calculations Using DD Regression Coefficients

Point Estimate,

March CPS

March CPS

Point Estimate, All CPS

All CPS

Working>0

Expanded in 2014 with no prior expansions vs Did not expand 2014 with no prior expansions

0.97 [0.044, 1] 0.99 [0.576, 1]

Expanded in 2014 with no prior expansions vs Did not expand 2014

1.00 [0.042, 1] 1.00 [0.648, 1]

Working>35

Expanded in 2014 with no prior expansions vs Did not expand 2014 with no prior expansions

0.98 [0.042, 1] 0.99 [0.578, 1]

Expanded in 2014 with no prior expansions vs Did not expand 2014

1.00 [0.040, 1] 1.00 [0.650, 1]

(We did not do the calculations for the ACS since the

treatment effects are often negative for this data set and

because of time constraints.)

42

• We see that, at the point estimates, we would

certainly expect to see the null hypothesis of

no ACA effect rejected almost all the time.

• But the CI for the power function based on

the CIs for the TennCare treatment effect are

very wide and are consistent with low power

for rejecting the null that the ACA effect is

zero given the CIs for the TennCare

coefficients.

43

Placebo Tests

1. For the DD equations, we estimate

regressions for 2000-2004 we use year

dummies and

i) put in a dummy for 2003-2004 for

Tennessee and

ii) Put in a dummy for 2002-2004 for

Tennessee.

• This gives us 2 tests of the null hypothesis

that Tennessee has the same trend as the

comparison states for each data set.

44

2. For the DDD equations, we estimate

regressions for 2000-2004 where we use

year dummies and

i) Put in a dummy for 2003-2004 for

Tennessee and

ii) Put in a dummy for 2002-2004 for

Tennessee.

This again gives us 2 tests of the null

hypothesis that the difference in trend

between those with children and those

without in Tennessee has the same trend as

the same groups for comparison states for

each data set.

45

Table 5 Regression Estimates of Placebo Treatment Effect

2002-2004 Using 2000-2004 Data MCPS ALLCPS ACS (1) (2) (3)


Working > 0 hours

Point Estimate -0.022** -0.009** 0.003 (0.011) (0.004) (0.004)

95 Percent CI [-0.045, 0.000] [-0.017, 0.000] [-0.005, 0.012]

Working ≥ 35 hours Point Estimate -0.015 0.001 0.000

(0.013) (0.005) (0.005)

95 Percent CI [-0.041, 0.011] [-0.009, 0.011] [-0.010, 0.009]


Working > 0 hours

Point Estimate 0.004 -0.011 -0.006 (0.024) (0.008) (0.008)

95 Percent CI [-0.044, 0.051] [-0.028, 0.005] [-0.021, 0.009]

Working ≥ 35 hours Point Estimate -0.022 -0.023** -0.011

(0.026) (0.009) (0.009) 95 Percent CI [-0.074, 0.029] [-0.041, -0.005] [-0.029, 0.006]

Microdata

149,253 1,247,650 1,355,926

• For DD h>0 we reject the model for the MCPS and the

ALLCPS.

• For DDD h>35 we reject the model for the ALLCPS.

46

Table 6 Regression Estimates of Placebo Treatment Effect

2003-2004 Using 2000-2004 Data MCPS ALLCPS ACS (1) (2) (3)


Working > 0 hours

Point Estimate -0.031** -0.018*** 0.005 (0.012) (0.004) (0.004)

95 Percent CI [-0.054, -0.008] [-0.026, -0.010] [-0.003, 0.013]

Working ≥ 35 hours Point Estimate -0.016 -0.012*** 0.004

(0.013) (0.004) (0.004) 95 Percent CI [-0.041, 0.010] [-0.020, -0.004] [-0.004, 0.013]


Working > 0 hours

Point Estimate -0.007 -0.018** -0.006 (0.023) (0.008) (0.008)

95 Percent CI [-0.052, 0.038] [-0.033, -0.003] [-0.022, 0.010]

Working ≥ 35 hours Point Estimate -0.032 -0.018** -0.009

(0.025) (0.009) (0.009) 95 Percent CI [-0.081, 0.016] [-0.035, -0.001] [-0.026, 0.008]

Microdata

149,253 1,247,650 1,355,926

47

• For DD h > 0 we reject the model for the

MCPS and the ALLCPS. For DD h > 35 we

reject the model for the ALLCPS.

• For DDD h>0 and DDD h>35 we reject the

model for the ALLCPS.

48

Table 2 Revisited

TennCare Impacts from Table 2 in Bold that Survive Both Placebo Tests in Tables 4 and 5

MCPS ALLCPS ACS (1) (2) (3)


Working > 0 hours

Point Estimate 0.021** 0.015*** -0.013*** (0.011) (0.004) (0.004)

95 Percent CI [0.000, 0.042] [0.007, 0.023] [-0.021, -0.005]

Working ≥ 35 hours Point Estimate 0.022* 0.016*** -0.015***

(0.012) (0.004) (0.004)

95 Percent CI [-0.001, 0.045]

[0.008, 0.024] [-0.024, -0.006]


Working > 0 hours

Point Estimate 0.043** 0.005 0.002 (0.022) (0.008) (0.006)

95 Percent CI [0.000, 0.086] [-0.010, 0.020] [-0.010, 0.014]

Working ≥ 35 hours Point Estimate 0.017 -0.006 0.013*

(0.023) (0.009) (0.007)

95 Percent CI [-0.028, 0.062]

[-0.023, 0.012] [0.000, 0.026]

Still a wide range of estimates that do not fail either placebo test.

49

• Why are the MCPS’ results different from the

ALLCPS and ACS’ results? One possibility

is that, e.g. 2003 in the MCPs covers April

2002-March 2003, while in the ALLCPS and

ACS 2003 covers January 2003-December

2003. We thought this may be more

important for those in more volatile

industries in the data sets. So we delete those

working in Construction and Manufacturing

in all the data sets, but this makes no

difference. Also it couldn’t explain

differences between the ALLCPS and ACS.

50

Table 7 Regression Estimates 2000-2007, Without 2005,

Omit Construction and Manufacturing


Panel A: Difference in Difference

Working > 0 hours Point Estimate 0.026** 0.015*** -0.010**

(0.013) (0.005) (0.005) 95 Percent CI [0.001, 0.050] [0.005, 0.025] [-0.019, -0.001]


Point Estimate 0.023* 0.017*** -0.011** (0.014) (0.005) (0.005)

95 Percent CI [-0.003, 0.050] [0.007, 0.026] [-0.020, -0.002]


Working > 0 hours Point Estimate 0.047* 0.011 -0.001

(0.024) (0.009) (0.007) 95 Percent CI [-0.001, 0.094] [-0.007, 0.028] [-0.015, 0.013]


Point Estimate 0.008 -0.003 0.009 (0.026) (0.009) (0.008)

95 Percent CI [-0.043, 0.059] [-0.022, 0.015] [-0.006, 0.023]

Microdata 178,274 1,477,819 2,022,102

51

Can we make the MCPS, ALLCPS and ACS

estimates look more similar by only looking at

those living in the Metropolitan Areas because

these individuals are overrepresented in the

ACS?

52

Table 8: Regression Estimates 2000-2007, Without 2005, Only Metropolitan Areas


Panel A: Difference in Difference

Working > 0 hours

Point Estimate 0.018 0.002 -0.012** (0.012) (0.004) (0.006)

95 Percent CI [-0.006, 0.042] [-0.006, 0.010] [-0.024, -0.001]

Working ≥ 35 hours Point Estimate 0.021 0.007 -0.014**

(0.013) (0.005) (0.007) 95 Percent CI [-0.005, 0.047] [-0.003, 0.017] [-0.027, -0.001]


Working > 0 hours

Point Estimate 0.039* 0.005 0.000 (0.024) (0.009) (0.009)

95 Percent CI [-0.008, 0.086] [-0.013, 0.022] [-0.017, 0.017]

Working ≥ 35 hours Point Estimate 0.010 -0.014 0.013

(0.027) (0.010) (0.009) 95 Percent CI [-0.042, 0.063] [-0.034, 0.006] [-0.006, 0.031]

Microdata

166,519 1,365,850 2,224,792

53

Move to Conley-Taber consistently estimated CIs

• Conley and Taber (2011) show you will not get

consistent estimates of the treatment effect if you assume that you are only adding comparison states.

• However they also show that you can consistently estimate the confidence interval for the treatment effects. Here we will get 87.5% CIs for their approach but can get more conventional ones (e.g 95%) by following their suggestions and adding more comparisons.

54

Table 9 Conley-Taber Consistent 87.5% Confidence Intervals for the TennCare Effect: on

Employment (Hours) by Database, 2000-2007, Omit 2005

MCPS ALLCPS ACS

(1) (2) (3)

Panel A: Difference-in-Difference

Working > 0 hours

Regression [0.004, 0.038]* [0.009, 0.021]* [-0.019, -0.007]*

Taber Conley [0.001, 0.046]* [-0.005, 0.040] [-0.025, 0.004]

Working > 35 hours

GGN [0.004, 0.040]* [0.009, 0.023]* [-0.022, -0.008]*

Taber Conley [0.002, 0.042]* [-0.001, 0.035] [-0.031, -0.002]


Working > 0 hours

GGN [0.009, 0.077]* [-0.007, 0.017] [-0.007, 0.011]

Taber Conley [0.001, 0.074]* [-0.017, 0.037] [-0.010, 0.017]

Working > 35 hours

GGN [-0.013, 0.057] [-0.020, 0.008] [0.003, 0.023]*

Taber Conley [-0.024, 0.047] [-0.036, 0.044] [-0.001, 0.023]

Microdata 216,751 1,789,894 2,491,229

55

Table 10 Power Calculations Based on Conley-Taber Consistent Confidence Intervals

MCPS

Confidence Intervals

ALLCPS Confidence

Intervals

Working>0

87.5% CIs When the ACA is expanded in 2014 for states with no prior expansions vs states that did not expand 2014 with no prior expansions

[0.042, 1] [0.102, 1]

87.5% CIs When the ACA is Expanded in 2014 for states with no prior

expansions vs Did not expand 2014 [0.070, 1] [0.150, 1]

Working>35

87.5% CIs When the ACA is expanded in 2014 for states with no prior expansions vs states that did not expand 2014 with no prior expansions

[0.048, 1] [0.048, 1]

87.5% CIs When the ACA is Expanded in 2014 for states with no prior

expansions vs Did not expand 2014 [0.098, 1] [0.088, 1]

Again consistent with a wide set of power values.

56

Placebo Tests for CT approach

Table 12 Conley-Taber Consistent 87.5% Confidence Intervals: Placebo Treatment Effect 2002-

2004, Using 2000-2004 Data


Difference in Difference

Working > 0 hours

GGN [-0.040, -0.005]* [-0.016, -0.002]* [-0.004, 0.010] Conley-Taber [-0.041, -0.007]* [-0.021, 0.003] [-0.008, 0.014]


GGN [-0.036, 0.006] [-0.006, 0.009] [-0.007, 0.007] Conley-Taber [-0.033, -0.003]* [-0.007, 0.008] [-0.016, 0.014]

Triple Difference

Working > 0 hours

GGN [-0.033, 0.040] [-0.024, 0.001] [-0.018, 0.006] Conley-Taber [-0.034, -0.011]* [-0.024, 0.012] [-0.002, 0.009]


GGN [-0.063, 0.018] [-0.037, -0.009]* [-0.025, 0.003] Conley-Taber [-0.023, -0.002]* [-0.019, 0.017] [-0.004, 0.005]

Microdata

149,253 1,247,650 1,355,926

57

Table 13 87.5 % Confidence Intervals for Regression Approach and CT Approach: Placebo

Treatment Effect 2003-2004, Using 2000-2004 Data


Difference in Difference

Working > 0 hours

GGN [-0.049, -0.013]* [-0.025, -0.012]* [-0.002, 0.011] Conley-Taber [-0.042, -0.012]* [-0.028, -0.008]* [-0.004, 0.015]


GGN [-0.036, 0.004] [-0.019, -0.006]* [-0.003, 0.011] Conley-Taber [-0.035, 0.010] [-0.020, -0.006]* [-0.008, 0.015]

Triple Difference

Working > 0 hours

GGN [-0.042, 0.029] [-0.030, -0.007]* [-0.018, 0.007] Conley-Taber [-0.040, -0.019]* [-0.035, -0.004]* [-0.003, 0.011]


GGN [-0.070, 0.005] [-0.031, -0.005]* [-0.022, 0.005] Conley-Taber [-0.025, -0.002]* [-0.041, 0.025] [-0.003, 0.010]

Microdata

149,253 1,247,650 1,355,926

58

Table 9 Revisited

Conley-Taber Consistent 87.5% Confidence Intervals for TennCare Impacts from Table 9 in Bold that Survive both Placebo Treatment Tests in Tables 12 and 13


Panel A: Difference-in-Difference

Working > 0 hours

87.5 Percent Confidence Interval [0.001, 0.046]* [-0.005, 0.040] [-0.025, 0.004]

Working > 35 hours 87.5 Percent Confidence Interval [0.002, 0.042]* [-0.001, 0.035] [-0.031, -0.002]


Working > 0 hours

87.5 Percent Confidence Interval [0.001, 0.074]* [-0.017, 0.037] [-0.010, 0.017]

Working > 35 hours 87.5 Percent Confidence Interval [-0.024, 0.047] [-0.036, 0.044] [-0.001, 0.023]

59

Our results for the CT CIs, using the

specifications that survive the tests, produce

a number of fairly wide confidence intervals

across the data sets that do not contradict each

other and that are consistent with the null

hypothesis of no TennCare effect. Hence

more in keeping with the previous literature.

• But recall that for the Regression results,

those that survive the placebo tests have

quite different implications and

contradictory results.

60

Table 2 Revisited: TennCare Impacts from Table 2 in Bold that Survive Both Placebo Tests in Tables 4 and 5



Working > 0 hours Point Estimate 0.021** 0.015*** -0.013***

(0.011) (0.004) (0.004)

95 Percent CI [0.000, 0.042] [0.007, 0.023] [-0.021, -0.005]


Point Estimate 0.022* 0.016*** -0.015*** (0.012) (0.004) (0.004)

95 Percent CI [-0.001, 0.045] [0.008, 0.024] [-0.024, -0.006]


Working > 0 hours Point Estimate 0.043** 0.005 0.002

(0.022) (0.008) (0.006) 95 Percent CI [0.000, 0.086] [-0.010, 0.020] [-0.010, 0.014]


Point Estimate 0.017 -0.006 0.013* (0.023) (0.009) (0.007)

95 Percent CI [-0.028, 0.062] [-0.023, 0.012] [0.000, 0.026]

slides for* · 2020. 8. 14. · too many slides here for 30 minutes but hopefully the extra slides...

Documents