Download - Steve Pudney Francesca Zantomio Institute for Social & Economic Research, University of Essex
Estimating the impact of a policy reform on welfare participation:
The 2001 extension to the Minimum Income Guarantee for UK pensioners
Steve PudneyFrancesca Zantomio
Institute for Social & Economic Research, University of Essex
Ruth HancockDepartment of Health & Human Sciences, University of
Essex
13 May 2006(2)
Outline • Introduction
• The MIG system and the 2001 reform
• Data and entitlements simulation
• Statistical Analysis
• Results
• Conclusions
13 May 2006(3)
“Non take-up”
• People are observed NOT to claim means-tested welfare benefits to which they are entitled.
• Take-up rate measures:
by caseload: # claimants / # entitled benefit units
by proportion of claimed government expenditure
• The phenomenon seem to be particulary severe for pensioners in the UK
13 May 2006(4)
UK Pensioners Take-up
• 2 million pensioners living in low income households
• Government commitment to ensure a decent income in retirement
• Welfare benefits (incl. state pension) make up the entire income of 15 % of pensioners
• Still between a quarter and a third of entitled pensioners do not claim income support
• Policy relevance
13 May 2006(5)
“Non take-up” rationale
Qualitative research (Costigan et al.,1999; CAG, 2002) suggests that welfare participation involves claiming costs due to:
Information and transaction costs
• ignorance of the existence of the program• insufficient knowledge of the administrative procedures or
entitlement criteria• language difficulties• troubles in filling the forms and gathering required
information and documentation,• time, hassle…
Intangible social stigma costs- distatste for welfare income- self-humiliating feeling• fear of stigmatisation from others• fear of losing independence
13 May 2006(6)
“Take-up” literature
Social sciences literatures exploring various aspects of take-up behaviour (Kerr,1982; Hirsh and Rank 1999; Kayser and Frick,2001; Castranove et al., 2001)
Economic literature (Moffit,1983; Blundell et al., 1988; Duclos, 1995; Pudney et al., 2006; Hernandez et al.,2006)
• Claiming as utility maximizing choice: expected benefit vs claiming cost
• Entitlement amount acts as positive incentive to take-up
• Econometric approach:entitlemnts simulation and parametric modelling of take-up probability, estimation of stigma cost
13 May 2006(7)
Our approach
Use of a policy change to pensioners income support system to identify the impact of variation in entitlement on the take-up behaviour
Non parametric analysis (misspecification)
Comparison with traditional parametric approach
Family Resorces Survey data( samples of pensioners interviewed before and after the reform
came into force)
13 May 2006(8) The Minimum Income Guarantee (MIG) Income support for pensioners (60+), income and asset-
tested
– brings income up to a minimum level, B = G(X) – means– withdrawn at 100% as income rises
• MIG in 2000/1: – assets between £3,000 & £8,000 converted to notional income– no eligibility if assets > £8,000– system of age-related additions
• Reformed system in 2001/2:– real increase in guarantee level– abolition of age additions– asset limits raised to (£6,000, £12,000)
• Every eligible pensioner gained; some became newly-eligible
13 May 2006(9)
Allowances and Premiums
(£ per week)
single couple
Basic allowance 52.2 81.95
Pensioner Premium
26.25 40.00
Enhanced Pensioner Premium
28.65 43.4
Higher Pensioner Premium
33.85 49.10
Capital limit 3,000-8,000
PRE-REFORM
Allowances and Premiums
(£ per week)
single couple
Basic allowance 53.05(52.21)
83.25(81.93)
Pensioner Premium
39.10(38.48)
57.30(56.39)
Enhanced PensionerPremium.
39.10(38.48)
57.30(56.39)
Higher Pensioner Premuim
39.10(38.48)
57.30(56.39)
Capital limit 6,000-12,000
POST-REFORM
13 May 2006(10)
MIG recipients, entitled non recipients and caseload take-up ratesCouple Single
MaleSingle Female All
Number of Recipients (thousands)
1999/2000 240 240 900 1390
2000/1 260 250 920 1430
2001/2 280 270 960 1520
Range of Entitled non Recipients
1999/2000 90-170 60-170 220-460 390-770
2000/1 110-170 80-140 230-380 450-670
2001/2 170-260 90-160 310-480 600-870
Caseload Take-up Range
1999/2000 59-72 59-79 66-80 64-78
2000/1 60-69 65-76 70-80 68-76
2001/2 52-62 64-75 67-75 63-72
(Ranges are 95% confidence interval to reflect sampling errors); source: DWP (2004)
13 May 2006(11)
The evaluation problem • Rubin-type evaluation methods require an experimental
implementation and a comparison group outside the experiments
• Limitations:– assume a limited number of policy options
– a non-universalist approach to policy that allows small-scale experiments
• In the UK experiments are rare because these conditions aren’t fulfilled– lots of policy options are considered in advance of reform
– there is a centralised or universalist approach to policy
– policy is designed with a very short time horizon – by the time an experiment would be designed, implemented & evaluated, government has lost interest
• The dominant approach remains “behavioural” modelling, which allows generalisation from past experience to hypothetical new policy environments
• Raises issues of:– specification error
– other modelling problems, such as measurement error
13 May 2006(12)
Our evaluation problem• want to identify the average treatment effect of the reform
on the take-up of the treated
• there can be no “control” group since:– all before-reform eligible people were affected– non-eligible people have no take-up behaviour to study
• for any pensioner we observe the take-up behaviour only under one benefit regime (either before or after the reform)
“Selection on observables” approach:
Pre-reform interviewed pensionersare are matched in terms of pre-reform and post reform entitlements (and demographic
characteristics) to post-reform interviewed pensioners and used as conterfactual
13 May 2006(13)
Family Resources Survey Data
• Annual cross-section surveys for Department for Work & Pensions (DWP)
• 2 years’ data, for t = 0 (FY 2000/1) & 1 (FY 2001/2)• Pensioner units at least 5 years over state retirement
age, in single unit hholds, without mortgages/earned income
• Extensive data cleaning of benefits data to minimize the scope for measurement error
• Benefit entitlement is simulated under the pre-reform rules and post-reform rules
• Final sample of entitled pensioners contains n0 = 845 in 2000/1 & n1 = 756 in 2001/2
13 May 2006(14)
Notation• Entitlement: Bt
r = amount of simulated MIG entitlement in circumstances of year t = 0, 1 under benefit system r = 0, 1
• Take-up outcome: Ttr = binary indicator of take-up status
that would be observed in circumstances of year t = 0, 1 under benefit system r = 0, 1. Tt
t is only meaningful if true Bt
t > 0• Survey response: Rt = binary indicator of survey response
that would be encountered in year t if selected for interviewing
• We observe:{Xt , Tt
t, Bt0, Bt
1 | Rt = 1} for t = 0, 1• Three population groups:
– People ineligible before & after reform (Bt0 = 0 , Bt
1 = 0)– People ineligible before & eligible after (Bt
0 = 0 , Bt1 > 0)
– People eligible before & after (Bt0 > 0 , Bt
1 > 0)• Our concern is with response to change in generosity of the
benefit system, so we concentrate on group 3
13 May 2006(15)
Change intake-up rate
Empirical take-up rates, pre- and post-reform(standard errors in parentheses)
Population groupPre-reform take-up rate(FRS 2000/1)
Post-reform take-up rate(FRS 2001/2)
Mean entitlement(£ per week)(FRS 2000/1)
Mean entitlement(£ per week)(FRS 2001/2)
Change in mean
entitlement(£ per week)
Single disabledn2000/1 = 189; n2001/2 =189
.577(.036)
.651(.034)
.074(.050)
44.68(1.739)
48.21 (1.858)
3.53(2.545)
Couple, at least one disabledn2000/1 =57; n2001/2 =34
.579(.066)
.618(.085)
.039(.108)
37.23(4.742)
41.37 (5.345)
4.14(7.145)
Single aged below 70n2000/1 =66; n2001/2 =38
.864(.043)
.868 (.055)
0.004(.070)
16.47(1.749)
32.31(3.689)
15.84(4.083)
Single aged 70-74n2000/1 =106; n2001/2 =97
.632(.047)
.835(.038)
0.203(.060)
16.14(1.663 )
29.42(2.105)
13.28(2.683)
Single aged 75-79n2000/1 =116; n2001/2 =119
.690(.043)
.731(.041)
0.041(.059)
14.58(1.396)
28.60(2.050)
14.02(2.480)
Single aged 80 or aboven2000/1 =215; n2001/2 =189
.637(.033)
.582(.036)
-0.055(.049)
18.56(1.251)
18.87(1.001)
0.31(1.602)
Couple at least one aged above 74n2000/1 =57; n2001/2 =45
.491(.067)
.311(.070)
-0.181(.097)
19.58(3.930)
31.23(4.244)
11.65(5.784)
Couple both below 74, one below 68n2000/1 =18; n2001/2 =21
.444(.120)
.476(.112)
0.032(.164)
48.69(10.485)
52.07(8.928)
3.38(13.771)
Couple both below 74, one above 68n2000/1 =21; n2001/2 =24
.381(.108)
.708(.095)
0.327(.144)
35.62(11.786)
47.71(7.022)
12.09(13.719)
All groupsn2000/1 =845; n2001/2 =756
.624(.017)
.656(.017)
.032(.024)
25.78(.911)
33.35(.942)
7.57(1.310)
13 May 2006(16)
Definitions & assumptionsCrude change in take-up rate
Causal impact of reform:
Assumptions: A1 conditionally ignorable non-responce
A1 Rt Ttt | Wt = w, for t = 0, 1 and all w, where Wt
= (Xt , Bt0, Bt
1)
A2 no counfunding macro level changes, conditional on Wt
A2 E(Ttr | Wt = w) independent of t for r = 0, 1 and all
w
Under these assumptions:= causal effect + effect of change in distribution of covariates
13 May 2006(17)
Matching approach
• Matching on variables, not PSM: “selection” into year 0 & year 1 samples is random so no confounding except for non-response
• stratification on Xt (sex, age, marital status, disability)
St k 1
9
Stk , t 0,1
• Nearest neighbour algorithm, distance minimization:
Di, ji Di, j j S1k
For each individual i within stratum S0k the matched individualj i S1k, satisfies
• Mahalanobis distance measure on (Bt0 , Bt
1)
• matching with replacement
• caliper options: reject matches breaching D i,j i
13 May 2006(18)
Matching approachStratum k estimator for change in take-up
k
1
n 0k iM0k
T1j i
1 T0i0
overall estimator of the reform-induced change
k 1
9
k k
Mkt is the set of nkt individuals in stratum k in year t, for whom a match can be found
where k is the relative size of stratum k in the baseline year t
13 May 2006(19)
Impact with 2000/1 characteristics:
*0
Impact with 2001/2 characteristics:
*1
Matching estimates with and without demographic stratification and caliper options(standard errors in parentheses)
caliper: none Є = 0.05
Є = 0.025
Є = 0.01
none Є = 0.05
Є = 0.025
Є = 0.01
2 strata: singles, couples.053
(.038).048
(.047).037
(.048).040
(.048).088
(.036).086
(.044).084
(.044).095
(.045)
3 strata: singles by gender, couples
.046(.039)
.036(.054)
.031(.055)
.027(.049)
.088(.037)
.086(.052)
.086(.052)
.087(.080)
9 strata: demographic groups(marital status, age, disability)
.035(.035)
.019(.046)
.021(.046)
.031(.057)
.086(.036)
.060(.043)
.067(.045)
.056(.055)
No stratification.092
(.040).090
(.042).085
(.042).082
(.043).094
(.037).099
(.038).091
(.038).092
(.039)
Results
13 May 2006(20)
Estimate of impact with 2000/1 characteristics: 0
Estimate of impact with 2001/2 characteristics: 1
Matching estimates by increase in entitlement, with and without stratification(standard errors in parentheses)
Size of increase in entitlement
Number of cases
Take-up rate2000/01
Take-up rate2001/02
(matched)
no stratification
2 strata 3 strata 9 strata
< £10 per week 542 0.638 0.6640.026
(0.051).007
(.065)-.013(.075)
.013(.093)
£10-15 per week 223 0.668 0.7710.103
(0.158).023
(.080).025
(.086).040
(.106)
>£15 per week 80 0.400 0.7370.337
(0.161)0.300
(0.145)0.303
(0.147)0.137
(0.106)
Size of increase in entitlement
Number of cases
Take-up2000/01
(matched)
Take-up2001/02
no stratification
2 strata 3 strata 9 strata
< £10 per week 484 0.566 0.6200.054
(0.048).045
(.058).017
(.068).047
(.088)
£10-15 per week 192 0.417 0.7710.354
(0.157).094
(.142).145
(.149).118
(.116)
>£15 per week 80 0.237 0.6000.362
(0.127)0.296
(0.097)0.290
(0.099)0.241
(0.101)
13 May 2006(21)
0%
20%
40%
60%
80%
100%
0.00
00
0.00
40
0.00
80
0.01
20
0.01
60
0.02
00
0.02
40
0.02
80
0.03
20
0.03
60
0.04
00
0.04
40
0.04
80
0.05
20
Distance threshold
Matching qualityPercentage of cases matched with D(i, j) < threshold
13 May 2006(22)
Parametric approach
PrT i 1|x i x i
0a 1
n 0 iS 0
x i1 A T0i
0
0p 1
n 0 iS 0
x i1 A x i
0 B
Probit model
Predicted change in the take-up rate (eg When i is sampled in 2000/1)
13 May 2006(23)
0ˆ a
0ˆ p
1ˆ a
1ˆ p
0ˆ a
0ˆ p
1ˆ a
1ˆ p
Parametric results: Predicted change in take-up rate (standard errors in parentheses)
Size of increase in entitlement
all < £10 per week £10-15 per week >£15 per week
Separate probits for each year, whole sample, full covariates set
.052(.018)
.015(.022)
.084(.032)
.212(.066)
.052(.002)
.038(.002)
.072(.004)
.104(.011)
n 839 540 221 78
.084(.018)
.048(.024)
.125(.025)
.138(.061)
.084(.004)
.058(.004)
.140(.015)
.123(.013)
n 753 408 267 78
Separate probits for each year, whole sample; same regressors as stratification variables in non parametric analysis
.069(.016)
.036(.020)
.098(.030)
.216(.056)
.069(.002)
.052(.002)
.096(.005)
.118(.010)
n 842 542 122 78
.100(.017)
.066(.024)
.139(.025)
.142(.058)
.100(.004)
.066(.003)
.135(.007)
.154(.015)
n 753 408 267 78
13 May 2006(24)
Conclusions• Even without a real or natural experiment or a control
group, it is worth trying to use nonparametric matching methods as a complement to parametric simulation models
• The take-up of MIG was significantly increased by the reform
• The effect was particularly large for those with largest potential gains from claiming
• Overall, a conventional probit take-up model for the 2001 MIG reform for UK pensioners works OK compared to the nonparametric results; but there is some evidence of probit under-prediction of the response to reform by people facing large gains – with potentially important welfare implications
• Next…
13 May 2006(25)
13 May 2006(26)
Pre-reform period
Publicity effect
Reform effect
Post-reform period
Publicity campaign
Time
Tak
e-up
pr
ope
nsit
y
2000 MIG publicity campaign
13 May 2006(27)
Balance of covariates with pair matching, no caliper, no stratification
Matching with 2000/1 characteristics
VariablesMean
treatedMean control
Mean matched control
Std % biasBefore
matching
Std % biasafter matching
%Reduction in Absolute Bias
Entitlement before reform 25.78 24.30 25.50 5.7 1.1 80.8
Entitlement after reform 34.74 33.35 34.57 5.3 0.7 87.4
Matching with 2001/2 characteristics
VariablesMean
treatedMean control
Mean matched control
Std % biasbefore
matching
Std % biasafter matching
%Reduction in Absolute Bias
Entitlement before reform 24.30 25.78 24.17 -5.7 0.5 90.9
Entitlement after reform 33.36 34.74 33.08 -5.3 1.1 79.9
13 May 2006(28) Take-up Probit estimates for single pensioners full covariates set
Sampled in 2000/1 Sampled in 2001/2
Regressor CoefficientStandard
errorCoefficient
Standard error
Owner -.444 .109 -.691 .118
Female .081 .154 -.175 .171
Black -.595 .448 -2.501 .644
Disabled .064 .118 .255 .132
Years worked -.001 .003 -.001 .003
Age -.086 .158 -.086 .169
Age2 .001 .001 .001 .001
Education -.032 .036 -.001 .019
ln entitlement .246 .055 .387 .091
Net income -.009 .001 -.011 .001
Constant 5.255 6.232 5.003 6.721
n 688 632
LR 2(10) 80.78 124.54
Pseudo R2 .0905 .1585
Log likelihood -405.789 -330.648
Test for parameter stability 2(10)=14.04
13 May 2006(29)Take-up Probit estimates for pensioners couples full covariates set
Sampled in 2000/2001 Sampled in 2001/2002
Covariate CoefficientStandard
errorCoefficient
Standard error
Owner -.560 .230 -.941 .256
Disabled head .256 .259 .561 .294
Disabled spouse .238 .246 .516 .276
Years worked head -.009 .009 .002 .010
Years worked spouse .010 .008 -.010 .008
Head’s education .087 .074 .014 .025
Spouse’s education -.040 .055 .061 .085
ln entitlement .356 .099 .406 .217
Net income -.001 .002 -.002 .003
Constant -1.399 1.337 -2.301 1.797
n 151 121
LR 2(9) 30.36 35.55
Pseudo R2 .1451 .2119
Log likelihood -89.455 -66.091
Test for parameter stability 2(9)=13.18
13 May 2006(30) Take-up probit estimates for pensioners stratification variables in non parametric analysis used as covariates
Sampled in 2000/2001 Sampled in 2001/2002
Covariate CoefficientStandard
errorCoefficient
Standard error
Single disabled -.532 .139 -.207 .153
Couple, at least one disabled -.312 .193 -.191 .247
Single aged below 70 .827 .216 .724 .278
Single aged 70-74 .065 .156 .607 .181
Single aged 75-79 .242 .153 .266 .157
Couple at least one aged over 74 -.380 .202 -.859 .227
Couple both below 74, one below 68 -.736 .338 -.558 .305
Couple both below 74, one over 68 -.473 .266 -.2597 .286
ln entitlement .356 .051 .430 .085
Constant -.555 .158 -.986 .255
n 842 753
LR 2(10) 82.38 81.20
Pseudo R2 0.0739 .0838
Log likelihood -515.975 -444.050
Chow test for parameters stability 2(10) = 16.5
13 May 2006(31)
D(i, j) = (P0i − P1j)’V −1 (P0i − P1j)