The Impact of Financial Incentives on Education Choice1

Lorraine Dearden, Carl Emmerson, Christine Frayne

and Costas Meghir

Institute for Fiscal Studies

March 2002

Handout prepared for CEE conference

Preliminary and incomplete – Incomplete referencing

1 ACKNOWLEDGEMENTS: The funding for this work has been provided by the DfES under the project for evaluating the Education Maintenance Allowance. We also thank our colleagues at the National Centre and CRIS for our collaboration on this project.

Introduction and Motivation

Education participation rates post the UK’s compulsory school leave age of 16 have

been traditionally quite low. The resulting low education levels for a large proportion

of the working population has been considered a source of skill shortage and a reason

for persistent poverty. The issue was considered even more serous as a result of the

increases in unemployment for low skill workers that have occurred since the early

1980s. These low levels of education are considered all the more surprising given the

relatively high returns to education. Thus the question is whether the low levels of

education were due to costs or perhaps liquidity constraints.

Over the years the participation rates in post-compulsory education increased

substantially reaching levels of 60% or 70% and far more in some areas. Moreover,

the percentage of individuals ending up with some formal school qualification also

increased, signifying possibly an increase in the educational level of those leaving

school at 16. Nevertheless, this still left a substantial proportion of individuals with

very low levels of schooling, which has been considered inefficient. The presumption

of the policy makers has been that these low levels of education are due to constraints

rather than to the outcome of an informed choice in an unconstrained environment.

There is little direct evidence on the importance or otherwise of liquidity constraints,

and this is not completely clear-cut (see Cameron and Heckman (1998), Cameron and

Taber (2000) and Dale and Krueger (1999)). Moreover, there is certainly no evidence

in the UK on this issue. Hence the UK government decided to pilot a scheme that

would offer grants to pupils from low-income families if they continued school post

16. We have collected data on this programme based on interviews with parents and

children in the pilot areas as well as in a set of carefully selected control areas. The

evaluation of this programme cannot necessarily provide information on the

importance of liquidity constraints on education, since it changes the relative costs of

remaining in school. However, it can provide valuable information on the sensitivity

of schooling choices on education costs. This is a key input in any policy that seeks to

increase participation in education. Moreover, in contrast to US studies that consider

the impact of tuition subsidies on College participation, this programme targets

younger, less educated pupils, i.e. the equivalent to those deciding whether to

continue or to drop out of high school. The group the government wishes to target are

thoe equivalent of US high school dropouts.

In this paper we use the data we have collected to evaluate the impact of education

subsidies on post-compulsory schooling choices. We consider the choice at 16 (to

continue or to drop out) and a year later. Thus we are able to measure the impact of

the subsidy on high school completion. The programme was not randomly allocated

to school districts (Local Education Authorities, LEAs). It was implemented fully in a

small number of areas. We had to pick a number of control areas in which we

collected information to define eligibility as well as a host of other characteristics.

Given the non-randomness of the programme we use propensity score matching to

balance the characteristics of the treatment and control sample. Of course our

assumption is that there is no selection on unobservables. This assumption is quite

credible in our context because of the large array of observable characteristics and

because there has been no opportunity of moving between areas to take advantage of

the reform. We show in this paper that the impact of the subsidy is quite substantial

both in the first year and subsequently. Of course we are not yet in a position to

measure the labour market returns for those who changed their behaviour due to the

programme, nor to assess whether the potential General Equilibrium effects will

reduce the impact of the programme if it is implemented on a national scale.

The paper proceeds as follows. In Section 2 we describe the programme and its

variants. In section 3 we describe the data and we compare the characteristics of the

treatment and control areas. In section 4 we discuss briefly the evaluation

methodology and in section 5 we discuss the results. In section 6 we offer some

concluding remarks.

The Education Maintenance Allowance

The Education Maintenance Allowance (EMA) pilots were launched in September

1999 in 10 Local Education Authorities. The scheme pays a means-tested benefit to

16-18 year-olds who remain in education after year 11. The payments consist of

weekly allowance (during term time only), a retention bonus every term and an

achievement bonus at the end of the course. The benefit can be claimed for up to 2

years (or 3 for young people with special needs). The pilot schemes ran for two

cohorts, so young people reaching the school leaving age in both 1999 and 2000 could

apply for the benefit.

Four different variants of the EMA are being tried out, with differences in the

generosity of the scheme and whether it is paid to a parent or the young person.

Details of the different variants are given in the table below. The basic EMA variant 1

was piloted in 3 urban areas and 1 rural area. Variants 2, 3 and 4 were all piloted in 2

urban areas.

Table 1. The Four Variants of EMA Variant Maximum

weekly EMA award

Weekly payment paid to

Retention bonus (per term)

Achievement bonus

1 £30 Young person £50 £50 2 £40 Young person £50 £50 3 £30 Parent £50 £50 4 £30 Young person £80 £140 In each area the full amount of EMA could be received by young people whose

parents incomes were £13,000 below or less.2 This was tapered away linearly so that

2 Income is defined as the taxable income of the biological parents in the previous tax year.

individuals whose parents income was £30,000 were eligible for £5 a week of EMA

with those whose parents earned more than £30,000 not eligible for any award.

The Data - Characteristics of the Pilot and Control Areas – Pre-reform

The programme was announced in the spring, just before the end of the school year.

The data used to evaluate the programme is based on a face-to-face interview with the

parents and the children. The data set was constructed so as to include both eligible

and ineligible individuals in both pilot and control areas. The first interview was

conducted at the beginning of the school year when the subsidy became available and

once those willing to participate in school could have implemented their decision. In

the following year the first cohort was followed up using a telephone interview.

Moreover a new cohort was interviewed in the same way as the first, with a face-to-

face interview. The sample sizes for the first and second cohort are give in Table 2.

There has been some attrition in the second wave of cohort 1 where about 76% of the

original sample was successfully followed up. In Table 3 we provide characteristics of

those who were interviewed in both waves and those who we only saw in wave 1. It is

clear from this information that those who dropped out of the sample were of lower

socio-economic background. Looking at the pilot areas only we find that the

participation rate for those who dropped out of the sample was 18% points lower than

those who were successfully reinterviewed. Controlling for observable characteristics

this diffeence declines to 7% percentage points with a standard error of 0.02. Thus

the impact of attrition may not be fully accounted for by observable characteristics

and we need to keep this in mind when interpreting the results. Interestingly the

attrition rate is similar in pilot and control areas, thus mitigating any problems that

attrition may create in the evaluation.

Table 2. Sample sizes Rural Urban Total Cohort 1, Wave 1 2,769 7,034 9,803 Cohort 1, Wave 2 2,231 5,226 7,457 Cohort 2, Wave 1 2,707 7,120 9,827

Table 3. Characteristics of those who attrit and those who did not attrit from the sample Non-attriters Attriters All Number of observations 7,457 2,346 9,803 Proportion staying on in ft education 0.782 0.582 0.734 Proportion going to work 0.125 0.199 0.143 Proportion in a pilot area 0.611 0.623 0.614 Weekly family income 417.00 311.64 391.78 Family receives means-tested benefit 0.21 0.34 0.24 Mother and Father figure present 0.67 0.48 0.62 Father figure present 0.79 0.66 0.76 Owner occupier 0.76 0.55 0.71 Council or Housing Association 0.19 0.35 0.23 Has statemented special needs 0.09 0.12 0.10 Mothers age 40.76 37.02 39.86 Fathers age 32.41 24.52 30.52 Mother has a levels or higher 0.29 0.18 0.27 Mother has o levels or equivalent 0.26 0.23 0.25 Father has a levels or higher 0.27 0.14 0.24 Fatther has o levels or equivalent 0.19 0.14 0.18 Father manager or professional 0.21 0.12 0.19 Father clerical or similar 0.25 0.20 0.24 Mother manager or professional 0.16 0.10 0.14 Mother clerical or similar 0.31 0.23 0.29 Father variables missing 0.32 0.48 0.36 1 or 2 parents in work when born 0.87 0.76 0.85 Attended 2 primary schools 0.28 0.26 0.27 Attended more than 2 primary schools 0.09 0.11 0.09 Received childcare as a child 0.92 0.87 0.91 1 set of Grandparents around when child 0.33 0.31 0.32 1 sets of Grandparents around when child 0.41 0.43 0.41 Grandparents provided care when child 0.29 0.30 0.29 Ill between 0 and 1 0.22 0.23 0.22 Number of older siblings 0.91 1.07 0.95 Number of younger siblings 0.91 0.98 0.93 Older sibling educated to 18 0.33 0.26 0.31 White 0.93 0.90 0.92 Father in full-time work 0.56 0.39 0.52 Father in part-time work 0.02 0.02 0.02 Mother in full-time work 0.35 0.28 0.33 Mother in part-time work 0.35 0.26 0.33 Achieved A–C GCSE Maths 0.51 0.32 0.46 Achieved A–C GCSE English 0.62 0.42 0.57 Multiple deprivation score 32.68 36.28 33.54 Income 26.60 29.03 27.18 Employment 14.74 15.67 14.96 Health Deprivation and Disability 0.76 0.89 0.79 Education, Skills and Training 0.50 0.73 0.56 Housing 0.24 0.45 0.29 Geographical Access to Services –0.29 –0.41 –0.32 Child poverty 38.54 42.30 39.44 per cent not staying on post 16 64.17 64.60 64.27 per cent not going to university 86.07 85.40 85.91 Class sizes in 1999 21.42 21.32 21.40 Authorised absences 8.51 8.75 8.57 % getting 5 GCSE A–C in 1999 39.78 37.55 39.25 % getting 0 GCSE A–G in 1999 5.68 6.33 5.83 School has 6th form? 0.46 0.43 0.46 Distance to nearest year 12 provider 2536.65 2242.65 2466.29

We now turn to a description of the pilot and control areas. The areas do differ in

some respects and it is important that this is taken into account. Tables 4 and 5

provide some pre-reform statistics on pilot and control areas, while Table A1 in the

appendix provides definitions of the deprivation indices we report (which are

computed by the government). When considering rural areas the pilot are clearly more

deprived and have lower school participation rates than the control areas. This is

partly due o the fact that the rural pilot area is Cornwall and it is very hard to find

areas sufficiently similar. However the urban pilot and control areas are much more

similar; nevertheless if there are any differences there again the pilot areas are more

deprived.

Table 4. Local area based variables – rural areas only

Variable Pilot Control Total Number of observations 2,076 1,936 4,012 Measures of local deprivation (index) Multiple deprivation score 27.61 23.29 25.53 Income 23.96 20.40 22.24 Employment 13.53 10.78 12.20 Health Deprivation and Disability 0.52 0.15 0.34 Education, Skills and Training –0.03 0.29 0.13 Housing 0.22 -0.19 0.02 Geographical Access to Services 0.25 0.19 0.22 Child poverty 35.68 29.95 32.92 Education participation rates per cent not staying on post 16 56.76 60.77 58.70 per cent not going to university 82.95 85.03 83.95 Nearest school data Class sizes in 1999 21.84 21.86 21.85 Authorised absences 8.08 8.18 8.13 % getting 5 GCSE A–C in 1999 52.41 46.15 49.39 % getting 0 GCSE A–G in 1999 3.12 4.21 3.65 School has 6th form? 0.57 0.59 0.58 Distance to nearest year 12 provider 4,022.72 4,237.83 4,126.52

Table 5. Local area based variables – urban areas only Variable Pilot Control Total Number of observations 7,266 3,607 10,873 Measures of local deprivation (index) Multiple deprivation score 40.43 38.88 39.91 Income 31.52 31.28 31.44 Employment 17.26 16.92 17.15 Health Deprivation and Disability 1.10 1.03 1.07 Education, Skills and Training 0.90 0.80 0.87 Housing 0.56 0.45 0.53 Geographical Access to Services –0.56 -0.62 -0.58 Child poverty 45.81 44.94 45.52 Education participation rates per cent not staying on post 16 69.53 67.16 68.74 Per cent not going to university 89.28 88.24 88.93 Nearest school data Class sizes in 1999 21.24 21.25 21.24 Authorised absences 8.78 9.05 8.87 % getting 5 GCSE A–C in 1999 34.06 33.92 34.01 % getting 0 GCSE A–G in 1999 7.08 6.84 7.00 School has 6th form? 0.44 0.34 0.40 Distance to nearest year 12 provider 1,578.23 1,958.04 1,704.23

The evaluation Methodology – Matching

Since the allocation to treatment and control was not random we have to be concerned

about possible systematic differences between treatment and control groups. The main

concern is of course that those in pilot areas differ in a systematic way from those in

the control area. One important advantage here is that there is limited or no scope for

moving as a response to the reform. This is because the reform was announced

relatively late in the year and the samples drawn soon after that. However this in no

way implies that the distribution of characteristics is similar in the treatment and

control area, but it may well imply that if we condition on a well-chosen set of

observables, similar (probably) to those used by policymakers to identify the pilot

areas we are likely to remove the effects of any composition bias.

Within each pilot area there are three types of individual: Those whose family income

makes them ineligible for the subsidy, those whose family income makes them

eligible for the full amount and finally, those on the taper, who receive some positive

subsidy which is less than the full amount. Thus in general the subsidy that

individuals receive (and hence the intensity of the treatment) will vary with household

income. One would wish to quantify the impact of the programme as a function of this

intensity. On the other hand household income may well be correlated with

unobservables affecting education choices. Consequently it may be difficult to

compare outcomes at different levels of income since the differences may be

confounded by heterogeneity. Thus, we follow two different approaches. In the first

approach we simply identify eligible and ineligible individuals and we allow the

impact of the treatment to vary by observable characteristics. We then also consider

differences between those with the full subsidy and those on the taper, but we do not

necessarily attribute a causal interpretation to the differences between the groups. The

approach allows for heterogeneity of treatment impact. Obviously, part of the

heterogeneity here is due to the fact that the treatment itself varies by the level of

income.

The outcome of interest in this paper will be participation in post-compulsory school,

i.e year 12 and year 13. Suppose the outcome of an individual with characteristics Xi

who is exposed to the EMA is Y . The same individual would have outcome Y were

she/he not to be exposed to the treatment. Obviously, either one or the other outcome

is observed.

1i

0i

The impact of the policy for the ith individual (Y ) is thus not observed. The

main evaluation parameter that we will estimate is the impact of treatment on the

treated, i.e. , where P is one for individuals in the pilot areas and

zero in the control areas. What we do observe is , which is the average

participation rate for those exposed to the EMA. To construct the counterfactual

we assume that which means

01ii Y−

|( 1ii PYE

() 0iYE=

)1|( 01 =− iii PYYE

)1

)1=

| iP =|( 0 =ii PYE ),0,1|( 0iiii XXPYE =

that given the observable characteristics the allocation to treatment and control is

random. Under this assumption it is now well known (see Rosenbaum and Rubin,

1983) that where

is the propensity score and is simply the probability of being allocated

to the pilot given observed characteristics. It follows that we can estimate the

counterfactual by the sample analog of

))|,1Pr(,0|())|,1Pr(,1|( 00iiiiiiii XPPYEXPPYE =====

))]|,1Pr(,0|([ 01 iiiiF XPPYEE ==

)|,1Pr( ii XP =

)1|( 0ii PYE ==

1FE

,

where denotes an expectation with respect to the distribution of the propensity

score in the treatment sample.

Implementing this involves the following steps. In the first step the propensity score is

estimated. In the second step we estimate the conditional expectation of the outcome

in the control areas given the propensity score using a kernel method. At this point we

are careful to ensure that all observations whose value of the propensity score is

outside the range of the propensity score in the treatment sample are deleted. This

imposes common support avoiding a major source of bias (see Heckman, Ichimura

and Todd, 1997). Finally the overall average is constructed using as weights the

distribution of the propensity score in the pilot areas.

The comparisons we make are the following

• EMA eligible urban young men in pilot areas with eligible urban young men in

control areas; EMA eligible urban young women in pilot areas with eligible urban

young women in control areas;

• EMA eligible rural young men in pilot areas with eligible rural young men in

control areas; and

• EMA eligible rural young women in pilot areas with eligible rural young women

in control areas.

One important issue in such policy experiments is the extent to which peer effects will

have an impact on ineligible individuals. This depends on a number of unknown

factors including of course who the peers are. If there is positive sorting by family

income this is not going to be much of an issue, particularly since the yaper will mean

that the incentives to participate for just eligible individuals and just ineligible ones

will be similar. However we can investigate this issue by considering the impact of

the programme on the entire population, rather than just on the ineligible ones; this is

an additional comparison that we carry out.

This separate matching is necessary if the differential impact of EMA on these groups

is to be estimated (to ensure the composition of the control is directly comparable).

All the data have been weighted3 by the appropriate pilot area population weights

derived from the FRS. This helps to overcome problems associated with non-random

panel attrition and sampling problems in our data. In addition when individuals are

weighed to the relevant sub-population, we are able to combine the effects in the rural

and urban areas. When the urban and rural data is not weighted in this way, our rural

results would be given too much weight since they are over-sampled in the EMA data.

The variables used to match the samples in the pilot and control are those given in

Table 3 (except of course the first three). These include both individual and household

level variables, characteristics of the ward in which the individual resides4,

characteristics of the nearest school as well as characteristics of the local education

authority.

The results

Impact of EMA on Year 12 Destinations

Table 2.1 shows estimates of the overall impact of EMA on young people’s initial

decisions to remain in full-time education, to move into employment or to become

NEET (Not in Education, Employment or Training). These results combine Cohort 1

and Cohort 2 and young men and young women together. The EMA effect has been

3 This is done using analytic weights. 4 The ward is a very small area of a local authority and will typically be quite homogeneous in terms of housing, levels of income and socio-economic background.

estimated separately for urban and rural areas and using both matched and unmatched

samples.5 The effects were estimated using pilot population weights.

5 Unmatched samples refer to just the mean of the relevant variable across the entire subgroup of interest. For example in unmatched samples the average participation in education among cohort 1 urban women in the pilot areas is compared with the mean in the control areas. It differs from the descriptive analysis later in this report since that compares only individuals from areas which have been deemed to have broadly similar characterstics.

Table 2.1 Impact of EMA on Year 12 Destination: all Eligible Young People

from Cohort 1 and Cohort 2, by Location – Pilot Weights

Per cent Unmatched sample Matched sample Pilot Control Pilot Control Increase Urban Results: FT Education 69.5 65.3 69.9 64.0 5.8 (S.E) (0.5) (0.8) (1.2) Work 14.4 16.2 14.3 17.8 -3.6 (S.E) (0.4) (0.6) (1.0) NEET 16.1 18.5 15.8 18.1 -2.3 (S.E) (0.4) (0.6) (1.0) Sample size 7266 3607 7111 7111 Population size 48,498 50,855 48,498 48,498 Rural Results: FT Education 83.1 74.5 83.7 77.6 6.1 (S.E) (0.8) (1.0) (3.4) Work 8.5 13.6 8.3 10.5 -2.3 (S.E) (0.6) (0.8) (2.4) NEET 8.4 11.9 8.0 11.8 -3.8 (S.E) (0.6) (0.7) (2.7) Sample size 2076 1936 1812 1812 Population size 5,804 5,628 5,804 5,804 Total: FT Education 71.0 66.3 71.3 65.5 5.9 (S.E) (0.5) (0.6) (1.1) Work/Training 13.8 15.9 13.7 17.1 -3.4 (S.E) (0.4) (0.5) (0.9) NEET 15.3 17.8 15.0 17.4 -2.4 (S.E) (0.4) (0.5) (0.9) Sample size 9342 5543 8923 8923 Population size 54,301 56,484 54,301 54,301

Note: Bootstrapped standard errors are reported based on 1,000 replications. EMA has had a positive and significant effect on post-compulsory education

participation among eligible young people. The overall estimate for both rural and

urban areas, combining Cohorts 1 and 2, is 5.9 percentage points.

Distinguishing between urban and rural areas is potentially important because the

local labour market opportunities are likely to differ. As we can see from the table this

difference may be reflected in the different schooling participation rates in urban and

rural areas, with the latter having much higher participation. It is of course a priori

unclear how this is likely to impact on the effect of the policy.

It turns out that the effect of the policy is broadly similar in both rural (6.1 percentage

points) and urban areas (5.8 percentage points). However, it is interesting to point out

that in earlier results, when we did not match on local area characteristics the findings

were different, leading to the impression of a much larger rural impact.

It is clear that this increase in post-compulsory education participation has drawn

young people from both employment and the inactivity group (NEET) in both urban

and rural areas.

Another important dimension of difference are men and women. Women have higher

post-16 school participation rates. As can be seen from Table 2.3 the impact for

women is lower. However, the difference is not significant (t-stat of 0.87)

Table 2.2 Impact of EMA on Year 12 destination: eligible urban young

males, by cohort, pilot weights.

Per cent Unmatched sample Matched sample Pilot Control Pilot Control Increase All urban men:

FT Education 67.3 62.4 67.4 60.6 6.9 (S.E) (0.8) (1.1) (1.7) Work/Training 17.2 19.1 17.1 21.9 -4.8 (S.E) (0.6) (0.9) (1.5) NEET 15.5 18.5 15.4 17.5 -2.1 (S.E) (0.6) (0.9) (1.3) Sample size 3613 1802 3548 3548 3548 All urban females: FT Education 71.8 68.2 72.3 67.5 4.8 (S.E) (0.7) (1.1) (1.7) Work/Training 11.6 13.2 11.6 13.9 -2.3 (S.E) (0.5) (0.8) (1.3) NEET 16.6 18.6 16.2 18.6 -2.4 (S.E) (0.6) (0.9) (1.5) Sample size 3653 1805 3563 3563

Note: Bootstrapped standard errors are reported based on 1000 replications.

A potentially important concern is the way that the programme impact evolves over

time. One concern may be that the programme may have had a low effect in the first

year of implementation because it was announced late in the academic year and pupils

may have committed to an alternative course of action. Moreover, the possibility of

continuing in Full time education is enhanced by good exam results at the end of

compulsory schooling. This presumably requires effort and preparation. On the other

hand there may be other differences between cohorts, including a change in the

economic environment, which could lead to different results. Finally it should be

pointed out that in some of the pilot areas the take up of the subsidy by those eligible

and participating in school was not 100%. Apart from the administrative issues in the

local authority which underlie some of the take up issue, this could also indicate that

pupils were not fully aware of the programme. However, there are differences

between the cohorts that would imply that the effect could go the other way [provide

table here]. First, since the eligibility threshold was fixed nominally (as well as the

subsidy) individuals who were fully eligible for the programme fell (from 3/5 to a

half). A larger proportion ended up being on the taper. This in itself could reduce the

response are the subsidy on average is now lower. Another key difference between the

cohorts is the sampling of individuals classed as vulnerable by the Department of

education (DfES). In the first cohort these were interviewed. However, halfway

through the process we were stopped from approaching these pupils. Cohort 2 does

not include such individuals.

Table ADD1 suggest that the effect of the EMA on education participation at year 12

is larger in the first cohort (7.1 percentage points) compared to the second cohort (4.6

percentage points), although this difference is not statistically significant. There seems

to be an increase in participation in education between the cohorts and a

corresponding fall in the proportion in work in the control areas.

Table ADD1 Impact of EMA on all Eligible Young People by cohort– pilot

weights

Per cent Pilot weights Pilot Control Increase Cohort 1: FT Education 71.2 64.1 7.1 (S.E) (1.6) Work 14.1 18.5 -4.5 (S.E) (1.4) NEET 14.7 17.4 -2.6 (S.E) (1.3) Sample size 4464 4464 Population size 27,000 27,000 Cohort 2: FT Education 71.5 66.9 4.6 (S.E) (1.6) Work 13.3 15.7 -2.4 (S.E) (1.3) NEET 15.2 17.4 -2.2 (S.E) (1.3) Sample size 4459 4459 Population size 27,297 27,297

Note: Bootstrapped standard errors are reported based on 1000 replications.

Eligibility for full or partial EMA awards

Only just over half of young people in Cohort 2 and three-fifths in Cohort 1 were

eligible for the maximum amount of weekly EMA available in their area. The

remainder of successful applicants for EMA would have received an amount below

the maximum to a minimum of £5 per week. This section distinguishes between

partial and full eligibility (rather than receipt) to see if the impact of EMA differs by

whether a person was fully or only partially eligible. For this analysis young people

were also matched within eligibility group, which results in the sample being slightly

smaller than for our earlier results. Of course it is important to note that when we

compare between eligibility groups more than the amount of the subsidy changes.

Crucially, overall family income is lower as well as other characteristics

Consequently, although the incentive may be higher for those on full eligibility other

household and local factors may limit the impact.

Among those who were estimated to be eligible for a full EMA award, EMA

increased full-time education participation in Year 12 by 7.5 percentage points for

young men and 4.7 percentage points for young women (Table 2.6). For those

estimated to be eligible for only a partial award, the corresponding figures are 3.2

percentage points and 5.5 percentage points. Thus from men the response of those

fully eligible is much larger than in the population who are facing the taper.

Nevertheless, the difference is not very significant.

Table 2.6 Impact of EMA on Year 12 destination: all eligible young people

from cohort 1 and cohort 2, by gender and amount of EMA, pilot weights.

Per cent

Males Females Pilot Control Increase Pilot Control Increase Fully eligible: FT Education 67.4 59.9 7.5 70.2 65.5 4.7 (S.E) (2.2) (2.1) Work/Training 15.5 20.9 -5.4 11.0 13.8 -2.8 (S.E) (1.9) (1.6) NEET 17.0 19.1 -2.1 18.8 20.7 -1.9 (S.E) (1.8) (1.7) Sample size 2437 2437 2490 2490 Population size 16,932 16,932 17,347 17,347 Taper: FT Education 71.1 67.9 3.2 80.0 74.5 5.5 (S.E) (3.6) (3.3) Work/Training 17.9 22.0 -4.1 11.2 12.0 -0.8 (S.E) (3.1) (2.3) NEET 11.0 10.1 0.9 8.8 13.5 -4.7 (S.E) (2.6) (2.7) Sample size 1706 1706 1648 1648 Population size 9,957 9,957 10,060 10,060 All Eligibles: FT Education 68.8 62.9 5.9 73.8 68.8 5.0 (S.E) (1.9) (1.8) Work/Training 16.4 21.3 -4.9 11.1 13.1 -2.0 (S.E) (1.6) (1.3) NEET 14.8 15.8 -1.0 15.1 18.0 -2.9 (S.E) (1.5) (1.5) Sample size 4143 4143 4138 4138 Population size 26,889 26,889 27,407 27,407 Note: matched samples only. Bootstrapped standard errors are reported based on 1000 replications.

Impact of EMA on eligible and ineligible young people

In all the results discussed so far, the focus has been solely on young people who were

eligible for EMA because of parental income. It is of policy interest, however, to

know what impact EMA has had on the entire population of young people, including

those who were ineligible on income grounds. This may turn out to be important if

the increased participation among the eligibles induces the ineligibles to do so as well.

This may be driven by peer effects as well as by a realisation that they need to keep

up if they are to be able to compete in the labour market.

The results of comparing the entire population of pilot and control areas is given in

Table 2.7. The EMA is estimated to have increased male participation rates by 4.3

percentage points 3.0 percentage points for young women and 3.7 overall. However,

comparing the magnitude of the results for the eligible population with the magnitude

of the effect on both the ineligible and eligible groups combined provides no evidence

of any positive spillover effects.

Table 2.7 Impact of EMA on Year 12 destination: all eligible and ineligible

young people from cohort 1 and cohort 2, by location and gender, pilot weights

and population weights

Per cent

Pilot weights Males Females Overall All: FT Education 4.3 3.0 3.7 (2.2) (2.2) (1.6) Work/Training -3.1 -1.3 -2.2 (2.0) (1.5) (1.3) NEET -1.2 -1.7 -1.5 (1.4) (1.8) (1.1) Sample size 5550 5508 11058 Population size 37,419 37,289 74,708 Note: Bootstrapped standard errors are reported based on 1000 replications.

Impact of EMA on Year 13 Destinations

So far the analysis has concentrated on the impact of the EMA on initial destinations

in Year 12, the first post-compulsory year. The subsidy is available to year 13 pupils

as well. A key policy question is the impact of the EMA on continuing school into

year 13, on overall educational achievement and, ultimately, on labour market

success. In this section, we focus on Cohort 1 and examine their destinations in Year

13, one year after the introduction of EMA.

The key problem for this analysis is attrition. We have already shown that differential

attrition rates by education attendance cannot be completely explained away by the

observables, although a large part can. However, it seems that attrition is similar in

nature in the pilot and control area an consequently unrelated to the existence of the

EMA. Consequently we can hope that the attrition bias balances out between

treatment and control and that no substantial bias remains.

To describe the sequence of events we now define four mutually exclusive outcomes:

• education in Year 12 and education in Year 13;

• education in Year 12 and other activity in Year 13;

• other activity in Year 12 and education in Year 13; and, finally,

• other activities in both year 12 and year 13.

Table 2.9 shows the impact of EMA based on the division of the population into the

four mutually exclusive groups described above. The important conclusion that omes

from this table is that whenever the EMA has been effective it has led to increase in

both year 12 and year 13 attendance and thus it is shown to have long term effects. It

also shows how EMA has affected education retention rates, defined as the proportion

of those in full-time education in Year 12 who were still in full-time education in

Year 13. EMA increased retention rates by 5.2 percentage points in urban areas (from

69.4 per cent to 74.6 per cent) and 10.8 percentage points in rural areas (from 76.2 per

cent to 87.0 per cent). This was despite the higher education participation rates

experienced in Year 12 as a result of the EMA.

Table 2.9 Impact of EMA on Year 12 and Year 13 destinations, Version II:

all eligible young people from cohort 1 who were re-interviewed in wave 2, by

location, pilot weights.

Per cent Pilot Control Increase Urban: Education Y12 → Education Y13 60.5 53.6 6.9 (S.E) (2.2) Education Y12 → Other activity Y13 14.1 15.8 -1.7 (S.E) (1.8) Other activity Y12 → Education Y13 2.2 2.4 -0.2 (S.E) (0.7) Other activity Y12 → Other activity Y13 23.2 28.2 -5.0 (S.E) (2.1) Retention Rate (for those in Edn in Y12) 74.6 69.4 5.2 (S.E) () Sample size 2497 2497 Rural: Education Y12 → Education Y13 75.9 61.6 14.3 (S.E) (6.6) Education Y12 → Other activity Y13 11.1 14.6 -3.5 (S.E) (4.2) Other activity Y12 → Education Y13 1.5 2.8 -1.3 (S.E) (2.3) Other activity Y12 → Other activity Y13 11.5 21.0 -9.5 (S.E) (5.8) Retention Rate (for those in Edn in Y12) 87.0 76.2 10.8 (S.E) () Sample size 708 708 All Areas: Education Y12 → Education Y13 62.1 54.5 7.6 (S.E) (2.2) Education Y12 → Other activity Y13 13.8 15.7 -1.9 (S.E) (1.7) Other activity Y12 → Education Y13 2.1 2.4 -0.3 (S.E) (0.7) Other activity Y12 → Other activity Y13 22.0 27.4 -5.4 (S.E) (1.9) Retention Rate (for those in Edn in Y12) 75.9 70.2 5.7 (S.E) () Sample size 3205 3205 Population size Note: Bootstrapped standard errors are reported based on 1000 replications.

Comparing the Different Variants of EMA

We now consider differences across variants. This is interesting because the variants

differ in the amount of the benefit offered and interestingly Variant 3 differs in that

the subsidy is paid to the mother. Moreover we have differences in the retention

bonus and the achievement bonus. These are important aspects of the programme that

are not reduced when the main subsidy is tapered off; hence they may impact on those

who have a reduced benefit due to family income. Conditional on the matching

assumption, by comparing across variants we should be able to estimate the impact of

the changing the structure of the programme.

To compare the variants we take as the base all young people in the sample in the

urban control areas. These are then matched to young people in the four different

urban Variants. This enables comparisons to be made across Variants on the basis of

the same set of control young people.

Table 2.14 reports the results (standard errors are yet to be computed) of this

matching exercise and suggests that:

Variant 1 increased education participation by 8.4 percentage points;

Variant 2 increased participation by 3.9 percentage points;

Variant 3 increased participation by 5.1 percentage points;

Variant 4 increased participation by 8.3 percentage points.6

However, the problem with these results is that the effects were calculated based on

different sub-samples of the controls. For example the individuals that were

successfully matched in variant 1 will have different characteristics to those

successfully matched in variant 2. As a result any difference in the EMA effect found

between variants could be due to the variant itself or the differences in individual

characteristics in the areas. For Variant 2 a large proportion of the control sample was

used whereas for Variant 4 a much smaller proportion was used.

6 The overall effect on education participation that we find by using this methodology is 6.2 percentage points. This is very slightly higher than the 5.9 percentage point EMA effect that we find in table 2.1. The reason for the difference is that table 2.14 is reporting the estimated effect of the EMA had it been introduced on the individuals living in the control areas whereas most of our analysis looks at the effect on the individuals living in the pilot areas. Since these individuals will, despite the matching process, have slightly different characteristics this will lead to slight differences in the effect found. The results are not statistically significantly different.

Table 2.14 Impact of the EMA on Year 12 destinations: all eligible young

people from cohort 1 and cohort 2, by variant, pilot weights.

Per cent Pilot Control Increase Variant 1: FT Education 74.6 66.2 8.4 Work/Training 14.2 16.1 -1.9 NEET 11.2 17.7 -6.5 Sample size 2875 2875 Variant 2: FT Education 69.8 65.9 3.9 Work/Training 17.7 15.9 1.8 NEET 12.5 18.2 -5.7 Sample size 3047 3047 Variant 3: FT Education 70.0 64.9 5.1 Work/Training 16.4 16.7 -0.3 NEET 13.7 18.4 -4.7 Sample size 2874 2874 Variant 4: FT Education 73.5 65.2 8.3 Work/Training 12.6 17.4 -4.8 NEET 13.9 17.4 -3.5 Sample size 2245 2245 All eligibles: FT Education 71.8 65.6 6.2 Work/Training 15.5 16.5 -1.0 NEET 12.8 18.0 -5.2 Sample size 11041 11041 Note: We have been unable to calculate standard errors due to problems of small samples in the bootstrapping process. This problem is overcome by presenting the results only for those control area young

people who have been found matches in all four of the Variants, thus respecting the

support conditions for the propensity score in all comparisons together. The result of

doing this is shown in Table 2.15 and suggests that the impact of:

Variant 1 on full-time education participation was 10.2 percentage points;

Variant 2 on full-time education was 4.7 percentage points;

Variant 3 on full-time education participation was 5.2 percentage points; and,

Variant 4 on full-time education was 7.1 percentage points.

The story emerging from both Tables 2.14 and 2.15 is relatively consistent. The

biggest impacts are found for Variant 1 (the core EMA Variant) and Variant 4 (where

larger retention bonuses were paid). However, Table 2.15 also shows that the sample

on which the common effect has been estimated was small and this could have

affected the precision of the results.

Conclusions

In this paper we use unique data collected for the purposes of evaluating the impact of

a schooling subsidy on school participation in England. Our results imply that

reducing schooling costs have significant and large effects on participation in school.

The elasticity of participation with respect to costs for 16 year olds is estimated

approximately at 1.5. This shows that the scope for affecting education decisions

using subsidies to education can be substantial, at least if we ignore General

Equilibrium effects.

More specifically, the results imply that the EMA has significantly raised post-16 full-

time education participation among eligibles in Year 12 by around 5.9 percentage

points and for the whole population (eligibles and ineligibles combined) by around

3.7 percentage points. The results also suggest that this participation gap for eligibles

widens between Year 12 and Year 13 and that this is largely driven by the significant

impact EMA has on retention in education in Year 13 for those eligibles who were in

education in Year 12. It also appears that the impact of EMA is only significant for

those receiving the full amount. When we compare the different variants there is

evidence that money paid to the child is more effective in both increasing education

participation, as well as retention, in education in Year 13 for those who were in full-

time education in Year 12. However, it is also clear that the most effective way to

increase retention is to increase bonuses. The retention outcomes for variant 4

individuals is significantly larger than for other groups. All of these measured effects,

however, vary by gender, rural/urban status and cohort. Despite this, our analysis

suggests that if the EMA were to be rolled out nationally, our estimated effects would

not alter significantly because of the different composition of pilot areas and England

as a whole.

Of course a number of important questions remain. First, we do not know whether

liquidity constraints are an important factor in driving the estimated factor. Second

and related to the previous question, we do not know what returns will be enjoyed by

those induced into staying on by the subsidy. Finally, we really have very little idea of

how these returns may change when the programme is rolled out nationally and the

supply of educated workers changes. This of course depends on many factors, not

least the production function. These are all important research and policy questions

that we will be pursuing.

Appendix Table A1. Indicators used in each deprivation score

Income Adults in Income Support households (DSS) for 1998 Children in Income Support households (DSS) for 1998 Adults in Income Based Job Seekers Allowance households (DSS) for 1998 Children in Income Based Job Seekers Allowance households (DSS) for 1998 Adults in Family Credit households (DSS) for 1999 Children in Family Credit households (DSS) for 1999 Adults in Disability Working Allowance households (DSS) for 1999 Children in Disability Working Allowance households (DSS) for 1999 Non-earning, non-IS pensioner and disabled Council Tax Benefit recipients (DSS) for 1998 apportioned to wards

Employment Unemployment claimant counts (JUVOS, ONS) average of May 1998, August 1998, November 1998 and February 1999 People out of work but in TEC delivered government supported training (DfEE) People aged 18–24 on New Deal options (ES) Incapacity Benefit recipients aged 16–59 (DSS) for 1998 Severe Disablement Allowance claimants aged 16–59 (DSS) for 1999

Health Deprivation and Disability

Comparative Mortality Ratios for men and women at ages under 65. District level figures for 1997 and 1998 applied to constituent wards (ONS) People receiving Attendance Allowance or Disability Living Allowance (DSS) in 1998 as a proportion of all people Proportion of people of working age (16–59) receiving Incapacity Benefit or Severe Disablement Allowance (DSS) for 1998 and 1999 respectively Age and sex standardized ratio of limiting long-term illness (1991 Census) Proportion of births of low birth weight (<2,500g) for 1993–97 (ONS)

Education, Skills and Training

Working age adults with no qualifications (3 years aggregated LFS data at district level, modelled to ward level) for 1995–1998 Children aged 16 and over who are not in full-time education (Child Benefit data – DSS) for 1999 Proportions of 17–19 year old population who have not successfully applied for HE (UCAS data) for 1997 and 1998 KS2 primary school performance data (DfEE, converted to ward level estimates) for 1998 Primary school children with English as an additional language (DfEE) for 1998 Absenteeism at primary level (all absences, not just unauthorised) (DfEE) for 1998

Housing Homeless households in temporary accommodation (Local Authority HIP Returns) for 1997–98 Household overcrowding (1991 Census) Poor private sector housing (modelled from 1996 English House Condition Survey and RESIDATA)

Geographical Access to Services

Access to a post office (General Post Office Counters) for April 1998 Access to food shops (Data Consultancy) 1998 Access to a GP (NHS, BMA, Scottish Health Service) for October 1997 Access to a primary school (DfEE) for 1999

Child poverty Percentage of children that live in families that claim means tested benefits (Income Support, Job Seekers Allowance (Income Based), Family Credit and Disability Working Allowance).

Source: Department of the Environment, Transport and the Regions (2001), Regeneration Research Summary: Indices of Deprivation 2000, (Number 31, 2000) (www.regeneration.dtlr.gov.uk/rs/03100/index.htm).

References Cameron, Stephen V & Heckman, James J, 1998. "Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts of American Males Cameron and Taber (2000) Borrowing Constraints and the Returns to Schooling, NBER Working Paper, 7761 Heckman, Ichimura and Todd (1997) Matching as an Econometric Evaluation Estimator, Review of Economic Studies Heckman, JJ, Lance Lochner, Christopher Taber (1998) Explaining Rising Wage Inequality: Explorations with a Dynamic General Equilibrium Model of Labor Earnings with Heterogeneous Agents , NBER working Paper 6384

Stacy Berg Dale and Alan B. Krueger (1999) Estimating the Payooff to Attending a More Selective College: An Application of Selection on Observables and Unobservables NBER working Paper 7322 Rosenbaum P.R., Rubin D.B. (1983): The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika 70, 41-55