EDUCATION 2: PROMISING AREAS AND INTERESTING PUZZLESMiles Tidmarsh

Behavioural Reading GroupMIAESR9 Dec 15

Introduction• What will this presentation do?

• Highlight some promising and interesting interventions

• Consider puzzles, (in)consistencies and future research options

• And… • consider applications in Australian context

Introduction• This presentation will cover

• 1. Parental Interventions• Engagement seminars and regular messages

• 2. Identity• Tertiary salience and performance

• 3. Major areas• Removing small barriers• School counselling• Informing students of returns to education

• 4. General points• Comprehensive approach • Relationship of interventions to psychological theory and economic models• General remarks

Introduction• Recap

• Educational decisions systematically sub-optimal• Under-investment of time/effort• Frequent non-attendance• Dropping out too soon• Not taking advantage of financial aid• Insufficient research on future outcomes

• Major welfare losses

• Interventions attempting to improve outcomes

Frameworks

• Oreopoulos, Lavecchia and Liu 2014• Inconsistent time preferences• Excessive routine reliance• Negative Identities• Inaction when facing many options or little information

• Congdon 2011• Imperfect optimization• Bounded self-control• Non-standard preferences

What counts as behavioural?• Some behavioural policies and concepts are already

considered in standard economic policy• E.g. Social Security, nominal wage rigidity, mandatory schooling

• Many could be thought of as psychological or educational changes• E.g. Goal setting, structured learning

• Who cares? (Chetty 2015)• If it works, it works. • Discipline boundaries and definitions can be useful, but also

constraining• Does it improve predictions and outcomes?

• Not just Nudges

1. Parents• Parental inputs are known to be a major factor in the

educational production function

• Increasing academic attention being paid to parents

• Relatively difficult to measure and influence with conventional means

• SES skill gap driver of intergenerational inequality• (this is a common theme: poorer students are harmed more by the

biases and benefit more from the interventions)

1a. Parents• Parental engagement seminars

• Teaching parents how to more effectively engage with their children

• Theoretical background: • Parents have motivation but often not knowledge to engage effectively• Inconsistent investments • Parents face stresses, competing demands and ‘now’ vs. ‘later’, which

undermine engagement in small, regular, non-urgent activities• Many could benefit

• So… bring the information to them

1a. Parents• Parental engagement seminars

• Avvisati et al 2014 – RCT in Paris schools• 7th grade students from disadvantaged district• Significant and persistent boost to parental engagement and non-

cognitive skills/behaviour. Test scores seem unaffected. • 100 times cheaper than smaller classes (Dee and West 2011)

• Banerji, Berry and Shotland 2015 – similar intervention in India• Mothers in 480 villages of age 5-8 children• Similar findings with very large sample and radically different context. • Suggests good external validity of Avvisati

1b. Parents• Regularly messaging parents on student’s schooling

context• Informing of absences, missed homework, upcoming

tests, class behaviour etc.

• Theoretical background• Teachers can choose to do this already, but rarely do• Imperfect optimisation of non-standard preferences

1b. Parents• Regular messaging to parents• ~100% volunteered, very popular• Cheap, ~$156/child/semester (Bergman 2014)• Results:

• +0.24 GPA, • <+25pp homework completion (Bergman 2014), • +30% class participation (Kraft and Dougherty 2013), • 8.8pp (69%) reduction in not earning course credit and • 1.8-3.3 SD improvement in range of outcomes (Kraft and Rodgers

2015)• Statistical issues with all three

• Observer effect?

2. Identity• Identities known to be major influence

• Non-constructive identities and low self-efficacy • linked to underinvestment in education

• Salience

• Effects on • Aspirations• Performance

2a. Identity salience and Aspirations• High-achieving, low-SES (HALS) students

• systematically lowered expectations and ambitions

• Far less likely to seriously apply to tertiary and to selective tertiary institutions.

• resulting in worse lifetime outcomes • (Dvorak 2010, Polidano et al 2012)

2a. Identity salience and Aspirations• Avery and Hoxby 2013

• Vast majority of HALS are “income typical”• do not apply to postsecondary institutions that fit their ability.• degree of disadvantage, graduation rates and parental education do not

explain the “achievement typical” minority…• Geography matters

• large urban areas• schools with concentration of other high achievers • have university graduates within a close radius of home.• Critical mass salience aspirations enrolment

• Teachers, neighbours, peers.

• Qualitative support • (Levine and Nidiffer 1996)

2b. Identity and Performance• Tall Poppy Effect

• Bursztyn and Jensen 2014a

• Sign-ups for SAT prep course, in low-SES school• Lower when decision was disclosed to classmates• Except in honours classes, where it was higher

• Supports importance placed on ‘fitting in’ with peers

2b. Identity and Performance• Disclosure of Top performers in a class

• Bursztyn and Jensen 2014b

• Results• Performance of all other students declined• Internalised low-performance• Fits existing research• Policy Implications

• Drawing attention to high-achievers may backfire

2b. Identity and Performance• Teacher-student identity (Gelbach et al 2015)

• Primary students• Emphasized similarities• Improved outcomes• Closed 60% of SES gap• Impractical, but illustrative

3. Major Categories• Other common types• Small barriers

• Fees• Processes

• Counselling• They work• Often substituting for parental knowledge

• Informing of returns to education• It doesn’t work

3a. Small barriers• Fee waivers

• Almost always effective• US: one extra free application 71% switched (Pallais 2013)• 20,000:1 cost-benefit

• More difficult in Australia

3a. Small barriers• Administrative process

• Inaction when faced with complexity• FAFSA assistance: +8pp college enrolment at $88 per participant

(Bettinger et al 2012)• In-class college application assistance: +15pp enrolments (Avery

2013)• Show, don’t tell• Scale invariant?

• Australia: bursaries, scholarships etc.

• Lack of skilled parental assistance (Arnold et al 2013, Perna and Titus 2005)• Especially for low-SES• Can be substituted for by (counsellor) assistance…

3b. School counsellors• General findings• It works

• Established• About as (cost-)effective as Financial Aid• But better for low-SES• More Australian applicability• Not cheap

• Other outcomes• Course choice• Staying in school• Financial aid applications

3b. School counsellors• Specific findings• Adding one general counsellor per school (Hurwitz and

Howell 2013, Avery 2010). • increases mid-ranking tertiary enrolment by 9-18pp

• Berman, Ortiz and Bos 2008• Discussing: options, cost, application procedure• 1/3 reduction in non-applications• 5pp from community to state colleges

• Bottom Line on low-SES (Castleman and Goodman 2014)• +41pp shift to recommended colleges • ESL and new college applicants saw largest effects• Supports administrative barrier hypothesis.

3b. School counsellors• Testing hypotheses for under-participation• Carrell and Sacerdote 2013 – assistance interventions

• Rule out procrastination, fear of process, lack of information and rational aversion to high drop-out risk.

• Are these four decisive? • 1/3 of gender gap in applications can be explained as rational

response to different expected income with only high-school diploma.

• Treatment effect strongly correlated with lacking parental or teacher assistance.

• Assistance is the main binding constraint

3b. School counsellors• Course choice quality (Borghans, Golsteyn and

Stenberg 2013)• Regret of course choice 18 months after graduation• ’Who’: Clarifying students’ own motivations and competencies• ’Why’: Labour market outcomes• ‘How’: courses, apprenticeships

• From 22% to 20% for each• Similar to studies in adults (Hainmuller et al 2009)

• Peer counselling is much less effective • (Berman et al 2012, Cunha and Miller 2009)

3c. Informing of returns to education• It doesn’t work

• Students systematically underestimate apparent benefits and overestimate costs

• Changes knowledge, but not intentions or outcomes• (Kerr, Pekkarinen, Sarvimaki and Uushitalo 2014, Dinkelman and

Martinez 20113, Booij, Leuven and Oosterbeek 2011, McGuigan, McNally and Wyness 2012).

• But… • Financial aid calculator (in-class) increased intentions (Oreopoulos and

Dunn 2013). • Application and enrolment behaviour unknown

• Personalised fee information helped HALS, but was matched with a fee waiver (Hoxby and Turner 2014).

• Neither conclusive

4. General Points• Comprehensive approach

• Theory and Evidence

• Relationship to broader theories• Psychological theory: success and failure• Integration into Economic models

• Extrapolation – challenges

• Costs and benefits

4a. Comprehensive approach• Theory• ‘Kitchen sink’ approach

• Many interventions at once

• Theory of pulling students beyond threshold to new equilibrium

• Higher total effect but lower cost-benefit ratio. (Madrian 2014)

• Makes extrapolation harder (Madrian 2014)

4a. Comprehensive approach• Evidence

• Pathways to Education: mentoring, tutoring, group activities, counselling, college transition assistance and incentives. • (Low-SES) High school completion from 38 to 53%, postsecondary

enrolment from 34 to 53% . (Oreopoulos, Brown and Lavecchia 2014)• A Worthwhile Canadian Initiative

• Broad evidence:• Using two seems more effective than either alone (Angrist, Oreopoulos

and Lang 2006)• But apparent lower marginal benefit, indicating zero to negative net

complementarities (List, Livingston and Neckermann 2012)• Still limited evidence either way

4b. Relationship to Psychological theory

• Useful…

• Psych theories inspire behavioural interventions

• Are these theories (reliably) predictive?

• Successes: • Routine/default – ‘how’• Identities – ‘Who’• SES gaps

4b. Relationship to Psychological theory

• …Sometimes• Failures

• ‘Why’ • ‘When’ – incentives ‘should’ work but don’t• A bias can be true and significant without being tractable• Ambiguity of predictions

• Magnitudes of effects of competing processes

• Psychology is suggestive, but fallible • Therefore, experimentation is important• Theory allows for generalisation

4b. Economic models of behavioural phenomena

• Recent work on integration• Little influence on interventions (so far?)• Not predictive

• Education-relevant Examples:• Default bias– (Koszegi and Rabin 2006)• Choice paralysis – (Lleras, Masatlioglu, Nakajima and Ozbay 2010)• Complexity paralysis (Dynarski and Scott-Clayton 2006)• Limited attention – (Manzini and Mariotti 2012) (Masatlioglu, Nakajima and

Ozbay 2012)• Information and processing limitations – (Masatlioglu and Nakajima 2013)

(Bergman 2014)• Limited willpower – (Masatlioglu, Nakajima and Ozdenoren 2013)• Status Quo Bias – (Masatiloglu and Ok 2013)• Framing – (Salant and Rubinstein 2008)

4c. Extrapolation from theory and evidence

• Sometimes high context dependence (e.g. fee waivers)• So… test in Australia• Some are reliable (e.g. Parental seminars)

• Intertwined processes• Multiple biases in one outcome• Multiple biases in one intervention• Multiple outcomes from one bias• What is driving X effect?• E.g. tertiary admission – proactive, identity, previous and current

support• Long-term follow-up essential to high confidence

• E.g. Perry Preschool

4d. Costs and Benefits• Baseline of conventional policies

• High uncertainty in some estimates• Large variance in between interventions• Assumptions on intermediate targets• Holistic impacts

• Behavioural interventions don’t solve problems• Alleviate• Supplement conventional policies• Best are low-hanging fruit

Conclusion• 1a. Parental engagement works for non-cognitive skills• 1b. Parental text messaging probably works for cognitive and

non-cognitive skills• 2a. Identity and geography are a major driver of tertiary SES gap• 2b. Promoting high achievers may backfire• 3a,b Small barriers are a binding constraint, but can be

overcome with personal assistance• 3c. ‘why’ interventions don’t work• 4a. Comprehensive approaches work but are difficult to extrapolate

from• 4b. Draw more from psychology than conventional Econ models

• Complex, but useful opportunities