what can we learn from education impact...

What can we learn from education impact evaluations?

Barbara Bruns Center for Global Development

SIEF Workshop

Dodoma, Tanzania October 4, 2017

Overview

1. Why evaluate? 3 reasons

2. What are we learning in education?

3. A birds’ eye view of our evidence base (from

Ganimian and Murnane) 3. Top ten policies based on evidence 4. Conclusions

Annex: Complete review of Ganimian/Murnane meta-study conclusions

Why evaluate impact?

To influence spending decisions

To influence spending decisions: Mozambique’s Escolinhas

7,38

3,15

26,21

17,82

5,37 5,55

2,86

14,01

11,48

3,60

Physical Health Communication CognitiveDevelopment

Social Competence Emotional Maturity

Preschool Control

• Low-cost pre-school program developed by Save the Children (30 communities); 30% of cost of traditional pre-schools

• Volunteers from the community trained to work

with children aged 3-5, given small stipend, learning materials and weekly feedback

An impact evaluation showed significant effects on children’s development → Govt. decided to scale up to

600 rural communities


To influence program design


• Forces us to articulate our “theory of change” • Forces us to think through implementation steps –

Who will act? What resources will they need? How long will each step really take?

• Stimulates the collection of key monitoring data –eg. classroom observation

India (indiv.)

0.3-0.45

Kenya

Teacher bonus pay, ca. 2009

Bonus Pay Programs by Impact Size (learning outcomes)

0 0,1 0,2 0,3 0,4 0,5

Effect Size

Mexico ALI

(indiv.)

India (group)

5 yr. impact Israel (group)

Israel (indiv.)

India (attend.)

0.18

Chile (group)

0.14-0.26

Source: Making Schools Work, 2011

- Program design matters a lot

- Some programs with strong learning results did not have clear explanations of “how”

Theory of action: Results caused by a change in teacher policy must change teacher effort and/or effectiveness in the classroom

ACTIVITIES

Curriculum design

Teacher

training

School

supervision

INPUTS

Financing

Salaries

School

construction

Books

ICT/Materials

OUTPUT

Teacher

practice in

the

classroom

Inputs

Activities

Results

OUTCOMES

Student

learning

Graduation

Rates

Employment

Wage gains

Use standardized classroom observation to measure

Pernambuco teacher bonus: 2009-2011

ACTIVITIES

School targets

for improvement

in test scores

and pass rates

School bonus for

reaching targets (one month salary

for all teachers in

school)

INPUTS

Financing (10%

increase in salary

bill)

OUTPUT

Teachers

absent less

and spend

more time on

instruction

Inputs

Activities

Results

OUTCOMES

Pass rates

improved more

than learning

Lowest-

performing

students had

biggest gains

Stallings method classroom observations in 20% of

schools

More time on instruction correlated with bonus pay

awards

** Statistically significant at 5%

*** at 1%

Pass rates drove bonus awards; learning gains modest, but largest for low-performing students

Pernambuco IDEB gains (secondary education)

Santa Catarina 3.5 Pernambuco 3.9

Minas Gerais 3.4 São Paulo 3.9

Rio Grande do Sul 3.4 Goiás 3.8

São Paulo 3.3 Espírito Santo 3.7

Paraná 3.3 Paraná 3.6

Roraima 3.2 Rio de Janeiro 3.6

Espírito Santo 3.1 Acre 3.5

Acre 3 Amazonas 3.5

Distrito Federal 3 Distrito Federal 3.5

Ceará 3 Mato Grosso do Sul 3.5

Rondônia 3 Minas Gerais 3.5

Goiás 2.9 Ceará 3.4

Tocantins 2.9 Roraima 3.4

Rio de Janeiro 2.8 Santa Catarina 3.4

Mato Grosso do Sul 2.8 Rio Grande do Sul 3.3

Alagoas 2.8 Rondônia 3.3

Sergipe 2.8 Tocantins 3.3

Pernambuco 2.7 Piauí 3.2

Amapá 2.7 Amapá 3.1

Bahia 2.7 Maranhão 3.1

Paraíba 2.6 Paraíba 3.1

Mato Grosso 2.6 Mato Grosso 3

Pará 2.6 Pará 3

Rio Grande do Norte 2.6 Bahia 2.9

Maranhão 2.4 Alagoas 2.8

Amazonas 2.3 Rio Grande do Norte 2.8

Piauí 2.3 Sergipe 2.6

2005 2015


To influence spending decisions


To influence ideas

To Influence Ideas: Brazil and Mexico launched CCTs in 1997

• Mexico implemented Progresa in a randomized rollout with rigorous evaluation • Brazil did no evaluation of Bolsa Escola

Conditional cash transfer programs, 2011

Mexico’s results have influenced the world

What are we learning in education?

• Big increase in rigorous evaluation over past 15 years

• Recent meta-studies

- Glewwe, 2014 - Evans and Popova, 2015 - Ganimian and Murnane, 2016

• 223 studies in 58 countries • Most comprehensive and nuanced framework • “Theory of action” approach

- Snilsveit et al, 2016 – evidence from low-income countries - JPAL (forthcoming) – compares cost-effectiveness

What can improve education results?

Children attend school and learn

Source: Ganimian & Murnane (2016)

Key goal:



Build demand for education

Improve supply of education






Reduce the costs of attending school

1. Reduce the direct costs of schooling

2. Reduce the costs of complements

3. Improve school amenities

Prepare children for school

1. Provide children with vital medications

2. Improve nutrition

3. Improve parenting practices

Make schooling pay

1. Compensate families for opportunity costs

2. Inform families of the benefits of schooling

3. Offer incentives to students

Improve access to better schools

1. Provide information about school quality

2. Expand schooling options

Increase the quantity and quality of resources

1. Increase resources at school

2. Increase resources at home

3. Expand instructional time

Address students’ learning needs

1. Help teachers personalize instruction

2. Provide additional help to students

3. Lett students learn at their own pace

Increse teacher and/or principal effort

1. Increase the role of parents in school management

2. Offer incentives to teachers and principals

3. Hire teachers on fixed-term contracts

Build teachers’ capacity

1. Train teachers on specific skills

2. Prepare lessons for teachers

3. Let students teach each other


Evidence: Reducing the costs of attending school

Reducing the direct costs of schooling

1


1 Lowering or eliminating school fees (Barrera-Osorio et al 2007; Gajigo 2014; Borkum 2012; Liu et al 2013).

2 Building new schools closer to students. (Duflo 2001; Mocan & Cannonier 2012; Burde & Linden 2013; Liu et al 2010; Berlinski et al 2009; Martinez et al 2013; Bouguen et al 2014).

3 Providing transportation to school. (Muralidharan & Prakash 2013).

Consistent evidence that these increase enrollments.

Reducing the complementary costs of attending school


1

Reducing the costs of complements to schooling

2

Reducing the costs of complements to schooling

1 Free school uniforms and eyeglasses (Evans et al 2009; Ma et al 2014; Glewwe et al 2014; Kremer et al 2003).

Evidence that these raise the learning achievement of students who receive them













Make schooling pay
























And even with all this… Geographical bias: 40% of evaluations from only 4 countries (China, India, Kenya, Mexico)

Topical bias: many cells have only 1 -2 country studies: 20% of total sample is CCT studies

Temporal bias: almost no studies measure impacts more than one year out

Non-comparable measures of learning: many evaluations use specialy designed tests which cannot be directly compared to tests used in other evaluations

Cost-effectiveness estimates limited: many evaluations do not present complete cost estimates and even good cost-estimates are limited by fact that the numerator (amount of learning increase) per dollar spent are not comparable

And even with all this…(most serious of all)

System reform “black hole”: hugely important reforms that are implemented system-wide are rarely evaluated

Curriculum reform

Mutli-shift schooling

Decentralization

Teacher career reforms: changes in the pay scale; standards for hiring; promotion criteria; performance evaluation; possibility of dismissal

What are the policies that can make these schools….

Become more like these schools?

Pre-primary class, Schule Ya Msingi Na Awali

Top 10 policies based on

evidence

Weighted by:

- Strength of evidence

- Cost-effectiveness/fiscal impact

- Generalizability

- Implementability

- Politics

“Don’ts” are as important as “Do’s”

Don’t raise teacher salaries without raising standards

Indonesia doubled teacher salaries in 2008

Only change in teacher performance expectations was a certification that all teachers could meet

Rigorous evaluation in 2014 (De Rhee et al.) found

No increase in student learning

No change in observable teacher effort

Huge fiscal deficit

Politically impossible to reverse

In Latin America, higher pay for teachers who pass tough competency exams – evidence this works (Mexico, Estrada 2013; Peru, 2014; Chile 2016)

Don’t expect cash transfers to raise learning

Cash transfers are effective in raising enrollments and student attendance in many countries

Size of these effects depend on whether:

Conditionality is monitored

Targeting is effective

Political winner, but impossible to reverse and costly over long-term

Best strategy to target carefully and monitor conditions

Complementary supply-side actions are needed for learning impact

Schultz 2004; Maluccio & Flores 2005; Galiani & McEwan 2013; Behrman et al 2009; Barrera-Osorio et al 2011; (Kremer et al 2009; Barrera-Osorio & Filmer 2015

Don’t expect computers to raise learning in the classroom

Expensive one-laptop-per child programs in Peru, Honduras and Colombia failed to increase student learning (Cristia et al. 2012; Barrera-Osorio and Linden, 2009)

Teachers failed to integrate them into classroom practice (classroom observations in Peru showed ITC use only 1% of class time)

BUT computer-assisted learning for targeted students in after school programs can be a powerful tool for remedial education

Banerjee et al 2007; Lai et al 2013; Mo et al 2014; Linden 2008; Lai et al 2015; Lai et al 2012; Yang et al 2013; Huang et al 2014.

Don’t expect parent involvement to transform school mgt.

Giving parents voice on school councils has generally had little impact on school results

Providing school grants that communities can manage has had some positive impacts

Niger, Mexico, El Salvador, Nicaragua – higher parent involvement in schools and satisfaction

The Gambia: lower student and teacher absence

But no strong impacts on learning anywhere

Slight exceptions: Senegal (small increase in girls’ test scores, but not boys) and Kenya (increased student test scores when combined with the hiring of an extra teacher)

Beasley & Huillery 2015; Gertler et al 2012; Blimpo, Evans and Lehire 2015; Carneiro et al 2015; Pradhan et al 2014; Lassibille et al 2010; Glewwe & Maïga 2011

Do build more schools

Decreasing the distance to school has big effects on student enrollment (.38 SD), attainment, and long-term wages

Also raises learning (.19 SD)

Improving school facilities (hygiene) also raises enrollment and attendance substantially

Big payoffs to:

Cost-effective construction

Careful mapping and planning

Duflo, 2001 evaluated impacts of Indonesia’s school expansion from 1973-78 (61,000 schools) using creative RD design

What about reducing class size?

Need evidence!

Do leverage the private sector Liberia’s Partnership Schools improved school management and

raised student learning 60% in one year

Teachers increased time on teaching from 32% to 48% (Stallings observations) and worked longer school day

Student attendance improved

BUT year 1 costs are unsustainable, so important to analyze further

Targeted vouchers for private schooling have:

Raised learning and attainment (Colombia)

Been cost-effective (India)

Romero, Sandefur and Sandholtz, 2017; Angrist et al 2002; Bettinger et al 2010; Angrist et al 2006; Muralidharan & Sundararaman 2015

Do invest in high quality student assessment

Learning assessment is a critical platform; makes it possible to:

Track system progress

Evaluate different policies

Reward schools for results

Compare private school quality and cost-effectiveness and make smart use of targeted vouchers

Inform parents about the benefits of schooling

Inform parents’ school choices

Empower parents to demand quality

A national student assessment system unlocks the door to

evidence-based policy

Do promote results focus and teacher interaction

School-based bonus pay can stimulate teachers to work together and raise student learning

Muralidharan & Sundararaman 2011; Muralidharan 2012; Lavy 2009, 2013; Contreras and Rau, 2012; Behrman et al, 2015; Ferraz and Bruns, 2014

Giving teachers feedback and coaching can also promote interaction and raise learning

Bruns, Costa and Cunha, 2017

Key conditions for impact:

Good quality national assessment that generates learning data for all schools (census-based)

Capacity to set school-level targets each year (or every 2 years)

Administrative integrity

Do equip teachers for multi-grade classes and big classes

Effective strategies for multi-grade classrooms include:

Self-paced learning materials

Students tutoring each other (cross-peer learning)

These can be helpful in large classrooms too

Organize classrooms so that all students get some individual attention and small group experience

Colombia’s Escuela Nueva an excellent model; Singapore’s student support policy Psacharopoulos, Rojas and Velez, 1992; Wachanga & Mwangi 2004; Beuermann et al 2013

Do prioritize early literacy and numeracy skills

“Structured pedagogy” and “Teaching to the right level” improve learning

Focus the curriculum for the first 3 grades on getting all children to read and comprehend

Assess students’ progress and group them by learning level, not age (EGRA, EGMA)

Provide remedial instruction (including computer-based programs and after-school teacher aides) to children who need it

Give teachers ongoing feedback and support

Liberia EGRA-plus Program – children scored .72 SD higher on reading

Mali Read-Learn-Read Program - +.24 SD

South Africa English and Operacy Program - + .40 SD

Ceara Brazil – “On Time Literacy” Program - +.17

Snilsveit, 2015; Piper and Korda, 2010; Costa and Carnoy, 2014; Banerjee et al 2007; Lavy & Schlosser 2005; Duc & Baulch 2012)

Conclusions

Huge increase in the evidence base in education is spreading promising ideas across the world

Replication is strengthening the evidence – but most consistent result is that country context, design details and implementation fidelity matter

Evaluation is expensive: should be focused on innovative programs and expensive policies

Where you can make a difference:

Thinking through the results chain/theory of action for new programs

Evaluating programs as they go to scale

Tracking learning impacts over the medium-term (4-5 years)

Finding creative ways to build evidence on key system-wide reforms

Esp, strategies for driving down class size

Thank you! [email protected]

Improving education in developing countries: Lessons from rigorous impact evaluations

Alejandro J. Ganimian Education Post-Doctoral Fellow, J-PAL

Outline 1. Demand: How to increase school participation 2. Supply: How to improve the quality of instruction 3. Action: How to use research to inform policy

Framework: How should we draw lessons?












Make schooling pay
























Preparing children for school

Providing children with vital medications

1


1 Deworming drugs—either alone or in combinations with micronutrients. (Miguel & Kremer 2004; Grigorenko et al 2006; Bobonis et al 2006; Jinabhai et al 2001; Maluccio et al 2009; Solon et al 2013).

Reduced absenteeism, but no consistent effects on learning.

2 Malaria drugs – (Brooker & Halliday 2015; Fernando et al 2006; Clarke et al 2008; Jukes et al 2006; Cutler et al 2007; Sylvia et al 2013).

No impact on student learning in short-or long-term.

Evidence: Preparing children for school

Improving nutrition

2


1

Improving nutrition

1 Iron supplements have generally improved children’s cognitive skills. (Baumgartner et al 2012; Manger et al 2008; Luo et al 2012; Rico et al 2006; Kleiman-Weiner et al 2013).

2 Other nutrients, such as fatty acids, have increased school participation, but not student achievement. (Hamazaki et al 2008; Mahawithanage et al. 2007; Osendarp et al 2007).

3 Free school meals – inconsistent impacts depending on the type of food and mode of delivery, among other factors. (Whaley et al 2003; Adelman et al 2008; McEwan 2013).

Preparing children for school

Improving parental practices

3

Improving nutrition

2


1

Improving parenting

1 Home visits to teach mothers how to stimulate young children’s cognitive and psycho-social skills. (Attanasio et al 2014; Walker et al 2005; Tieffenberg et al 2000; Rocha & Soares 2010; Kagitcibasi et al 2001; Rosero & Oosterbeek 2011).

Positive impacts on school attendance, cognitive skills, and adult wages.

Evidence: Making schooling pay

Compensating parents for foregone opportunities

1

Compensating parents for opportunity costs

1 Conditional cash transfers . (Schultz 2004; Maluccio & Flores 2005; Galiani & McEwan 2013; Behrman et al 2009; Barrera-Osorio et al 2011). (Kremer et al 2009; Barrera-Osorio & Filmer 2015).

Consistent positive impacts on enrollment and

attendance but not learning. Size of effects depends on design.



1

Informing families of the benefits of schooling

2

Informing families of the benefits of schooling

1

2

. (Jensen 2010, 2012).

2 But mode of delivery of the information makes a difference. (Nguyen 2009).

Give students and parents information on the

returns to schooling (e.g., the wages of

individuals with different levels of schooling)

Positive impacts on attainment and learning.



1

Informing families of the long-term benefits of schooling

2

Offering incentives

to students 3

Offering incentives to students

1 Rewarding students for test scores has not produced consistently positive results and in some cases has stimulated cheating. (Angrist & Lavy 2009). Sharma 2010; Berry 2014; Li et al 2014). Behrman et al 2015).

Improving access to better schools

Providing information about school quality

1


1 Providing parents with information on schools’ performance may help them make better decisions about where to send their children to school and to demand changes in their schools.

2 The evidence is modest, but encouraging. The increased learning has mostly been in private, not public schools. This may be because they have more flexibility to respond to information. (Andrabi et al 2009; Mizala & Urquiola 2013; Camargo et al 2011).

Evidence: Improving access to better schools


1

Expanding schooling

options 2

Evidence: Expanding schooling options

1 “Universal” voucher programs are difficult to evaluate, but appear to have little effect on overall system learning results and attainment, but do increase student sorting. (Hsieh & Urquiola 2006; Contreras 2001).

2 “Targeted” voucher programs giving low-income parents money to send children to private schools have improved enrollment, attainment, and achievement. (Angrist et al 2002; Bettinger et al 2010; Angrist et al 2006).

3 Voucher winners in private schools performed no better than students who stayed in public schools, but private schools operated at a third of the cost of public schools. (Muralidharan & Sundararaman 2015).

Supply: Increasing the quantity and quality of school resources

Increasing resources at school

1


1 Providing schools with inputs such as textbooks, libraries, and flipcharts has typically not improved student achievement.

2 Sometimes, they are not even used. (Glewwe et al 2009; Sabarwal et al 2013).

3 Other times, they do not improve children’s daily experiences—typically, because teachers lack the knowledge to use these resources to improve instruction. (Glewwe et al 2004; Borkum et al 2013; Barrera-Osorio & Linden 2009; Cristia et al 2012).

4 Yet other times, school resources “crowd out” resources at home. (Das et al 2013; Björkman 2007).

Evidence: Increasing the quantity and quality of resources

Increasing resources at home

2


1

Evidence: Increasing resources at home

1 Many children in developing countries lack basic educational resources at home. To stimulate learning, several initiatives have provided children with resources—typically, computers.

2 These initiatives have not improved student achievement. In fact, students report using computers for playing games. (Malamud & Pop-Eleches 2011).

3 In some contexts, students get more apt at handing the computer that they are given, but not at computer skills. (Beuermann et al 2013).

Evidence: Increasing the quantity and quality of resources

Expanding instructional

time

3

Increasing resources at home

2


1

Evidence: Expanding instructional time

1 There is only one study of the effects of simply increasing instructional time. This study finds an impact on student learning. (Orkin 2013).

2 Other evaluations of initiatives that lengthened the school day have also captured the effects of simultaneous reforms (e.g., more resources, infrastructure upgrades, and increased per-pupil spending). (Bellei 2009).

Evidence: Addressing students’ learning needs

Helping teachers

personalize instruction

1

Evidence: Helping teachers personalize instruction

1 Class size reductions, by themselves, have not improved student learning. Yet, they have improved test scores when students were “tracked” into different classrooms based on their performance on an initial assessment. (Duflo et al 2007, 2011).

2 Tracking students across schools has improved achievement, but also elicited unintended consequences (e.g., teacher sorting). (Pop-Eleches & Urquiola 2013; de Hoop 2010).

3 Providing teachers with feedback on students’ learning has improved results only when accompanied by actions to build teacher capacity or make their jobs manageable. (Muralidharan & Sundararaman 2010; Duflo et al 2015; Piper & Korda 2011).

Evidence: Providing additional help to students

Helping teachers


1

2 Providing additional help to students

Evidence: Providing additional help to students

1 Low-achieving students often get very little from regular instruction. Pulling them out of the classroom to provide them with remedial instruction has improved their performance. (Banerjee et al 2007; Lavy & Schlosser 2005; Duc & Baulch 2012).

2 The merits of non-academic remedial support, however, are more uncertain. (Cabezas et al 2011; Huan et al 2014).

Evidence: Addressing students’ learning needs

Helping teachers


1

2 Providing additional help to students

Letting students learn at

their own pace

3

Evidence: Letting students learn at their own pace

1 Many initiatives have experimented with computer-assisted learning (i.e., questions that adjust to students’ skill level). These programs have generally improved student achievement. (Banerjee et al 2007; Lai et al 2013; Mo et al 2014).

2 These initiatives are most effective when used as a complement to, rather than a substitute for, regular classroom instruction. (Linden 2008; Lai et al 2015; Lai et al 2012; Yang et al 2013; Huang et al 2014).

2 These programs are often combined with free laptops, which by themselves have no impact on student achievement. (Mo et al 2013; Carrillo et al 2011).

Evidence: Increasing teacher and/or principal effort

Increasing the role of parents in school management

1

Evidence: Increasing the role of parents in school management

1 Many interventions have provided information to communities about how to get involved in school management. Yet, more information has not always translated into more action. (Banerjee et al 2010; Barr et al 2012).

2 Some interventions have affected intermediate outcomes, but have had small and isolated effects on student achievement. (Pandey et al 2009).

3 Providing schools with monetary grants that communities can manage have varied widely in their effects, depending on the pre-existing level of community capacity. (Beasley & Huillery 2015; Gertler et al 2012; Blimpo & Evans 2011; Carneiro et al 2015; Pradhan et al 2014; Lassibille et al 2010; Glewwe & Maïga 2011).



1

2 Offering incentives to teachers/principals

Incentives to teachers/principals

1 Doubling teacher salaries without increasing standards has no impact on teacher effort or student achievement. (de Ree et al 2015)

2 Rewarding teachers’ for effort (i.e., to attend school) has increased their effort and student learning when it has relied on non-discretional, systematic monitoring. (Duflo et al 2012; Chen et al 2001; de Laat et al 2008).

3 Rewarding teachers’ for performance (i.e., to increase test scores) has increased student learning in the short- and medium-term. (Muralidharan & Sundararaman 2011; Muralidharan 2012; Lavy 2009, 2013).

4 In some contexts, these programs elicited “teaching to the test”. (Glewwe et al 2010).



1

2 Offering incentives to teachers/principals

3

Hiring teachers on fixed-term contracts

Evidence: Hiring teachers on fixed-term contracts

1 In many countries, it is very difficult to dismiss public school teachers for poor performance. This has led some to hire teachers on fixed-term, renewable contracts.

2 In the short-term, these initiatives have resulted in higher effort among “contract” teachers and higher student achievement. (Muralidharan & Sundararaman 2013).

3 Yet, in some contexts, contracts were given to relatives of existing teachers and regular teachers reduced their effort. (Duflo et al 2015).

4 These initiatives have been more difficult to implement when led by governments, resulting in no gains in teacher effort or learning. (Bold et al 2013).

Evidence: Building teachers’ capacity

1 Training teachers

on specific skills

Evidence: Training teachers on specific skills

1 Training teachers “on-the-job” is difficult. The few programs that have been evaluated have documented wide variability in program implementation and no effect on student learning. (Yoshikawa et al 2015; Angrist & Lavy 2001).

2 However, even when faithfully implemented, these programs have had no effect on student achievement. (Zhang et al 2013; Yue et al 2012).


1 Training teachers

on specific skills

2

Preparing lessons for teachers

Evidence: Preparing lessons for teachers

1 Countries where teacher capacity is low have experimented with “scaffolding” in which teachers are given the materials to teach a lesson and trained on how to teach it.

2 Some prepare lessons for teachers to facilitate students’ independent learning (e.g., a “readathon”). (Abeberese et al 2014).

3 Others prepare lessons that teach teachers how to impart a specific skill (e.g., early grade reading). (Lucas et al 2014; Githua & Nyabwa 2008; Spratt et al 2013; Van Staden 2011; He et al 2008).

4 In some cases, the lesson is actually scripted or pre-recorded. (Naslund-Hadley et al 2014; Piper 2009).


3

Letting students

teach each other

1 Training teachers

on specific skills

2

Preparing lessons for teachers

Evidence: Letting students teach each other

1 Some initiatives have attempted to bypass low teacher capacity by encouraging peer-to-peer learning.

2 In secondary school, when students are more independent, these programs have had positive impacts on student learning. (Wachanga & Mwangi 2004; Beuermann et al 2013).

3 In primary school, when students rely on the teacher as facilitator, these programs have been far less successful. (Berlinski & Busso 2013).

Evidence: Important caveats in interpreting current evidence

1 Details of design and implementation matter.

2

Average effects may mask considerable heterogeneity by groups.

3

Consequences are likely to depend on circumstances in the particular setting. • Many children out of school → focus on demand-side

interventions. • Most children in school, low learning → improving

instruction is crucial. • Teachers have capacity but low effort → aligning the

incentives of students, parents, and teachers may make more sense.

4 Most of what we know concerns short-term outcomes. Examining longer term consequences of promising interventions is important.

Thank you E-mail: [email protected] Website: http://scholar.harvard.edu/alejandro_ganimian

mailto:[email protected]

http://scholar.harvard.edu/alejandro_ganimian

http://scholar.harvard.edu/alejandro_ganimian

what can we learn from education impact...

Documents