Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Harlem Children’s Zone Started by Geoffrey Canada in a 97 block area of Harlem in NYC. Combines high energy “no excuses” type charter schools (Promise

Academy) with a web of other community services. 1,300 students attend “Promise Academy” schools, community

programs served 8,058 kids and 5,291 adults. Meant to support children from “birth to college.”

Evaluating Harlem Children’s Zone

Schools Extended school day and school year. Saturday school for students needing extra help. Overall, roughly twice as many hours as other comparable students

around city.

High quality teachers and use “value” added pay incentives. High turnover, but lots of support and feedback. Schools provide medical, dental, mental health services, meals,

incentives for achievement.

Notably, they aim to provide a “culture of achievement” with no excuses.

Widely stated goal (outside and inside of schools) is to send every student to college.

Evaluating Harlem Children’s Zone

Community Programs Baby college early childhood programs, After-school programs (e.g. karate, dance, etc) Family health programs Over 20 programs available to members of the community.

Evaluating Harlem Children’s Zone

Questions of interest:1. How much do kids benefit from this program relative to other

alternatives in the area?

2. Is it the schooling or community services or both that provide benefit?

What strategies do Dobbie and Fryer use to try to “identify” the impact of HCZ?

Evaluating Harlem Children’s Zone

Lottery Too many applicants for spaces in schools, so there is a lottery to

determine who gets offered a slot. Empirical design is essentially to compare lottery winners to lottery

losers in terms of standardized test scores.

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Cohort and Zone Boundaries (Instrumental Variable or IV) Due to timing of when the programs were started, only some aged kids

had access (i.e., had to be entering sixth grade or kindergarten the year the school was opened, those entering those grades the year before weren’t eligible)

While all kids in eligible cohorts can apply for lottery to attend schools, only those within the HCZ were actively recruited. Argument is that this made those kids in HCZ more likely to enroll relative to otherwise similar kids outside of HCZ.

Evaluating Harlem Children’s Zone

So basic design is a diff-in-diff. Compare test score outcomes of eligible cohorts to non-eligible

cohorts within boundary, difference out how these two groups compare just outside of boundary.

Estimated impact = (TSE,In - TSNE,In) - (TSE,Out - TSNE,Out)

Via regression?

TS = α + β1EligibleCohort + β2InHCZ + β3EligibleCohort*InHCZ

Key Assumptions?

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Evaluating Harlem Children’s Zone

Is it the school or the “other” services? How can they try to identify one from the other?

Evaluating Harlem Children’s Zone

IV results If it was the “other” community services, then cohort/zone IV shouldn’t

have shown anything.

That method only produces an “effect” if treated cohort outperforms untreated cohort within HCZ by more than treated cohort outperforms untreated cohort outside HCZ.

Since both treated and untreated cohorts in HCZ are eligible for other services, the estimated effects must be coming primarily from schools.

Evaluating Harlem Children’s Zone

Estimate the effect of having a “sibling” in one of the academies for those who aren’t enrolled in one of the academies. Idea: Siblings of those enrolled in one of the academies should have

similar access to “other” community programs, so if it is those programs that primarily impact kids’ outcomes, siblings of those enrolled should do better than siblings of those not enrolled.

Evaluating Harlem Children’s Zone

Why is Promise Academy so effective at educating the poorest minority students? Teacher quality?

Extremely high turnover (48% left after first year)

Curriculum Seem to focus on what has shown to be effective (210 instruction

days per year, 8 hour instructional days, Saturday school, after-school program, avg class size of 20, no broken windows discipline policy, administers internal evaluations of teachers, requires school uniforms and a dress code, requires a parental contract, etc.)

Money NYC Dept of Ed provides $12,443 per student, HCZ spends and

estimated $19,272 per pupil.

Evaluating Harlem Children’s Zone Summary

Growing body of evidence seems to suggest that early, often relatively large scale interventions can lead to both short-term and long-term impacts.

Questions still remain how to effectively do this at a large level. Things like Perry Preschool and HCZ are very specialized, can they

be scaled up? While some evidence that large scale programs like Head Start have

impacts, impacts are much more modest.

KIPP schools (Knowledge is Power Program) have tried to develop a replicable but flexible platform for effectively educating at-risk kids. A relatively recent paper suggests that at least one of the KIPP

schools in the Boston area is having substantial success.