magnet schools and peers: effects on student achievement dale ballou vanderbilt university november,...
Post on 21-Dec-2015
214 views
TRANSCRIPT
Magnet Schools and Peers: Effects on Student Achievement
Dale Ballou
Vanderbilt University
November, 2007
Thanks to Steve Rivkin, Julie Berry Cullen, Adam Gamoran, Ellen Goldring and Keke Liu.
Research Questions
• Has attending a magnet school caused an increase in mathematics achievement?
• How large is the influence of peers on mathematics achievement?
• How much of the magnet school effect remains after controlling for the influence of peers?
Study Setting
• Middle Schools in a Large Southern District
1 selective academic magnet
4 non-selective magnets
5 student cohorts
6 years: 1998-99 through 2003-04
Grades 5 & 6
Admissions Lotteries
• Oversubscribed magnets conduct lotteries
• Students may enter multiple lotteries
• Students who are not outright winners are placed on wait lists
• Wait-listed students accepted until the first week of school
Non-lottery Admissions
• Sibling preferences• Promotion from a feeder school• Geographic priority zone
These students are not included in the study sample (though they do enter the calculation of peer characteristics).
Research Design
• Lotteries assign students randomly to school type and to peers (magnet school peers vs. non-magnet peers).
• Randomized design circumvents biases arising from self-selection of schools and peers.
Limitations of Design
• Results may not generalize beyond lottery participants.
• Effects are relative (magnet schools vs. mix of non-magnet schools attended by lottery losers).
Lottery Participation Academic Composite
Non-Academic
Applicants 2315 2594
Outright Winners
883 1450
Delayed Winners
223 756
Losers,This Lottery
1209 388
Losers, All Lotteries
539 199
Grade 5 Enrollments
Academic Composite Non-Academic
This Magnet 758 1061
Other Magnets 287 339
Non-Magnets 834 846
Left system or Not Tested
436 346
• Substantial non-compliance, especially among winners of non-academic lotteries, attenuates estimated treatment & peer effects based on comparison of winners and losers.
Remedy: Use lottery outcomes as instruments to predict probability of attending magnet school, outcomes interacted with peer variables at magnet & zoned schools as instruments for peer characteristics.
Potential Pitfalls
• High rates of attrition from district can introduce systematic differences between treatment and control groups.
Remedies:Control for student characteristics (race, income, ESL, special ed, gender, prior achievement).Analyze attrition patterns for evidence of differences between winners and losers.
• Participating in multiple lotteries increases chances of winning. “Multiple participants” may differ in ways related to achievement.
Remedy: Control for the combination of lotteries each student entered. Winners are compared to losers who entered the same combination.
• Lotteries randomly assign students to magnet school peers or peers in their neighborhood (zoned) school, but lotteries do not determine the characteristics of the latter—residential decisions do.
Remedy: Control for characteristics of the peers in the zoned school.
Peer Characteristics
• Percentages black, low income (free & reduced-price lunch program), special ed, ESL, female
• Absenteeism rate
• Disciplinary incidents (rate per student)
• Intra-year mobility
• Prior achievement in math and reading
Model (Summary)
• Two treatment variables (academic magnet, composite non-academic magnet)
• Variation in peers resulting from lottery outcomes
• Other controls (student characteristics, peers at the zoned school, lottery participation indicators, year by grade effects)
Findings
• When model does not include peer characteristics
- Academic magnet, + 18% in grade 5, drops to +10% in grade 6 (% of normal year growth)
- Non-academic magnet, no grade 5 effect, +54% in grade 6
• When models include peer characteristics- Reducing percent black from 75% to 25% increases scores by 60% of normal year growth.- Effect of percent low income is about half that large.- Other peer characteristics have no statistically significant effect.
- Controlling for either percent black or percent low income, the effect of the academic magnet disappears.
- The large 6th grade effect in the non-academic magnets remains substantially undiminished.
Checking Alternative Interpretations
• Are peers a proxy for heterogeneous response to treatment?
Check: Interact magnet treatment indicators with all observed student characteristics.
Finding: Peer effects are undiminished.
• Are peers a proxy for teacher quality?
Check: Control for teacher quality by including teacher fixed effects.
Finding: Peer effects are undiminished.
Attrition, Academic Magnet
Lottery Participants
Left System After Grade: Winners Losers
4 13% 21%
5 8% 14%
6 9% 11%
7 6% 9%
Attrition, Composite Non-Academic Magnet
Lottery Participants
Left System After Grade: Winners Losers
4 8% 12%
5 12% 9%
6 10% 4%
7 12% 16%
Potential Attrition Biases
• Lottery losers are more likely to leave the system than winners.
• Losers are also more likely to leave when they can afford private schooling. These tend to be higher-achieving students.
Result: Losers who remain in the system have lower achievement than winners who remain.
• Unfavorable peers at zoned school make losers more likely to leave system.
• Effect greatest among those who can afford private schooling.
Result: Quality of peers positively correlated with losers’ achievement. Estimated peer effects appear too strong.
Checking Attrition Bias
• Are rates of attrition correlated with variables that predict individual achievement (race, income, prior achievement)?
• Yes, but not differently for winners and losers.
Conclusions
• For at least some students in some places, magnet schools have a positive effect on academic achievement.
• There are very strong peer effects on middle school achievement. Do not appear to operate through behaviors readily quantified with administrative data (attendance, disruptions, mobility).