![Page 1: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/1.jpg)
Portability of Teacher Effectiveness across School Settings
Zeyu Xu, Umut Ozek, Matthew Corritore
April 21, 2023
Bill & Melinda Gates FoundationEvaluation of the Intensive Partnership Sites initiative
![Page 2: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/2.jpg)
Motivation Redistributing effective teachers at the center of several education
policy initiatives Teacher is the most important school input related to student learning The distribution of effective teachers is uneven (recruiting, who moves, and
to where)
Key assumption: Teachers effectiveness is portable Students face different challenges in learning School culture, environment and working conditions may affect teacher
learning, practices, efforts, burnout, etc.
Literature Jackson (2010), Jackson & Bruegmann (2009), Goldhaber & Hansen (2010) Sanders, Wright & Langevin (2009)
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 3: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/3.jpg)
Research Questions Do teachers retain their effectiveness across schools
On average Across schools with similar settings Across schools with different settings (by the direction of the change)
Teacher effectiveness measured by Value-added
Settings defined by School performance levels School poverty levels
Conditional on teachers switching schools
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 4: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/4.jpg)
Preview of Findings Among teachers who changed schools, on average their VA was
unchanged or slightly improved
The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move
High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years
This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves
Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 5: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/5.jpg)
Organization Data and samples
Methodology
Findings
Summary and discussion
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 6: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/6.jpg)
Data North Carolina 1998-99 through 2008-09
Elementary level (4th and 5th grade math and reading teachers, self-contained classrooms)
Secondary level (algebra I and English I teachers, “Algebra I”, “Algebra I-B”, “Integrated Math II”, “English I” classrooms)
Florida 2002-03 through 2008-09 Elementary level (4th and 5th grade math and reading teachers, “core
courses” in a given subject) Secondary level (9th and 10th grade math and reading teachers, “core
courses” in a given subject)
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 7: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/7.jpg)
Sample restrictions Remove charter schools
Remove students and teachers who changed schools during a school year (about 2-4% of obs)
Remove students with missing values on covariates
Keep classrooms with 10~40 students
Remove classrooms with >50% special education students
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 8: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/8.jpg)
Sample sizes
North Carolina Florida
Elementary Secondary Elementary Secondary
Math 21,119 4,999 29,989 9,101
Reading 21,119 3,775 29,354 9,681
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Number of Unique Teachers in the Analytic Samples
![Page 9: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/9.jpg)
Two-Stage Analysis
Estimate teacher-year value-added
Difference-in-differences analysis
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 10: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/10.jpg)
Estimate Teacher VA
Test scores standardized by year, grade and subject (mean=0, sd=1) (X) Covariates include:
1) grade repetition, 2) FRPL, 3) sex, 4) race/ethnicity, 5) gifted, 6) special education, 7) student school mobility and 8) grade level.
Bias (no school FE) Noise (EB adjustment) Alternative model specifications (achievement levels model)
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
𝐴𝑖𝑡 −𝐴𝑖𝑡−1 = 𝑇𝑖𝑡𝛽+𝑋𝑖𝑡𝛾+𝜀𝑖𝑡
![Page 11: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/11.jpg)
DiD
Three groups: non-movers, movers to a similar school setting, movers to a different school setting
FGLS, se clustered at the teacher level (Y) Year and (T) teacher FEs (X) Teacher experience (0-2, 3-5, 6-12, 13 or more years of exp) (S) School quality (average peer VA) (C) Classroom characteristics (FRL %, mean pretest score, sd of
pretest score) (Post) Post-move years indicator (DP, DN) Indicators for school setting differences
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
𝛽መ𝑗𝑡 = 𝑌𝑡 +𝑇𝑖 +𝑋𝑗𝑡𝑣1 +𝑆𝑗𝑣2 +𝐶𝑗𝑡𝑣3 +𝑃𝑜𝑠𝑡𝑗𝑡𝑣4 +𝑃𝑜𝑠𝑡𝑗𝑡𝐷𝑃𝑗𝑣5 +𝑃𝑜𝑠𝑡𝑗𝑡𝐷𝑁𝑗𝑣6 +𝜀𝑗𝑡
![Page 12: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/12.jpg)
Define School Settings School performance
NC: % students performing at or above grade level FL: School performance scores based on both levels and growth Standardized by year and aggregated across all years
School poverty % FRPL Aggregated across all years in which a teacher taught in that school
Change in school setting measures ∆ = Receiving school – Sending school Similar setting = within half a SD around the mean of the ∆ distribution DP = 1 if ∆ > 0.25 (performance) or ∆ > 0.15 (poverty) DN = 1 if ∆ < -0.25 (performance) or ∆ < -0.15 (poverty)
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 13: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/13.jpg)
Alternative DiD Specs Last pre-move year and first post-move year
Between- vs. within-district moves
Replace the post-move indicator with individual year dummies (It-1, It-2, It-3…; It+1, It+2, It+3)
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 14: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/14.jpg)
Distribution of Movers› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
By school performance setting change
![Page 15: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/15.jpg)
Distribution of Movers› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
By school poverty setting change
![Page 16: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/16.jpg)
Mover Characteristics› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
NC elementary school teachers, by mobility status
![Page 17: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/17.jpg)
Pre-Post Change in VA (elem)› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
North Carolina Florida
Math Reading Math Reading
All 0.004 0.005 -0.001 0.002
By school perf.
Higher to lower 0.019 0.011 0.002 0.002
Similar 0.004 0.004 0.007 -0.001
Lower to higher -0.002 0.003 -0.005 0.004
By school poverty
Higher to lower -0.005 0.002 -0.004 0.002
Similar 0.005 0.004 0.000 0.002
Lower to higher 0.020 0.017 0.009 0.009
![Page 18: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/18.jpg)
Pre-Post Change in VA (sec)› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
North Carolina Florida
Math Reading Math Reading
All 0.056 0.003 0.003 0.005
By school perf.
Higher to lower 0.067 -0.011 0.003 0.013
Similar 0.085 0.014 0.006 0.008
Lower to higher 0.030 0.005 0.002 0.000
By school poverty
Higher to lower 0.111 0.002 0.002 -0.006
Similar 0.057 0.010 0.004 0.006
Lower to higher -0.010 -0.020 -0.003 0.019
![Page 19: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/19.jpg)
By Pre-Move VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Actual year of move “Pseudo” move
![Page 20: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/20.jpg)
By Pre-Move VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Elementary math teachers Elementary math teachers (pseudo move)
![Page 21: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/21.jpg)
By Pre-Move VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Elementary reading teachers Elementary reading teachers (pseudo move)
![Page 22: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/22.jpg)
By Pre-Move VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Secondary math teachers Secondary math teachers (pseudo move)
![Page 23: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/23.jpg)
By Pre-Move VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Secondary reading teachers Secondary reading teachers (pseudo move)
![Page 24: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/24.jpg)
Adjacent Year Correlations› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Correlation
North Carolina Florida
Math Reading Math Reading
Yt-2, Yt-1 0.483 0.298 0.380 0.187
(0.426, 0.535) (0.232, 0.362) (0.314, 0.443) (0.111, 0.260)
Yt-1, Yt+1 0.341 0.270 0.302 0.138
(0.256, 0.420) (0.182, 0.354) (0.231, 0.369) (0.061, 0.213)
Yt-+1 Yt-+2 0.463 0.269 0.427 0.191
(0.381, 0.537) (0.175, 0.358) (0.363, 0.487) (0.115, 0.264)
![Page 25: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/25.jpg)
Pre-Post Comparisons of VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
North Carolina
![Page 26: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/26.jpg)
Pre-Post Comparisons of VA› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
Florida
![Page 27: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/27.jpg)
Summary Among teachers who changed schools, on average their VA was
unchanged or slightly improved
The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move
High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years
This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves
Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023
![Page 28: Portability of Teacher Effectiveness across School Settings](https://reader036.vdocuments.us/reader036/viewer/2022062423/568145a7550346895db29bfc/html5/thumbnails/28.jpg)
Discussion Teacher effectiveness does not appear to be hurt by moving to
schools with different settings.
Multiple years of VA estimates can be used with other teacher evaluation data to identify effective teachers, capturing persistent teacher performance better and reducing post-move year shrinkage.
All results take teacher school changes as given.
› Introduction› Data and Samples› Methodology› Findings› Summary and Discussion
April 21, 2023