1. Taking Control of the Teacher Evaluation Frameworkfor South CarolinaJohn Cronin, Ph.D. Senior Director of Education ResearchNorthwest Evaluation Association

2. What NWEA supports The evaluation process should focus onhelping teachers improve. The principal or designated evaluator shouldcontrol the evaluation. Tests should not be the deciding factor in anevaluation. Multiple measures should be used. 3. A simple framework for teacherevaluationEvidence ofprofessionalresponsibilitiesEffective teachingand professionaljob performanceEvidence ofstudentlearningThe evaluation of teachingby classroom observationand use of artifactsEvidence ofprofessionalpracticeThe evaluation of theteachers The evaluation effectiveness of ainmaking teachers progress contribution towardtotheir student goals learning and fulfilling andtheresponsibilities growthof aprofessional educator. 4. What teacher effectivenessinfers Testing A claim that the improvement inlearning (or lack of it) reflected on one ormore tests is caused by the teacher. Classroom observation That the observersratings or conclusions are reliable andassociated with behaviors that causeimproved learning in the classroom. 5. Distinguishing teacher effectivenessfrom teacher evaluation Teacher effectiveness The judgment of a teachersability to positively impact learning in the classroom. Teacher evaluation The judgment of a teachersoverall performance including: Teacher effectiveness Common standards of job performance Participation in the school community Adherence to professional standards 6. Purposes of summative evaluation Make an accurate and defensible judgment of an educatorsjob performance. Provide ratings of performance that provide meaningfuldifferentiation across educators. Goals of evaluation Help educators focus on their students and their practice Retain your top educators Dismiss ineffective educators 7. Learn from others mistakes. 8. Policy has focused on dismissal of poorteachers rather than retention ofexcellent ones.In baseball, exceptionalplayers are much rarerthan average ones.Thus it is vital for ateam to keep its bestplayers. 9. Employment of Elementary Teachers2007-2012NUMBER OF TEACHERS1538000 1544270 15443001485600The elementary schoolteacher workforce shrunk by178,000 teachers (11%)between May, 2007 and May,2012.141500013603802007 2008 2009 2010 2011 2012Source: (2012, May) Bureau of Labor Statistics Occupational Employment StatisticsNumbers exclude special education and kindergarten teachers 10. The impact of seniority based layoffs onschool qualityIn a simulation study of implementation of a layoff of 5%of teachers using New York City data, reliance on senioritybased layoffs resulted would: Result in 25% more teachers laid off. Teachers laid off would be .31 standard deviations moreeffective (using a value-added criterion) than those lostusing an effectiveness criterion. 84% of teachers with unsatisfactory ratings would beretained.Source: Boyd, L., Lankford, H., Loeb, S., and Wycoff, J. (2011). Center for Education Policy.Stanford University. 11. Teacher observation as a part ofteacher evaluationSystematic observation of teacher performanceis a central part of every states teacherevaluation plan. 12. If evaluators donot differentiatetheir ratings,then alldifferentiationcomes from thetest. 13. If performanceratings arentconsistent withschool growth,the media andpublic willdemand to knowwhy. 14. The (Race to the Top teacher evaluation) changes, alreadyunder way in some cities and states, are intended toprovide meaningful feedback and, critically, to weed outweak performers. And here are some of the early results:In Florida, 97 percent of teachers were deemed effectiveor highly effective in the most recent evaluations. InTennessee, 98 percent of teachers were judged to be atexpectations. In Michigan, 98 percent of teachers wererated effective or better.Source: New York Times (2013, March 30). Curious Grade for Teachers: Nearly all Pass.Retrieved from:http://www.nytimes.com/2013/03/31/education/curious-grade-for-teachers-nearly-all-pass.html?pagewanted=all&_r=0 15. Teacher Evaluation Ratings in Six FloridaDistricts 2013FloridaDistrictHighlyEffectiveEffective NeedsImprovementDeveloping Unsatisfactory VA Score FloridaRanking1 44.4% 55.6% 0.0% 0.0% 0.0%2 25.0% 75.0% 0.0% 0.0% 0.0%3 90.9% 9.1% 0.0% 0.0% 0.0%4 60.7% 39.3% 0.0% 0.0% 0.0%5 81.2% 18.8% 0.0% 0.0% 0.0%6 37.3% 54.2% 1.7% 0.0% 6.8%7 81.3% 18.8% 0.0% 0.0% 0.0%8 41.7% 55.6% 1.4% 1.4% 0.0%9 52.2% 47.8% 0.0% 0.0% 0.0%10 27.0% 66.2% 1.4% 0.0% 5.4%11 7.1% 72.6% 9.5% 10.7% 0.0% 16. Teacher Evaluation Ratings in Six FloridaDistricts 2013FloridaDistrictHighlyEffectiveEffective NeedsImprovementDeveloping Unsatisfactory VA Score FloridaRanking1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 1092 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 1213 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 28024 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 27975 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 28316 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 8807 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 4028 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 32749 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 66410 27.0% 66.2% 1.4% 0.0% 5.4% 0 176411 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445 17. Teacher Evaluation Ratings in Six FloridaDistricts 2013FloridaDistrictHighlyEffectiveEffective NeedsImprovementDeveloping Unsatisfactory Ranking1 37.5% 57.9% 3.3% 0.3% 1.0%2 30.7% 30.2% 1.4% 1.2% 0.0%3 10.7% 44.7% 2.0% 1.4% 0.1%4 67.7% 9.4% 0.1% 0.1% 0.0%5 7.0% 71.7% 0.8% 0.1% 0.0%6 18.6% 29.6% 0.2% 0.2% 0.5%How did they rank on value addedscores? 18. Teacher Evaluation Ratings in Six FloridaDistricts 2013FloridaDistrictHighlyEffectiveEffective NeedsImprovementDeveloping Unsatisfactory Ranking VA Score1 37.5% 57.9% 3.3% 0.3% 1.0% 5th -.062 30.7% 30.2% 1.4% 1.2% 0.0% 6th -.073 10.7% 44.7% 2.0% 1.4% 0.1% 3rd -.024 67.7% 9.4% 0.1% 0.1% 0.0% 1st +.035 7.0% 71.7% 0.8% 0.1% 0.0% 2nd +.026 18.6% 29.6% 0.2% 0.2% 0.5% 4th -.04 19. How should tests,observations, and surveys beweighted? 20. Reliability of evaluation weights in predictedstability of student growth gains year to yearObservation by Reliability coefficient(relative to state testvalue-added gain)Proportion of testvarianceexplainedModel 1 State test 81%Student surveys 17% ClassroomObservations 2%.51 26.0%Model 2 State test 50%Student Surveys 25%Classroom Observation 25%.66 43.5%Model 3 State test 33% -Student Surveys 33%Classroom Observations 33%.76 57.7%%Model 4 Classroom Observation50%State test 25%Student surveys 25%.75 56.2%Bill and Melina Gates Foundation (2013, January). Ensuring Fair and Reliable Measures of EffectiveTeaching: Culminating Findings from the MET Projects Three-Year Study 21. Requirements forobservations ofteachers canoverwhelmadministrators. 22. What model of observation ismost efficient? 23. Reliability of a variety of teacher observationimplementationsObservation by Reliability coefficient(relative to state testvalue-added gain)Proportion of testvarianceexplainedPrincipal 1 .51 26.0%Principal 2 .58 33.6%Principal and other administrator .67 44.9%Principal and three short.67 44.9%observations by peer observersTwo principal observations andtwo peer observations.66 43.6%Two principal observations andtwo different peer observers.69 47.6%Two principal observations onepeer observation and three shortobservations by peers.72 51.8%Bill and Melina Gates Foundation (2013, January). Ensuring Fair andReliable Measures of Effective Teaching: Culminating Findings from theMET Projects Three-Year Study 24. Some simple rules to helpyou with Student LearningObjectives. 25. Rule 1 - The goal should ALWAYS beimprovement in a domain (subject)! 26. Rule 2 Goals should be evaluatedindividually and align to the teachersinstructional responsibilities.Although we encourage teachers to collaborateon the setting and implementation of learninggoals. 27. Common problems with instructionalalignment Using school level math and readingresults in the evaluation of music,art, and other specials teachers. Using general tests of a discipline(reading, math, science) as a majorcomponent of the evaluation highschool teachers delivering specializedcourses. 28. Rule 3 - All students should be in playrelative to the goal. 29. Rule 4 The goal should be challengingbut attainable. 30. The difference between aspirational andevaluation goalsAspirational I will meet my targetweight by losing 50 pounds during thenext year and sustain that weight for oneyear.Evaluation I intend to lose 15 pounds inthe next six months, which will move mefrom the obese to the overweightcategory, and sustain that weight for oneyear. 31. Is this goal attainable?62% of students at John Glenn Elementary met or exceeded proficiency inReading/Literature last year. Their goal is to improve their rate to 82% thisyear. Is the goal reasonable?Oregon schools change inReading/Literature proficiency 2009-10 to362 3512010-1129117373 14 34003002001000Growth> -30%> -20% > -10% > 0% > 10% > 20% > 30% 32. Is this goal attainable?45% of the students at La Brea elementary showed average growth or betterlast year. Their goal is to improve that rate to 50% this year. Is their goalreasonable?100%80%60%40%20%0%Students with average or better annualgrowth in Repus school districtLaBrea DistrictAverage 33. Rule 5 - There should ALWAYS be multipledata sources. 34. Rule 6 Consider including a non-cognitivegoal if allowed. 35. The importance of non-cognitivefactors inteacher evaluation 36. Non-cognitive factorsIn education, value-addedmeasurement has focusedpolicy-makers on the teacherscontribution to academicsuccess, as reflected in testscores.Jackson (2012) argues thatteachers may have more impacton non-cognitive factors that areessential to student success likeattendance, grades, andsuspensions.These are not the only measuresthat matter however. 37. Non-cognitive factors Lowered the average student absenteeism by 7.4 days. Improved the probability that students would enroll inthe next grade by 5 percentage points.Employing value-added methodologies, Jacksonfound that teachers had a substantive effect onnon-cognitive outcomes that was independentof their effect on test scores Reduced the likelihood of suspension by 2.8% Improved the average GPA by .09 (Algebra) or .05(English)Source: Jackson, K. (2013). Non-Cognitive Ability, Test Scores and TeacherQuality: Evidence from 9th Grade Teachers in North Carolina. NorthwesternUniversity and NBER 38. Ultimately the principal decides Evaluation inherently involvesjudgment not a bad thing. Evidence should inform and notdirect your judgment. The implemented system shoulddifferentiate performance. Courts respect the judgment ofschool administrators relative topersonnel decisions. 39. Watch out for unintendedconsequences.