educator effectiveness: state frameworks and local practice(??) ccsso annual conference, june 2012...
TRANSCRIPT
![Page 1: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/1.jpg)
Educator Effectiveness:State Frameworks and
Local Practice(??)
CCSSO Annual Conference, June 2012
Juan M. D’BrotExecutive Director of Assessment and Accountability
West Virginia Department of Education
![Page 2: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/2.jpg)
Assessment Continuum in WV
• Promoting a balanced assessment system against WV State Standards
Formative DiagnosticsFormative Diagnostics
Interim DiagnosticsInterim Diagnostics
Interim BenchmarksInterim Benchmarks
Summative AssessmentsSummative Assessments
![Page 3: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/3.jpg)
Formative, Interim, and Summative Assessments
• Summative – WESTEST 2– Online Writing Assessment (PT)
• Interim– Benchmarks in Math, ELA, Science, Social
Studies• Formative– Acuity Item Banks in Math, ELA, Science, Social
Studies– West Virginia Writes: Practice Writing Program– Other District Purchased Supports– Teacher Developed Assessments
![Page 4: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/4.jpg)
Major Question: Purpose? • Purpose of Summative
– Evaluative against a set of standards– Least flexible: long-cycle– Broad sense of the educational state of the state or
district• Purpose of Interim
– Instructional,– Evaluative, and/or– Predictive– Medium-cycle
• Purpose of Formative – Process to adjust teaching and learning – Instructional diagnostics– Typically classroom specific– Very short-cycle
![Page 5: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/5.jpg)
Examining the Use of Formative and Interim
• High Use vs. Effective Use– Emerging case study of a district and school
• Context– Targeted professional development– Dedicated school leadership and staff– State-level support around the use of
formative and interim assessments
• Early Results:
![Page 6: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/6.jpg)
Grade 9 School A and B
![Page 7: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/7.jpg)
Grade 10 School A and B
![Page 8: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/8.jpg)
Grade 11 School A and B
![Page 9: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/9.jpg)
WESTEST 2 Writing: At or Above Proficiency
![Page 10: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/10.jpg)
Examining the Use of Formative and Interim
• Strong support for use of writing short-cycle assessments
• Currently examining use of other short-cycle (formative) and medium-cycle (interim) assessments on summative assessment results
• Implications on teacher evaluation systems looking for triangulated evidence of student learning
![Page 11: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/11.jpg)
Multiple Use Concerns
• Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments)
Adequate Yearly Progress Improvement and Growth
Teacher Evaluation Systems Student Learning
Conflicting?
![Page 12: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/12.jpg)
Multiple Use Concerns• Tension between Compliance (accountability)
and Continuous Improvement (short-cycle assessments)– Summative Assessment
• Gaming• Cheating• “Teaching to the Test”
– Interim/Formative Assessment• Fabricated student learning gains? • Student sorting to highlight teacher “gains”? • What does the research say? Not impact/effect, but
valid link to teacher input/output?• Cognitive vs. Non-Cognitive measures? Scratching
the surface
![Page 13: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/13.jpg)
The West Virginia Case
• A transition from nothing to something– Old: 5 years experience evaluation “only
when the principal determines it to be necessary…”
– New: Yearly evaluation for all educators, principals, and evaluators using research-based professional standards and evidence of student learning
![Page 14: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/14.jpg)
Developing the System• Goal: To develop a supportive system
around valid data to drive improvement
• Access: A variety of data to support self-reflective practice:– Summative Growth Data– Interim Benchmarks– Formative Diagnostic
![Page 15: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/15.jpg)
Developing the System• Context: – Politics: National vs. State Pressure– Policy: Accountability– Pragmatics: Using actionable and valid data
• Development: Stakeholder driven– Board of Education – SEA Staff– LEA staff– Teacher Unions– Legislators– Community Members
![Page 16: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/16.jpg)
Category Input Output
5 Professional Teaching Standards 80%
14 Critical Standard Elements – 4 pt. rubric
5.7%/CSE
Student Learning Outcomes 20%
Summative Growth Data•Math & ELA Only
5%
Other Student Learning Evidence•Any Content 15%
Professionalism (Mandatory) N/A N/A
The System• System Components
![Page 17: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/17.jpg)
The System• 20% Student Learning Outcomes– Minimum 5% Student Assessment Data• School-wide Growth – Student Growth
Percentiles
– A host of evidence– Agreed upon with principal input– Multiple measures• Acuity (Math, ELA, Science, Social Studies)• WV Writes (Writing)• Other evidence of student learning• Teacher Developed Assessments
![Page 18: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/18.jpg)
The System• Overall Ratings
Category 100 Point Scale
Distinguished 88-100
Accomplished 54-88
Emerging 33-54
Unsatisfactory <33
![Page 19: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/19.jpg)
Implementation
• 2011-2012 – Concluded 1st pilot year in a small number of schools – Self-ratings– Accomplished across the board…
• 2012-2013 – 2nd pilot year with schools in every district represented
• 2013-2014 – Full statewide implementation for all schools in the state
![Page 20: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/20.jpg)
Implications for a “Young System”
• Embedded Program Evaluation – WVDE Office of Research small scale
implementation study and program evaluation
– Formative data has resulted in changes to the system and program
– Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)
![Page 21: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/21.jpg)
Lessons Learned• Politics vs. Pragmatism– Representative inclusion vs. Over/under-
representation of certain views– Political support to push a bill to law
• Capacity and Foresight– In-house development and reinventing the
wheel? (…more politics)– Building a system for the present vs.
building a system for the future– Revising the re-invention
![Page 22: Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f345503460f94c51e00/html5/thumbnails/22.jpg)
Thank You
Juan M. D’Brot([email protected])
Executive Director
Office of Assessment and Accountability
West Virginia Department of Education