Download - Laura Goe, Ph.D
How can we measure teachers’ contributions to student learning growth
in the “non-tested” subjects and grades?
Laura Goe, Ph.D.Research Scientist, ETS, and Principal Investigator for the
National Comprehensive Center for Teacher Quality
Connecticut Association of Public School Superintendents Conference
Tuesday, December 6, 2011 Cromwell, CT
2
Laura Goe, Ph.D.
• Former teacher in rural & urban schools Special education (7th & 8th grade, Tunica, MS) Language arts (7th grade, Memphis, TN)
• Graduate of UC Berkeley’s Policy, Organizations, Measurement & Evaluation doctoral program
• Principal Investigator for the National Comprehensive Center for Teacher Quality
• Research Scientist in the Performance Research Group at ETS
3
The National Comprehensive Center for Teacher Quality
• A federally-funded partnership whose mission is to help states carry out the teacher quality mandates of ESEA
• Vanderbilt University• Learning Point Associates, an affiliate of
American Institutes for Research• Educational Testing Service
4
Today’s presentation available online
• To download a copy of this presentation or look at it on your iPad, smart phone or laptop, go to www.lauragoe.com Go to Publications and Presentations page Today’s presentation is at the bottom of the
page
5
The goal of teacher evaluation
The ultimate goal of all teacher evaluation should be…
TO IMPROVE TEACHING AND
LEARNING
6
A simple definition of teacher effectiveness
Anderson (1991) stated that “… an effective teacher is one who quite consistently achieves goals which either directly or indirectly focus on the learning oftheir students” (p. 18).
7
Race to the Top definition of student growth
• Student growth means the change in student achievement (as defined in this notice) for an individual student between two or more points in time. A State may also include other measures that are rigorous and comparable across classrooms. (pg 11)
7
8
Measures: The right choice depends on what you want to measure
9
Measures and models: Definitions
• Measures are the instruments, assessments, protocols, rubrics, and tools that are used in determining teacher effectiveness
• Models are the state or district systems of teacher evaluation including all of the inputs and decision points (measures, instruments, processes, training, and scoring, etc.) that result in determinations about individual teachers’ effectiveness
10
Multiple measures of teacher effectiveness
• Evidence of growth in student learning and competency Standardized tests, pre/post tests in untested subjects Student performance (art, music, etc.) Curriculum-based tests given in a standardized manner Classroom-based tests such as DIBELS
• Evidence of instructional quality Classroom observations Lesson plans, assignments, and student work Student surveys such as Harvard’s Tripod Evidence binder (next generation of portfolio)
• Evidence of professional responsibility Administrator/supervisor reports, parent surveys Teacher reflection and self-reports, records of contributions
11
12
Recommendation from NBPTS Task Force (Linn et al., 2011)
Recommendation 2: Employ measures of student learning explicitly aligned with the elements of curriculum for which the teachers are responsible. This recommendation emphasizes the importance of ensuring that teachers are evaluated for what they are teaching.
13
Validity• Measures don’t “have” validity—depends on how
the measure is used• There is little research-based support for the
validity of using ANY measures, including student growth measures, for teacher evaluation Multiple measures may serve as a means of
triangulating teacher effectiveness results• Herman et al. (2011) state, “Validity is a matter of
degree (based on the extent to which an evidence-based argument justifies the use of an assessment for a specific purpose).” (pg. 1)
14
Measuring teachers’ contributions to student learning growth: A summary of current models
Model Description
Student learning objectives
Teachers assess students at beginning of year and set objectives then assesses again at end of year; principal or designee works with teacher, determines success
Subject & grade alike team models (“Ask a Teacher”)
Teachers meet in grade-specific and/or subject-specific teams to consider and agree on appropriate measures that they will all use to determine their individual contributions to student learning growth
Pre-and post-tests model
Identify or create pre- and post-tests for every grade and subject
School-wide value-added
Teachers in tested subjects & grades receive their own value-added score; all other teachers get the school-wide average
15
Washington DC IMPACT:Instructions for teachers in non-tested
subjects/grades
“In the fall, you will meet with your administrator to decide which assessment(s) you will use to evaluate your students’ achievement. If you are using multiple assessments, you will decide how to weight them. Finally, you will also decide on your specific student learning targets for the year. Please note that your administrator must approve your choice of assessments, the weights you assign to them, and your achievement targets. Please also note that your administrator may choose to meet with groups of teachers from similar content areas rather than with each teacher individually.”
15
16
Rhode Island’s SLO language
• “Student Learning Objectives are not set by educators in isolation; rather, they are developed by teams of administrators, grade-level teams or groups of content-alike teachers and, are aligned to district and school priorities, wherever possible.” (pg 12)
From Rhode Island’s “Guide to Measures of Student Learning for Administrators and Teachers 2011-2012” http://www.ride.ri.gov/educatorquality/educatorevaluation/Docs/GuideSLO.pdf
17
18
Tripod Survey• Tripod Survey – the 7 C’s
– Caring about students (nurturing productive relationships);– Controlling behavior (promoting cooperation and peer
support);– Clarifying ideas and lessons (making success seem feasible);– Challenging students to work hard and think hard (pressing for
effort and rigor);– Captivating students (making learning interesting and
relevant);– Conferring (eliciting students’ feedback and respecting their
ideas);– Consolidating (connecting and integrating ideas to support
learning)
4 types of musical behaviors: Types of assessment
1.Responding
2.Creating
3.Performing
4.Listening
1. Rubrics2. Playing tests3. Written tests4. Practice sheets5. Teacher Observation6. Portfolios7. Peer and Self-
Assessment
Assessing Musical Behaviors: The type of assessment must match the knowledge or skill
Slide used with permission of authors Carla Maltas, Ph.D. and Steve Williams, M.Ed. See reference list for details.
20
What assessments are teachers and schools going to use?
• Existing measures Curriculum-based assessments (come with packaged curriculum) Classroom-based individual testing (DRA, DIBELS) Formative assessments such as NWEA Progress monitoring tools (for Response to Intervention) National tests, certifications tests
• Rigorous new measures (may be teacher created)• The 4 Ps: Portfolios/products/performance/projects• School-wide or team-based growth• Pro-rated scores in co-teaching situations• Student learning objectives• Any measure that demonstrates students’ growth towards
proficiency in appropriate standards
21
Using evidence of student learning growth in teacher evaluation
• Teacher preparation for measuring student learning growth is limited or non-existent
• Most principals, support providers, instructional managers, and coaches are poorly prepared to make judgments about teachers’ contribution to student learning growth
• They need to know how to Evaluate the appropriateness of various measures of
student learning for use in teacher evaluation- Work closely with teachers to select appropriate student
growth measures and ensure that they are using them correctly and consistently
22
Collect evidence in a standardized way (to the extent possible)
• Evidence of student learning growth Locate or develop rubrics with explicit
instructions and clear indicators of proficiency for each level of the rubric
Establish time for teachers to collectively examine student work and come to a consensus on performance at each level- Identify “anchor” papers or examples
Provide training for teachers to determine how and when assessments should be given, and how to record results in specific formats
23
Washington DC IMPACT:Rubric for Determining Success (for teachers in non-
tested subjects/grades)
24
Washington DC IMPACT:Rubric for Determining Success (for teachers in non-
tested subjects/grades)
25
Considerations for implementing measurement system
• Consider whether human resources and capacity are sufficient to ensure fidelity of implementation
Poor implementation threatens validity of results• Establish a plan to evaluate measures to determine if
they can effectively differentiate among teacher performance
Need to identify potential “widget effects” in measures If measure is not differentiating among teachers, may be
faulty training or poor implementation, not the measure itself• Examine correlations among results from measures• Evaluate processes and data each year and make
needed adjustments
26
Moving forward
• Create (or revisit) timeline for final decisions and implementation
What do you still need to know to make appropriate recommendations?
• Who needs to be involved in decision-making? Department of Ed? Districts? Teachers? Union? State responsibilities vs. district responsibilities
• How/when will decisions be communicated to stakeholders?
• What resources will be required for training and implementation and where will they come from?
27
Resources and links
• Harvard’s Tripod Survey http://www.tripodproject.org/index.php/index/
• National Response to Intervention Center Progress Monitoring Tools http://www.rti4success.org/chart/progressMonitoring/progressmonitoringtoolschart.htm
• New Haven Teacher Evaluation http://www.nhps.net/scc/index
• New York State Evaluation http://engageny.org/administrators/
• Rhode Island Department of Education Teacher Evaluation – Student Learning Objectives http://www.ride.ri.gov/educatorquality/educatorevaluation/SLO.aspx
• Tennessee Teacher Evaluation http://team-tn.org/
28
ReferencesAnderson, L. (1991). Increasing teacher effectiveness. Paris: UNESCO, International
Institute for Educational Planning.Herman, J. L., Heritage, M., & Goldschmidt, P. (2011). Developing and selecting measures
of student growth for use in teacher evaluation. Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
http://www.aacompcenter.org/cs/aacc/view/rs/26719 Linn, R., Bond, L., Darling-Hammond, L., Harris, D., Hess, F., & Shulman, L. (2011). Student learning,
student achievement: How do teachers measure up? Arlington, VA: National Board for Professional Teaching Standards.
http://www.nbpts.org/index.cfm?t=downloader.cfm&id=1305 Malta, C., and Williams, S. (January 27, 2010). Meaningful assessment in the music classroom.
Presented at Missouri Music Educators Association Conference, Jefferson City, MO.http://dese.mo.gov/divimprove/curriculum/fa/AssessmentintheMusicClassroom.pptx National Response to Intervention Center Progress Monitoring Tools
http://www.rti4success.org/chart/progressMonitoring/progressmonitoringtoolschart.htm
29
Questions?