the wicked problem of measuring the impact of teacher ... · pupil change* academic behavioral...
TRANSCRIPT
![Page 1: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/1.jpg)
The Wicked Problem of Measuring the Impact of Teacher Prepara:on: Increasing Rigor in
Documen:ng Prepara:on Prac:ces
Larry Maheady, Buffalo State University
Kris:n Sayeski, University of Georgia
The IRIS Center is funded through a coopera6ve agreement U.S. Department of Educa6on, Office of Special Educa6on Programs (OSEP) Grant #H325E120002. The contents of thispresenta6ondonotnecessarilyrepresentthepolicyoftheU.S.DepartmentofEduca6on,andyoushouldnotassumeendorsementbytheFederalGovernment.ProjectOfficer,SarahAllen.
![Page 2: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/2.jpg)
Lignugaris-Kraft, B., Sindelar, P. T., McCray, E. D., & Kimerling, J. (2014). The “wicked question” of teacher education effects and what to do about it. In P. T. Sindelar, E. D. McCray, M. T., & B. Lignugaris-Kraft (Eds.), Handbook of research on special education teacher preparation (pp. 461-471).
Rittel, H. W., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155-169.
The “Wicked Problem” of Measuring the Impact of Teacher Preparation Larry Maheady & Krist in Sayeski, CEEDAR-IRIS Cross-State Convening on June 24, 2016
“To untangle a snarl, loosen all jams or knots and open a hole through the mass at the point where the longest end leaves the snarl…Keep the snarl open and loose at all times…permit it to unfold itself…No snarl is too complicated to be solved by this method; only patience is required.”
—The Ashely Book of Knots (1944)
Target Outcome Context for Assessment Measures Research Models Attitudes/Beliefs Student Potential
Teacher Efficacy Coursework School-based Observations Practica Tutoring
Concept Maps Open-Ended Questions Efficacy Scales
Survey (parametric and non parametric) Interview (quantitative and qualitative)
Knowledge Theory Practice Content
Coursework Seminars Workshops
Multiple Choice Short Answer Essay Application
Experimental Pre-/Posttest Qualitative Analyses
Skills Evidence-based Practices (EBP) High-leverage Practices (HLP) Data-based Individualization
Microteaching TeachLIVE Practica Tutoring Microcredentialling
Low-Inference Observations High-Inference Observations Progress-Monitoring Measures
Experimental Pre-/Posttest Single-case design
Pupil Change* Academic Behavioral Social Functional
Practica Tutoring Behavior Change Project Capstone Project
CBM (Progress Monitoring) Achievement Measures Behavior Measures Criterion-based Measures
Experimental Pre-/Posttest Single-case design
*Pupil Change measured as a result of the application of HLPs and EBPs.
candidate characteristics (presage variables)
context
mentor teacher selection & training
faculty knowledge & skills
content selection & focus
quality of instruction
intensity of experiencequality of supervision
practice-based focus
![Page 3: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/3.jpg)
IRIS Impact and Other TPP Studies
There is a need • to move beyond discussions related to “what” needs to be addressed within
TPP to discussions related to “how” content and skills should be addressed • for TPP to “take the lead” on documen:ng impact (rather than leaving it to
others to determine what should be collected and how those data are to be interpreted) • to gain meaningful insights on prac:ce in order to capture the poten:al of TPP
(program evalua:on)
![Page 4: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/4.jpg)
Study #1: Knowledge and Percep:on Study (IRIS Impact)
Experimental, Pretest-PosTest-Maintenance Across Three Condi:ons (n = 115)
Group #1 Group #2
Group #3
Week 1: Peer Assisted Learning Strategies
Independent Flipped Facilitated
Week 2: Comprehensive Classroom Management
Flipped Facilitated Independent
Week 3: Accommoda:ons: Instruc:onal and Tes:ng
Facilitated Independent Flipped
![Page 5: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/5.jpg)
Study #2: Skill Development: Distributed vs. Massed Prac:ce (Phoneme-Grapheme Correspondence)
![Page 6: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/6.jpg)
Study #3: Skill Development of a HLP: Opportuni:es to Respond Experimental, Pretest-PosTest Across 2 Condi:ons (n = 48)
• Does distributed prac:ce with performance feedback result in differences in teacher candidates’ knowledge of OTR when compared to a massed prac:ce condi:on with no prac:ce feedback?
• Does distributed prac:ce with performance feedback result in teacher candidates’ delivery of a higher ra:o of OTR during a microteaching lesson when compared to the instruc:onal delivery of candidates who received massed prac:ce with no prac:ce feedback?
• Does distributed prac:ce with performance feedback result in differences in teacher candidates’ capacity to accurately deliver specific OTR strategies when compared to a massed prac:ce condi:on with no prac:ce feedback?
• Does distributed prac:ce with performance feedback result in differences in teacher candidates’ percep:ons of competence for the delivery of OTR?
![Page 7: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/7.jpg)
Two Significant Gaps
Teacher Prepara:on Teacher Prac:ce • What effects do prepara:on experiences (i.e., coursework, clinical experiences,
IRIS modules, and student teaching) have on candidate prac:ce (i.e., what they do instruc:onally in authen:c secngs)? • Which program components are most likely to impact candidate prac:ce? • Which prac:ces should be taught to improve student outcomes? • How should prepara:on programs be structured and implemented to improve candidate
prac:ce? Teacher Prac:ce Student Learning • What effect(s) does candidate prac:ce have on important student outcomes? • Are some prac:ces more effec:ve than others in improving student outcomes? • How might these prac:ces be disseminated more broadly and sustained in P-12 secngs?
![Page 8: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/8.jpg)
Strengthen Prepara:on-to-Prac:ce Link
• Short-term • Examine effects of varied prepara:on experiences (e.g., modeling, role-
playing, IRIS modules) on limited range of teaching prac:ce(s); linking to student learning if possible) • Clinical experiences delivered via School-University Partnerships built
around improving student outcomes
• Long term • Track candidates beyond ini:al school placements, monitor performance,
and link to ongoing measures of student progress • Examine effects of induc:on and professional development models on
prac:ce and student learning
![Page 9: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/9.jpg)
Strengthen Prac:ce-to-Learning Link
• Short term • Conduct more research linking candidate prac:ce to important student
outcomes • Consider infusing studies as program requirements (e.g., capstone
research experiences) • Link research to specific student and school needs (i.e., What is your
biggest instruc:onal challenge & how can we help?)
• Long Term • Comprehensive data collec:on, management, and analysis systems • Longitudinal research on teachers, prac:ce, and student learning • Build a broader evidence-based culture in schools
![Page 10: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/10.jpg)
A Continuum of Options for Developing Practice
Clinical year & coaching
![Page 11: The Wicked Problem of Measuring the Impact of Teacher ... · Pupil Change* Academic Behavioral Social Functional Practica Tutoring Behavior Change Project Capstone Project CBM (Progress](https://reader036.vdocuments.us/reader036/viewer/2022070917/5fb7516c2b94b60f052f238d/html5/thumbnails/11.jpg)
Final Thoughts
• Linking teacher educa:on to student outcomes is a “wicked problem” • It cannot be “solved,” but we can strengthen links between • Our prepara:on experiences and what candidates do instruc:onally • Learn what effects their prac:ce has on important student outcomes
• Strengthen these links by: • Building solid School-University partnerships focused on improving student
outcomes • Alloca:ng more :me and effort to promo:ng use of signature set of
prac:ces (High Leverage Prac:ces and Evidence-Based Prac:ces) • Studying prepara:on, prac:ce, and student learning using rigorous research
designs