Evaluation of Teacher Preparation Programs:

Purposes, Methods, and Policy Options

Robert E. Floden, Michigan State University & Jeanne Burns, Louisiana Board of Regents

CCSSO WebinarDecember 10, 2013

National Academy of Education report

• Feuer, M. J., Floden, R. E., Chudowsky, N., & Ahn, J. (2013). Evaluation of Teacher Preparation Programs: Purposes, Methods, and Policy Options. Washington, DC: National Academy of Education.

• Supported by NSF Award No. 1153848• Full report: http://bit.ly/193kNof

Purpose of Report

• Foster thoughtful discussion of approaches to evaluation of teacher preparation programs (TPP)

• Describe dimensions of variability in current systems of TPP evaluation:– Purposes– Systems– Sources of evidence

• Describe examples from other fields and nations• Offer guidance for TPP evaluation system design and

revision

Focus on Preparation Programs

• Evaluation of Programs, not Individual Teachers

• Most programs are part of higher education system

• Approximately 1,400 higher education institutions prepare teachers in the US

• Some teacher preparation done by other organizations, such as Teach For America

Purposes for Evaluating Teacher Preparation Programs

• Hold programs accountable • Provide consumer information to– prospective TPP students – their potential future employers

• Support program self-improvement

Systems for TPP Evaluation

• Federal (Title II) reporting requirements• National non-governmental (e.g., CAEP)• State program approval• Media and independent organizations (e.g.,

US News & NCTQ)• Teacher preparation programs – individually

and in networks

Principles Guiding our Analysis• Importance of validity (i.e., defensible conclusions

and desired consequences)• Program evaluation is not sufficient for improvement• Multiple players with varying purposes and interests• Weigh limits and benefits of system• Consider differential effects on diverse populations• Systems should be adaptable• Systems should be transparent and held accountable

Examples of Types of Evidence

• “Input” measures– Selectivity– Faculty qualifications– Quality and substance of instruction– Student teaching experience

• “Output” measures– Teacher tests and performance assessments– Hiring and retention– Surveys of employers– Impact on student learning

Typical Input MeasureStrengths and Weaknesses

• Strengths– Some evidence relatively easy to gather– Face validity – e.g., faculty should be experts,

teachers should study learning theories• Weaknesses– Little empirical evidence of connection to teacher

quality– Difficult to gather evidence on quality of instruction– May produce superficial compliance

Typical Output Measure Strengths and Weaknesses

• Strengths– Focuses directly on desired outcomes– Familiar to key constituencies – funders, citizens,

employers• Weaknesses– Many important outcomes difficult to measure– May ignore variation in institutional mission– Challenging to account for differences in incoming

students

Strengths and Weaknesses: An output example

• Average Value Added of TPP graduates:– Strengths• Connection to outcome valued by policy makers and

employers• May be used in system by which graduates are evaluated

– Weaknesses• Confounded with school placements• Little information about change in TTP students due to

program• Grain size too coarse for program improvement

Questions to Ask When Designing or Revising TPP Evaluation Systems

• 1: What is the primary purpose of the TPP evaluation system?

• 2: Which aspects of teacher preparation matter most?• 3: What sources of evidence will provide the most

accurate and useful information about those aspects?• 4: How will the measures be analyzed and combined?• 5: What are the intended and potentially unintended

consequences of the evaluation system?• 6: How will transparency be achieved? • 7: How will the evaluation system be monitored?

Teacher Preparation Transformation GuideElements Descriptions

Teacher Preparation Concerns/Needs:  

Primary Purpose of Teacher Preparation Transformation:

   

Primary Stakeholders to be Engaged in the Teacher Preparation Transformation:

  

Aspects of Teacher Preparation Programs that Matter the Most:

Characteristics of “Learner Ready” New Teachers: 

 

Initiatives that Support the Teacher Preparation Transformation:

. 

Types of Evidence for Outcomes: 

Web Site Links: 

 

Reports to Guide Teacher Preparation Transformation Discussions:

 

Tool to Guide NTEP & NAE Report Discussions

Screens/Assessments/Evidence for Teacher Preparation Programs

 Teacher Preparation Stages

Types of Screens/Assessments/EvidenceCurrent University Requirements

Current State Law/Policy/Procedures

Recommendations of Other Stakeholders

 Admission to Universities

Entry Into Teacher Preparation Programs (e.g., Professional Coursework)

Teacher Preparation Progress & Entry into Student Teaching 

Completion of Teacher Preparation Program 

Teacher Licensure 

Post-Graduate Assessment of Teacher Preparation Completers 

Evaluation of Teacher Preparation Programs by State and National Agencies 

Tool to Guide NTEP & NAE Report Discussions

http://www.naeducation.org/xpedio/groups/naedsite/documents/webpage/naed_085581.pdf