5 years later: tracking graduates into the field

Five Years Later: Tracking Graduates into the

Field

AACTE 2016

East Carolina University, College of EducationUniversity of North Carolina- Chapel Hill, Education Policy Initiative at Carolina

KRISTEN CUTHRELL JOY STAPLETONLIZ FOGARTY

ANN BULLOCK

ADRIENNE SMITH AUBREY COMPERATORE

Presentation Goals

● Challenges in efficiently evaluating COE innovations○ Following graduates○ Robust measurements and pertinent outcomes○ Effective partnerships with external evaluators

● Results of our evaluations● Lessons learned

COE Innovations

Challenge #1

How do we follow our pre-service teachers into the field?

● Partnered with Educational Policy Initiative at Carolina (EPIC) ● access to large, longitudinal datasets● ability to track all ECU graduates teaching

in NC public schools

Challenge #2

How can we look at effectiveness across a wide range of outcomes?

● Collaborated with EPIC to explore○ ways graduates teach ○ how much their students academically gain ○ how graduates feel about their preparation ○ how long graduates intend to stay in the

teaching profession

Challenge #3

How do we know if some of the innovations are more effective than others?

● Established a data system to track each pre-service teacher’s exposure to individual reforms

● Designed mini-pilots to explore the effectiveness of individual reforms

● Formed research groups to grow internal research capacity and promote faculty buy-in

● Developed explicit descriptions of the innovations to clearly define treatment for replication

Results

● CLASS and Instructional Strategies● Student Achievement● Perceptions of Self-Confidence and Job

Satisfaction● Retention

CLASS

Positive Climate

Negative Climate

Teacher Sensitivity

Respect for Student Perspective

Classroom Organization

Productivity

Content Understanding

Instructional Learning FormatsAnalysis and Problem Solving

Quality of Feedback

Instructional DialogueStudent Engagement

Instructional Strategies

Thinking Maps

Advanced Maps

Examples & Non-examples

Compare & Contrast

Games

Higher-level Questioning

Think, Pair, Share

Jigsaw

Formative Assessment

Summative Assessment

The difference in teacher EVAAS scores (generated using student achievement scores) between ECU and UNC non-ECU comparison sample teachers is not statistically significant for any subject-grade combination.–Results do not change even after controlling for school level covariates

We looked at:

Elementary Grades Math

Elementary Grades Reading

Elementary Grades Science

Middle Grades Math

Middle Grades Reading

Middle Grades Science

Student Achievement

The difference in scores between ECU and UNC non-ECU comparison sample teachers is not statistically significant for self-confidence or job satisfaction composites.

–Results do not change even after controlling for school level covariates

We looked at:

Perceptions of Self-Confidence

(e.g., I feel confident in my ability to differentiate instruction based on student needs)

Job Satisfaction

(e.g., I consider teaching to be my ideal career)

Graduate Perceptions of Self-Confidence and Job Satisfaction

Retention

95%

92% 92%

91%

*

**

Retention

ECU nonECU

**

*

**

*

ECU non ECU

**

* *

*

Retention

Lessons Learned

What did we learn?

● Gather and use actionable data to drive reflection and program improvement.○ teach strategies graduates can use once in the

classroom● Develop innovations across program areas:

○ create consistency in program○ systemic and longstanding

• sustain and measure impact ● Continue to implement the innovations and track

graduates. ● Maintain momentum.

Questions?●Questions,

Concerns, Ideas, Suggestions…

●Ask now or

●Send an email○ Kristen Cuthrell

cuthrellma@ecu.edu○ Adrienne Smithadrsmith@email.unc.edu