Making Inferences About Impact on Student Growth

Dr. Patricia Reeves, Associate Professor Educational Leadership & ResearchWestern Michigan University – February 2014Achievement Centered Leadership (ACL) Grant Project: Cohort 1 – Session 1

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

1

The Pending Legislation on Growth• There will be a State Growth and/or Value Added Model

• There will be criteria for mandatory State assessments used for teacher, school, and district effectiveness ratings

• Districts will need to augment the State Growth and Value-Added rating system with a Local Growth Model

• The point of growth models is to make inferences about how educators, schools, and districts are influencing learning and learning rates

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

2

Growth Rating Recap: State Growth Rating + Local Growth Rating

Under the proposed legislation: •25% of evaluation based on growth starting 2014-15 •In 2017-18 growth becomes 50%

•State will provide growth ratings on State assessments •Districts can develop growth ratings based on a local growth model

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

3

Building upon your District implementation plan for School ADvance: Growth Ratings

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

4

Building upon your District implementation plan for School ADvance: Growth Ratings

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

5

Building upon your District implementation plan for School ADvance: Growth Ratings

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

6

How do you know students are successful in your schools?1. How do you define success?

a. What is the district definition?

b. What is the school level definition?

2. What measures help you track how well students are doing?

3. How do those measures align with your definitions of success? 7

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

Three Decision Points for a Value Added or Growth Model

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

8

Influences on Student Learning

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

9

Assessing School, Teacher, and Administrator Influence on Growth

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

10

Developing a Local Growth Model

1.What indicators and measures have you been using to evaluate teachers’ and administrators’ performance?

2.How well aligned are those indicators with your school improvement plan and the district improvement plan?

3.How well aligned are they with your definition of student success?

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

11

Regardless of students’ entering achievement levels, growth is:

KEEPING THEM MOVING UP AND EXPANDING: Goal is to either maintain or accelerate growth rates if at, or above, target achievement levels to stay ahead of a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets) and to branch out.

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

12

MAKING SURE THEY KEEP UP: Goal is to maintain or accelerate the growth rates if at, or above, target achievement levels to stay on a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets).

MOVING THEM UP: Goal is to accelerate growth rates until these students are also on target to reach achievement targets by certain grades in order to get on a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets).

Higher

Middle

Lower

The Point of Growth Models• How are we doing at ensuring that our higher academic

performers are continuing to MOVE UP and branch out*?

• How are we doing at ensuring that our middle academic performers (the majority of our students) are continuing to either KEEP UP or MOVE UP and branch out*?

• How are we doing at ensuring that our lowest performers are on track to CATCH UP?

*Branch out means to expand development in other academic and/or non-academic indicators of student success, e.g. the arts, community service, athletics, and non-core curricular subjects (languages, logic, philosophy, social sciences, etc.)

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

13

Performance Status(descriptive stat models)

Value-Added Models or VAM (linear stat models)

Growth to Proficiency Models (curvi-linear stat models)

Compares performance status of each student to fixed performance targets (e.g. proficiency level or X amount of pre-post growth).

Ignores differences in student growth rates and differences in student background.

Teachers rated on # or % of students achieving targets – based on either proficiency levels or growth levels.

Teacher information shows where each student performed in relation to performance targets for that year.

Compares growth of students to their academic peers and then compares groups of students to other groups of students by teacher.

Often controls for differences in student background.

Teachers rated on teacher to teacher comparisons of student growth by academic peer group or aggregate groups.

Teacher information shows how each teacher’s class grew compared to other teachers’ classes for that year, either by whole comparison group or sub-groups (e.g. low SES, ESL, etc.)

Compares growth of each student to growth of that student’s academic history and academic peers (i.e. students with the same performance history).

Does not need to control for differences in student background because teachers not compared to one another.Teachers rated on how they are moving students toward growth targets; i.e. catching up, keeping up, moving up.

Teacher information shows how each student is growing compared to growth targets for their academic peer group based on rate of growth needed to reach performance targets over several grades.

3 Common approaches for creating growth ratings

14AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

The Logic of aLocal Growth Model

Projected Achievement

Actual Achievement

Value-added

Growth

Individual Student Past Performance

Instruction

15

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

Value-Added Ratings

Growth & Value Added

Changes in student achievement and other success indicators across time

Estimations of Teacher and school influence

16AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Value-Added Models: How they work

Pre-TestYEAR 1

Post-TestYEAR 2

Actual student scale score

Predicted student scale score

Student scale score

Based on observationally similar students

VALUE ADDED

17AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Visualization with a SGP ModelEach student plotted against a backdrop of typical growth and status

18AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Individual Student Monitoring and RTI with a SGP Model

19AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Higher Than Expected

Setting Growth Standards

Value-added

GrowthGrowth

Thresholds

Expected Growth

Lower Than Expected

Unsatisfactory Growth

Evaluation

Categories

Highly Effective

Effective

Needs Improvement

Unsatisfactory

20AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Teacher Evaluation Example with a SGP Model

LGMs can create a single comparable growth metric across all tests.

21AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Growth to proficiency SGM Sets yearly targets that will put low-achievers on pace to meet proficient and narrow achievement gaps

Growth targets (based on proficient by grade 7)

22AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

Processing the Challenge of Rating Individual Teachers and Administrators on Student Growth1. With your district team, discuss what measures you are

using now to determine how teachers are influencing student growth

2. How are you analyzing the data to determine a value-added rating?

3. Where do you think you need to go from here?

4. What conversations will you have back in your district?

23AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

A tale of 2 teachers

One has an Effective RatingThe other,

a Highly Effective Rating

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

24

Reminder: Three Purposes for Performance Evaluation

1. Achieve Organizational Goals, i.e. Student Outcomes

2. Guide the learning, growth, and development of personnel, e.g. teachers and leaders

3. Make decisions about hiring, placement, retention, promotion, and compensation AC

L:Se

ssio

n1:D

DIM

.Ree

ves2

.27.

14

25

Three Purposes for Performance Evaluation

1. Achieve Organizational Goals, i.e. Student Outcomes

2. Guide the learning, growth, and development of personnel, e.g. teachers and leaders

3. Make decisions about hiring, placement, retention, promotion, and compensation

26

Reminders:1. We can follow student success by identifying

success indicators and measuring growth toward established performance standards and beyond

2. We can grow student success by analyzing growth rates and intervening where/how needed with evidence based practices (strategies)

3. We can grow teacher, administrator, school and district performance by assessing both practice and results (i.e. student growth) and setting improvement goals accordingly

27

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

Final Reminder: Growth Models work best when all growth goals are aligned across the school and district and based on definitions (indicators) of student success

ACL:

Sess

ion1

:DD

IM.R

eeve

s2.2

7.14

28

Teachers Administrators

Superintendent Board