valid data for school improvement final

Welcome!

John Cronin, Ph.D.Vice President of Education [email protected]

Using Valid Data to support School Improvement Goals

NWEAIs a not for profit assessment organization that partners with over 7,000 school systems in the U.S. and in 30 countries.

We are a leader in computer-adaptive and formative assessment.

During this session you’ll learn

• Why accurate valid tests are essential.• That are three purposes for school

improvement goals.• What it means to set reasonable goals.• What to consider when writing a goal.

Three purposes of school improvement

goals

Professional Development

Support teacher improvement by tying professional development to learning in

the classroom


Key idea – The focus is on teacher improvement, and the student learning

goal is simply one way to assess progress

Summative Evaluation

Use improvement goals as part of the principal or teacher evaluation process


Key Idea – The goal should be at a level of performance that would exceed that of

the likely “replacement educator”.

Aspirational Goals

A goal that puts students on track to achieve their aspirations (college or

career)

Aspirational Goals

Key idea – Aspirational goals are not always met, but they show students the

level of improvement needed for success.

Why accurate assessments are essential

Computer adaptive tests are accurate for ALL students

Adult Literacy

Beginning Reading

Experience: low performing student

This student’s instructional level (50/50)

RIT Scale

Computer –adaptive tests are accurate for ALL students

Adult Literacy

Beginning Reading

Experience: High performing student

This student’s

instructional level (50/50)

RIT Scale

RTI Cut Score (FSA Level 1)

RTI Identification – Student score is at NWEA 18th percentile, cut score is at the NWEA 24th Percentile (FSA Level 1 Cut Score)

Computer-adaptive test standard error – 2.9

15% of students misidentified

The importance of accuracy in RTI placement

RTI Cut Score (FSA Level 1)

RTI Identification – Student score is at NWEA 18th percentile, cut score is at the NWEA 24th Percentile (FSA Level 1 Cut Score)

Non-adaptive test with FSA equivalent standard error at Level 1 – 4.78


The importance of accuracy in RTI placement

Advanced math cut score (FSA Level 5)

Advanced math identification – Student score is at NWEA 82nd percentile, cut score is at the NWEA 87th Percentile (FSA Level 5 Cut Score)

Computer adaptive test with standard error of 2.9


The importance of accuracy in Gifted Placement

Advanced math cut score (FSA Level 5)

Advanced math identification – Student score is at NWEA 82nd percentile, cut score is at the NWEA 87th Percentile (FSA Level 5 Cut Score)

Non-adaptive test with standard error equivalent to FSA Level 5 – 7.21


The importance of accuracy in Gifted Placement

Fall Term Spring Term190

195

200

205

210

215

220

Standard error of math growth for student at FSA Level 1 on NWEA MAP

Red Lines = Error of growth based on FSA standard error at Level 1Black Lines = Error of growth based on NWEA MAP standard error at Level 1 equivalent.

Fall Term Spring Term220

225

230

235

240

245

250

Standard error of math growth for student at FSA Level 5 on NWEA MAP

Red Lines = Error of growth based on FSA standard error at Level 5Black Lines = Error of growth based on NWEA MAP standard error at Level 5 equivalent.


Support teacher improvement by tying professional development to learning in

the classroom

Key findings from research

If a teacher sets a high quality goal, it is associated with improved student achievement.

1

*Catalyst for change (2004), Community Training and Assistance Center, retrieved 10-2-13, http://www.ctacusa.com/wp-content/uploads/2013/11/CatalystForChange.pdf*It’s more than money (2013), Community Training and Assistance Center, retrieved 10-2-13, http://www.ctacusa.com/wp-content/uploads/2013/11/MoreThanMoney.pdf

http://www.ctacusa.com/wp-content/uploads/2013/11/CatalystForChange.pdf

http://www.ctacusa.com/wp-content/uploads/2013/11/MoreThanMoney.pdf

If a teacher sets and meets a goal, it is associated with increased student achievement.

2




Implementation of student learning objectives can make a 12-13% difference in the growth of student achievement.

3




The bad news

When used for higher stakes, teachers find SLO’s threatening and don’t trust goal setting processes.

Teacher responses to the Rhode Island SLO pilot

*Community Assistance and Training Center (2013, September) Focus on Rhode Island, Student Learning Objectives and Evaluation. http://www.ctacusa.com/wp-content/uploads/2013/11/FocusOnRhodeIsland.pdf

Continuous performance improvement is the core intent of SLOs in teacher evaluation

53%

20%

28%

Disagree Undecided Agree

http://www.ctacusa.com/wp-content/uploads/2013/11/FocusOnRhodeIsland.pdf



SLO’s provide a credible link between a teacher’s craft and learning outcomes. 77%

12%

10%





SLO’s improve student achievement outcomes

71%

17%

12%





SLO’s provide sound evidence for measuring teacher performance

85%

9%

6%



Addressing these problems

For most educators the focus is formative,

Goal setting should focus on opportunities to learn and improve and NOT on the outcome. When goals are achieved, educators gain confidence and become more efficacious.

For most educators the focus is formative,

Tie student learning goals to opportunities for teachers to learn and grow. These may include peer coaching and observation, professional development opportunities, and other strategies.

When the purpose is professional development

• The primary goal is to connect teacher professional development to the learning of their students.

• The primary purpose is to help teachers improve their practice, which may lead to improved results

• When done well, this process improves student learning.


Evaluate the performance of a school for purposes of retaining the principal

For a small minority of educators the focus is summative.

Goal setting challenges these educators to improve their performance and demonstrate their competence.

Every educator should know which category they are in!

In baseball, the goal of the general manager is to hire players who all hit .300. In reality, very few players meet this aspirational goal.

Instead, general managers rely on a statistic called WAR.

Players aren’t released because they don’t hit .300, they are released when a rookie or minor leaguer would perform better.

Because new baseball managers inherit their roster, their accountability increases as their control increases.

What happens when educators are held accountable to unreasonable improvement goals?

They cheat

Morale collapses

They focus on metrics rather than learning

Here’s an example

Serena is “one year” behind proficiency for her grade

1.5Her goal is to attain

years of growth this year.

10

9

8

7

6

5

4

3

2

1

K

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

45%

43%

39%

34%

31%

26%

21%

18%

16%

12%

9%

55%

57%

61%

66%

69%

74%

79%

82%

84%

88%

91%

Proportion of students meeting 1.5 years of growth in math-ematics

Met 1.5 years Did not meet

Students start at 50th percentile of 2015 norms

Student growth across a school year

Fall Winter Spring Next Fall130

150

170

190

210

230

250

Mathematics

K 1 2 3 4 5 6 7 8 9

Axis

Title

Student growth across a school year

Fall Winter Spring Next Fall

Reading

K 1 2 3 4 5 6 7 8 9

10

9

8

7

6

5

4

3

2

1

K

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

13%

9%

2%

1%

1%

1%

1%

1%

1%

1%

1%

87%

91%

98%

99%

99%

99%

99%

99%

99%

99%

99%

Proportion of schools averaging 1.5 years of growth in mathematics


Students start at 50th percentile of 2015 norms

10

9

8

7

6

5

4

3

2

1

K

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

44%

42%

38%

34%

30%

25%

21%

17%

14%

11%

6%

56%

58%

62%

66%

70%

75%

79%

83%

86%

89%

94%

Proportion of low performing students meeting 1.5 years of growth in mathematics


Students start at 20th percentile of 2015 school norms

10

9

8

7

6

5

4

3

2

1

K

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

9%

6%

1%

1%

1%

1%

1%

1%

1%

1%

1%

91%

94%

99%

99%

99%

99%

99%

99%

99%

99%

99%

Proportion of schools with low performing students averaging 1.5 years of growth in mathematics


Students start at 20th percentile of 2015 school norms

What constitutes a reasonable school improvement goal?

What makes a school improvement goal realistic?

• What is the past track record of the school?

• What is the range of improvement that is normal for schools like this?

• What is the level of control that the principal has over the conditions of improvement?

It’s important to have a “racing form” for student achievement

This spring, 80% of my third grade students will meet their growth projections according to their MAP

Reading scores.

Historical Growth: School Data• 2014 45% of the students in the class

met their growth projections• 2015 48% of the students in the class

met their growth projections

This spring, my students will show a median growth percentile of 65%

Historical Growth: School Data• 2014 The median growth percentile

was 57%• 2015 The median growth percentile

was 52%

Aspirational Goals

A goal that puts students on track to achieve their aspirations (college or

career)

When the purpose is aspirational

• The primary purpose of the goal is to clarify what level of performance is needed to attain the aspired result.

• The goal isn’t necessarily realistic, because it represents what is needed for success.

• The goal is never used summatively.

Ben wants to attend the University of Florida. What growth must he show to qualify?

https://www.nwea.org/research/innovation-leadership/research-data-galleries/mapping-the-road-to-college/

Address of NWEA College Explorer Tool

The goal – median ACT of the University of Georgiafreshman class

MAP Score = 260ACT Score = 29

Grade 7 Grade 8 Grade 9

244250

256

235

244

253

What makes an improvement goal realistic?

• What is the past track record of the school?• What is the range of improvement that is

normal for schools like this?• What is the level of control that the principal

has over the conditions of improvement?

The simple formula for setting school improvement goals

All students should be in play relative to the goal.1

Fifth grade math achievement in one school system

Mathematics

No Change Down Up

Fall RIT

Num

ber o

f Stu

dent

s

Mathematics

Below projected growth Met or above projected growth

Student’s score in fallNumber of 5th grade students meeting projected growth in the same district

Num

ber o

f Stu

dent

sFifth grade math achievement in one school system

The goal should always be improvement in a subject.

2

Thank you!E-mail: [email protected]

Organization Website: www.nwea.org

mailto:[email protected]

http://www.nwea.org/