powerful protocols for systematic and effective assessment

Powerful Protocols for Systemat ic and Ef fect ive Assessment Analysis

Presented by: Katherine Pat tonLearning Forward Annual ConferenceDecember 6, 2017

Developing common language and a systematic process in data teams provides guidance for educators in making efficient and effective decisions w ith literacy data.

Burns, M. K., Riley-Tillman, T. C., & VanDerheyden, A. M. 2012

Engage with protocols to analyze literacy data systematically, efficiently, and effectively.

Develop common language and a systematic process to enhance educators’ collaborative decision-making.

Participate actively as you reflect on impact stories from the field and as you work through how, when, and where to

implement protocols to improve student proficiency.

Apply the protocols demonstrated in this session immediately into your elementary school setting.

Objectives

Agenda ● Research● Screening Data Protocol

○ Model, Practice & Reflect● Progress Monitoring Protocol

○ Model, Practice & Reflect● Problem Solving Protocol

○ Model & Reflect● Application and Final Reflection

I have found that when teams have a set structure, useful forms, and commonly understood operating procedures, they are effective and team members display great satisfaction with the process.

Conversely, the lack of structure, poorly designed forms, and uncertain

procedures breed ineffectiveness and dissatisfaction (or even annoyance) with

the team process. Kovaleski, 2016

Why are systems of support needed?

2015 National Assessment of Education Progress Results (NAEP)National Center for Education Statistics, 2015

Response t o Intervent ion – Rt I

Mult i-Tiered Systems of Support – MTSS

Embedded Ongoing Professional

Learning

Data-Driven Decision Making and Dynamic

Grouping

Tiered SupplementalInstructional

Support

Coyne, et. al, 2016

Although there is w idespread agreement about the practices

associated w ith multitieredsystems of support (MTSS)

frameworks in beginning reading,

we often underestimate the systems and infrastructures

that schools need to implement and sustain these practices.

Screening Data ProtocolIt is not enough to do your best; you must know what to do, and then do your best.W . Edwards Deming

Leading experts agree, you cannot “intervene” your way out of inadequate achievement w ith students .

Minnesota Center for Reading Researchwww.cehd.umn.edu/reading/PRESS

Tier 3What is the causal variable for

an individual student?

Tier 2 What is the category of the

problem for individual students?

Tier 1 Is there a classwide need?

Problem analysis is central to the RTI process and should occur at all three tiers. Christ, Burns, & Ysseldyke, 2005

Protocols give educators a

powerful set of tools to guide

and share conversations.

Foltos, 2013

Decision Making

Implement Intervention

Data Collect ion

Protocols

Analysis to Action

Progress Monitoring W orksheet

Tier 2 Problem Solving

Analyze Screening

Data

Analyze Ongoing PM Data

Analyze Fidelity and

PM Data


Tier 1 Is there a classwide

need?

If you or your school has a larger number of students identified at risk, you are best served to first put significant time, attention, and energy into the instructional core.Lesaux and Marietta, 2012

Protocols

Analysis to Action



Analyze Screening

Data



PM Data

Fall Benchmark 65Student Grade WRC Errors

F 2 18 2 E 2 21 1B 2 22 5K 2 26 4Q 2 32 6R 2 35 2N 2 46 8S 2 51 1M 2 54 0G 2 60 0A 2 64 5D 2 68 4H 2 70 2O 2 70 3T 2 71 1P 2 75 0C 2 77 0J 2 77 0I 2 84 0L 2 89 1Class Median

65Sept. Ms. D ORF 62

Is a classwide intervention needed?

“Tell me and I forget, teach me and I may remember, involve me and I learn.” Benjamin Franklin

91

Sept. Mr. B ORF 94

Students Below Benchmark at

Screening

Students Below Benchmark after

Classwide InterventionThird Grade Class 1 (20 students) 10 5Third Grade Class 2 (23 students) 13 5

Classwide Impact


Tier 2 What is the

category of the problem for

individual students?

(One) reason that students might not perform a task

sufficiently is that they lack prerequisite skills for

completing the task. This difficulty is often referred to

as a skill deficit.Hosp, J. L., & Ardoin, S. P. 2008

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Accuracy Rate

WEEKS

Fluency Intervention Decoding Intervention/s Fluency Intervention

Reading Comprehension

Code Based Skills

Meaning Based Skills

Let ter ident if icat ion

Phonics

FluencyVocabulary

Comprehension

Writ ing

Lesaux and Marietta, 2012

Research suggests that using assessment for learning supports educators in their decision making process to determine individual student needs.

91Sept. Mr. B ORF 94

Student A

59 91% 169 Decoding Decoding InterventionStudent S 75 89% 179 Decoding Decoding InterventionStudent A

Student DStudent NStudent M

Student T88 96% 189 Fluency Fluency Intervention

91 97% 195 Comprehension Comp. Intervention

Student A 59 91% 169 Decoding Decoding InterventionStudent S 75 89% 179 Decoding Decoding InterventionStudent T 83 90% 185 Decoding Decoding InterventionStudent D 88 96% 189 Fluency Fluency InterventionStudent N 90 98% 195 Fluency Fluency InterventionStudent M 91 97% 195 Comprehension Comp. InterventionStudent Q 91 96% 197 Comprehension Comp. Intervention

Protocols

Analysis to Action



This protocol could…

Progress Monit or ing ProtocolIt is not enough to do your best; you must know what to do, and then do your best.W . Edwards Deming

RtI involves:

interventions match to

student need

monitoring progress

frequently

Batsche, et al., 2006

providing high

quality instruction

Protocols

Analysis to Action



Analyze Screening

Data



PM Data

General Outcome MeasuresHow are the students progressing at grade level?

Kindergarten LSF

First Grade NW F

2nd GradeORF

3rd GradeORF

4th GradeORF

5th GradeORF

Skill MeasuresHow are the students progressing at

their instructional level?

phoneme manipulat ion? letter sounds? decoding ccvc

words?

3rd grade instruct ional level text?

comprehension skills?

Track skill growth Track GOM growth

52

90

benc

hmar

ks

Spring – Fall = Total Growth 90 - 52 = 38

GOM Growth

= Expected Growth1.15 words per week

Total Growth ÷ # of weeks38 ÷ 33 = 1.15

Student Sarai Teacher Mr. R

Week of Skill Type Skill Scorescorrect/errors

AccuracyGoal 90%

GOM Type GOM Scorecorrect/errors

AccuracyGoal 93%

Fall Benchmark Target: 32

SeptemberBenchmark Median 10/10 50%

September

September LSF 1 21/8 72% NWF 1 10/8 56%

October LSF 2 28/5 85%

October LSF 3 32/3 91% NWF 2 15/7 68%

October CVC 1 1/7 13%

October CVC 2 3/6 33% NWF 3 17/8 68%

November CVC 3 5/5 50%

November CVC 4 9/6 60% NWF 4 22/7 76%


Proficiency:Being proficient means having a

thorough competence derived from training and practice.

www.merriam-webster.com

90% is probably the most appropriate accuracy criterion for these tasks. (letter sounds, sight words, spelling, etc...)Burns, 2004

Student Sarai Teacher Mr. RWeek of Skill Type Skill Scores

correct/errorsAccuracyGoal 90%


AccuracyGoal 93%


September Benchmark Median 10/10 50%September

September LSF 1 21/8 72% NWF 1 10/8 56%October LSF 2 28/5 85%October LSF 3 32/3 91% NWF 2 15/7 68%


October CVC 2 3/6 33% NWF 3 17/8 68%November CVC 3 5/5 50%November CVC 4 9/6 60% NWF 4 22/7 76%November CVC 5 14/4 78%

Sarai CVC words

5/5

50%

9/6

60%

14/4

78%

22/7

76%




AccuracyGoal 93%


September Benchmark Median 10/10 50%

September September LSF 1 21/8 72% NWF 1 10/8 56%

October LSF 2 28/5 85%

October LSF 3 32/3 91% NWF 2 15/7 68%


October CVC 2 3/6 33% NWF 3 17/8 68%


November CVC 4 9/6 60% NWF 4 22/7 76%


22 - 10 = 12

12 ÷ 8 = 1.5 Calculating individual student’s growth




AccuracyGoal 93%






Sarai CVC words

5/5

50%

9/6

60%

14/4

78%

22/7

76%

Works hard in intervention, becoming more automatic

1.5




AccuracyGoal 93%






Sarai CVC words

5/5

50%

9/6

60%

14/4

78%

22/7

76%

Works hard in intervention, becoming more automatic

1.5x

K ISM3/7

30%

6/4

60%16

Attendance consistent, motivated

1.0x

3rd Vowel teams

42/8

84%

50/6

89%

90/5

95%

Consistent with vowel teams

1.58x

Int. Teacher December 15

25 110

1.00.93

55/3

95%

Protocols

Analysis to Action

Progress Monitoring Worksheet



Problem Solv ing ProtocolIt is not enough to do your best; you must know what to do, and then do your best.W. Edwards Deming

Protocols

Analysis to Action



Analyze Screening

Data



PM Data

Why do intervent ions fail?

incorrectly targeted

not correctly implemented

not effectively managed w ith ongoing progress monitoring and

troubleshootingBurns, M. K., Riley-Tillman, T. C., & VanDerHeyden, A. M., 2012

Acq

uisi

tion

Is the student acquiring the

skills?

Risa Blends 7/13

35%

7/15

32%

5/10

33%

36/14

72%

Works hard in the intervention, but blends have been a challenge.

0.71

Fide

lity Has the

intervention been implemented with

fidelity?

Cons

iste

ncy

• Length?• Attendance?• Time?

Beha

vior

• Difficulties?• Motivation?

Nex

t Ste

ps • Modifications• Action Plan • Action Plan Review

Protocols

Analysis to Action

Progress Monitoring Worksheet



Ref lect ion

IMPACT

Learning

Protocols

Thank you!Kathy Patton email: [email protected]

Minnesota Center for Reading Research: www.cehd.umn.edu/reading/

Path to Reading Excellence in School Sites: www.PRESScommunity.org

powerful protocols for systematic and effective assessment

Documents