teaming/data/interventions rti infrastructure: data rti partnership coaches meeting january 6, 2011...
TRANSCRIPT
Teaming/Data/Interventions
RtI Infrastructure:Data
RtI Partnership Coaches meetingJanuary 6, 2011
Terry Schuster, RtI Partnership Lead Coach
Key Features:•4 Purposes of Assessment•Knowing if it’s Scientifically Based•Building a Common Data System•Importance of Tools, Training, and Support
Integrated Assessment Systems
Assessment
Instruction
Assessment Instruction
Assessment FOR Instruction..
Not this
Schools Use Specific Tools for Specific Assessment Purposes
Type Feature Example
Universal Screening Reliable, Valid, Low Cost, Accurate, Production Type Responses, Sensitive to Between Persons Differences
CBM Family Members; Common Assessments; ODRs
Diagnostic Lots of Items, Production-Type Responses
Placement Tests; Curriculum Assessments; Can’t Do/Won’t Do; Curriculum-Based Evaluation thought process
Progress Monitoring Reliable, Valid, Low Cost, Accurate, Production Type Responses, REPEATABLE, Sensitive to Within Persons
Differences
CBM Family Members; Common Formative Assessments
Program Evaluation Linked to Important Outcomes
ISAT; CBM; Common Assessments; GPA; ODRs
Linking Screening and Progress Monitoring Assessment Across Tiers
Academic Basic Skills
Tier 1:
Universal Screening / Benchmarking
Tier 2:
Strategic Monitoring
Tier 3:
Frequent Progress Monitoring
Material Generalized Curriculum CBM probes
Generalized Curriculum CBM probes
Generalized Curriculum CBM probes
Frequency 3 x per year 2 x per month 1-2 x per week
Common Assessments
• Definition• Rationale• Formative / Summative
What Would Be The Consequences of Allocating Services Based on This Triangle?
• Inefficient• Spreads supplemental & intensive resources too thin• Makes specialized services less special
Or THIS Triangle?
Images and Analyses Courtesy of Ben Ditkowsky, [email protected]
Triangles are About Program Evaluation
Schools Use CBM in Universal Screening Instead of Referral Driven Practices
• Local – school or district• Think box plots or %-ile charts
e.g., < 25thTier 2 Candidates
e.g., <10thIndividual Problem Solving and/or Tier 3 Candidates
Progress Monitoring• Survey Level
Assessment– Too often skipped
• Goal Setting– Should NOT be set in
Instructional Level material
– Criterion for success should at least begin to close the gap with peers
R-CBM: Indian Prairie Average Range of Performance
9
27
58 58
85
104
82
106
121
101
119
132
113
133
150
61
94
121
112
138
154
138
155
171
148
168
182
167
182
199
0
20
40
60
80
100
120
140
160
180
200
220
Fall
Wint
er
Spring Fall
Wint
er
Spring Fall
Wint
er
Spring Fall
Wint
er
Spring Fall
Wint
er
Spring
First (ROI = 1.9) Second (ROI = 1.3) Third (ROI = 1.0) Fourth (ROI = .9) Fifth (ROI = 1.0)
# o
f WR
CSurvey Level Assessment: Test backward using successively lower grade level probes until the student's score falls within the green…Choose at least one grade level above for goal writing and progress monitoring… Use a value greater than the specified average ROI to begin to close the gap. Goal = SLA Score in Goal Material + (Goal Material ROI) (# of weeks)
Set goal & monitorAt 4th or 5th gr.
DiagnosticsIt’s not just about one measure . . .
• Classroom performance– Observations– Independent work
• District assessments/ ISAT• Curriculum assessments
– Pre-tests; Placement tests– Chapter & Unit tests– Curriculum skill/indicator checklists
• Skill Deficit versus Performance Deficit• When Tier 3 & still need more info - CBE
It’s not always about one student . . .
Diagnostic Assessment
• Needs to connect with instruction; tells teachers what needs to be taught
• Skills, not scores
• The process is more important than the “test”– Adequate number of items per skill– Error analysis
What happens when these data systems aren’t in place?
• Lack of direction for knowing what needs to be improved
• Don’t know which tier and students need additional support/intervention
• Don’t know if change efforts have been successful
Key Questions
• What assessments do you currently use for each purpose in your district/building? • Do they each meet the requirements for that purpose? • Do you have too many assessments for a particular purpose? Too few?• Tools? Training? Support?
Assessment Worksheet
Type Assessments Currentlyin Place
Meets Criteria for Purpose(comments)
Action Plan
Universal Screening
Frequency?
Rigby/Fountas & Pinnell Leveling 3x/yrISEL 2x/yrGates 3rd -8th 1x/yrR-CBM 1st-5th 3x/yr
EL (all 4) K-1st 3x/yrMaze 6th-8th 3x/yr
NoNoNoYes
NoNo
Move to Diagnostic tool – not all students 3x/yrMove to Diagnostic tool – not all students 3x/yrMove to Diagnostic tool – not all students Keep as Universal Screener – firm up US use in target schools - separate US from Program EvalEstablish a multi-gate systemEither use as true universal screener or drop – consider using R-CBM on bottom 25%
Diagnostic/Intervention Planning
WIATWJ-IIIDRAIRI’sWADE (Wilson placement test)Horizons placement test
Add Rigby/Fountas & Pinnell Leveling , ISEL, and Gates Add regular curriculum assessments – need to decide this at a district, or at least a school levelAdd assessments for determining Skill vs Performance deficitsAdd some CBE-type skills
Progress Monitoring
Frequency?
CBM (all types) Tier 2 2x/mo Tier 3 1x/wk
Yes
Continue to provide coaching support to schools/staff with low compliance on monitoring;Decrease Maze monitoring;Decrease # of students being monitored on more than one CBM measure
Program Evaluation
ISAT 1x/yrCBM – ISAT correlate triangles 3x/yr
YesYes
ContinueContinue, but separate from US decisions
Example
R-CBM
Ongoing for new staff; annual AIRS; sd/coaching during grade level data mtgs
3x per year
R-CBM
yes yes
Ongoing for new staff; annual AIRS; sd/coaching during data review mtgs
2x /mo for sts in Tier 21x/wk for sts in Tier 3
‘15 hour’ grade level data mtgs
4-6 wk data review mtgs – all sts being monitored
School level percentiles
District ISAT cut scores into triangles
R-CBM
yes
2x per year
SIP team &‘15 hour’ grade level data mtgs
sd/coaching during SIP & grade level mtgs
Key Features
• 4 Purposes of Assessment
• Knowing if it’s Scientifically Based
• Building a Common Data System
• Importance of Tools, Training, and Support
Assessment Worksheet
Type Assessments Currentlyin Place
Meets Criteria for Purpose
(comments)
Action Plan
Universal Screening
Diagnostic/Intervention Planning
Progress Monitoring
Program Evaluation