benchmark screening: what, why and how

MN RtI Center1

Benchmark Screening: What, Why and How

A module for pre-service and in-service professional development

MN RTI CenterAuthor: Lisa H. Stewart, PhD

Minnesota State University Moorhead

www.scred.k12.mn.us click on RTI Center

MN RtI Center2

MN RTI Center Training Modules

This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center

for use in preservice and inservice training:

Module Title Authors

1. RTI Overview Kim Gibbons & Lisa Stewart

2. Measurement and RTI Overview Lisa Stewart

3. Curriculum Based Measurement and RTI Lisa Stewart

4. Universal Screening (Benchmarking): (Two parts)

What, Why and How

Using Screening Data

Lisa Stewart

5. Progress Monitoring: (Two parts)

What, Why and How

Using Progress Monitoring Data

Lisa Stewart & Adam Christ

6. Evidence-Based Practices Ann Casey

7. Problem Solving in RTI Kerry Bollman

8. Differentiated Instruction Peggy Ballard

9. Tiered Service Delivery and Instruction Wendy Robinson

10. Leadership and RTI Jane Thompson & Ann Casey

11. Family involvement and RTI Amy Reschly

12. Five Areas of Reading Kerry Bollman

13. Schoolwide Organization Kim Gibbons

MN RtI Center3

Overview

This module is Part 1 of 2

Module 1: Benchmark Screening: What, Why and How What is screening? Why screen students? Criteria for screeners/what tools? Screening logistics

Module 2: Using Benchmark Screening Data

MN RtI Center4

Adapted from Logan City School District, 2002

Curriculum and Instruction Assessment

School Wide Organization &

Problem Solving Systems

(Teams, Process, etc)

Assessment: One of the Key Components in RTI

MN RtI Center5

Assessment and Response to Intervention (RTI)

A core feature of RTI is identifying a measurement system Screen large numbers of students

Identify students in need of additional intervention

Monitor students of concern more frequently 1 to 4x per month

Typically weekly Diagnostic testing used for instructional planning to

help target interventions as needed

MN RtI Center6

Why Do Screening?

Activity

What does it mean to “screen” students?

Why is screening so important in a Response to Intervention system? (e.g., what assumptions of RTI require a good screening system?)

What happens if you do NOT have an efficient, systematic screening system in place in the school?

MN RtI Center7

Screening is part of a problem-solving system

Helps identify students at-risk in a PROACTIVE way Gives feedback to the system about how students progress

throughout the year at a gross (3x per year) level If students are on track in the fall are they still on track in the

winter? What is happening with students who started the year below

target, are they catching up? Gives feedback to the system about changes from year to

year Is our new reading curriculum having the impact we were

expecting?

MN RtI Center8

What Screening Looks Like in a Nutshell

School decides on brief tests to be given at each grade level and trains staff in the administration, scoring and use of the data

Students are given the tests 3x per year (Fall, Winter, Spring) Person or team assigned in each building to organize data collection All students are given the tests for their grade level within a short time

frame (e.g., 1-2 weeks or less). Some tests may be group administered, others are individually administered. Benchmark testing: about 5 minutes per student, desk to test

(individually administered) Administered by special ed, reading, or general ed teachers or paras

Entered into a computer/web based reporting system by clerical staff Reports show the spread of student skills and lists student scores, etc.

to use in instructional and resource planning

MN RtI CenterDRAFT May 27, 2009 9

Example Screening Data:Spring Gr 1 Oral Reading Fluency 10/51 (20%) high risk 22/51 (43%) some risk 19/51 (37%) low risk: on or above target

Class lists then identify specific students (and scores) in each category


Screening Data Gives an idea of what the range of student

skills is like in your building and how much growth over time students are making

MN RtI Center11

Screening Data can be linked to Progress Monitoring

The goal is to have a cohesive system.

If possible, use the same measures for both screening and progress monitoring (e.g, CBM).

Screen ALL students 3x per year (F, W, S)

Strategic Support and MonitoringStudents at Some Risk

Intensive Support & Monitoring for

Students at Extreme Risk

MN RtI Center

12

Academic Systems Behavioral Systems

5-10% 5-10%

10-15% 10-15%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity•Of longer duration

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

75-85% 75-85%Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

A Smart System StructureSchool-Wide Systems for Student Success

MN RtI Center13

Terminology Check Screening

Collecting data on all or a targeted group of students in a grade level or in the school

Universal Screening Same as above but implies that all students are screened

Benchmarking Often used synonymously with the terms above, but

typically implies universal screening done 3x per year and data are interpreted using criterion target or “benchmark” scores

MN RtI Center14

“Benchmark” Screening

Schools typically use cut off or criterion scores to decide if a student is at-risk or not. Those scores or targets are also referred to as “benchmarks”, thus the term “benchmarking”

Some states or published curriculum also use the term benchmarking but in a different way (e.g., to refer to the documentation of achieving a specific state standard) that has nothing to do with screening.

MN RtI Center

What to Measure for Screening?Create a “Measurement Net”:

Beginning of Year Middle of Year End of Year Grade Reading Math Writing Reading Math Writing Reading Math Writing K Letter Names

Rhyming Letter Sounds

Test of Early Numeracy

--- Letter Names Rhyming Phoneme

Segment Nonsense

Words


--- Letter Names Rhyming Phoneme

Segment Nonsense

Words


---

1 Letter Names Phoneme

Segment Nonsense

Words Word Use


Math CBM

-- Phoneme Segment

Nonsense Words

Word Use Oral Reading


Math CBM

Written Expression CBM

Phoneme Segment

Nonsense Words


Math CBM


2 Nonsense Words


Math CBM


Oral Reading Word Use

Math CBM


Oral Reading

Word Use

Math CBM


3 Oral Reading

Math CBM


Oral Reading

Math CBM


Oral Reading

Math CBM


4 Oral Reading

Math CBM


Oral Reading

Math CBM


Oral Reading

Math CBM


5 Reading Maze

Math CBM


Reading Maze

Math CBM


Reading Maze

Math CBM


6 Reading Maze

Math CBM


Reading Maze

Math CBM


Reading Maze

Math CBM


Note. All measures are fluency measures using standardized administration and scoring. Reading measures are a combination of CBM, DIBELS and IGDIs measures

MN RtI Center16

How do you decide what Measures to Use for Screening?

Lots of ways to measure reading in the schools: Measure of Academic Progress (MAP) Guided Reading (Leveled Reading) Statewide Accountability Tests Published Curriculum Tests Teacher Made Tests General Outcome Measures (Curriculum-Based Measurement “family”) STAR Reading Etc

Not all of these are appropriate. Some are not reliable enough for screening, others are designed for another purpose and are not valid or practical for screening all students 3x per year

MN RtI Center

17

valid

reliable

simple

quick

inexpensive

easily understood

can be given often

sensitive to growth over short periods of time

Characteristics of An Effective Measurement System for RTI

Credit: K Gibbons, M Shinn

MN RtI Center18

Effective Screening Measures Specific

Identifies at risk students who really are at risk

Sensitive Students who “pass” really do go on to do well

Practical Brief and simple (cheap is nice too)

Do no harm If a student is identified as at risk will they get help or is it

just a label?

Reference: Hughes & Dexter, RTI Action Network

MN RtI Center19

Buyer Beware!

Many tools may make claims about being a good “screener”

MN RtI Center

Measurement and RTI: Screening

Reliability coefficients of at least r =.80. Higher is better, especially for screening specificity.

Well documented predictive validity Evidence the criterion (cut score) being used is

reasonable and creates not too many false positives (students identified as at risk who aren’t) or false negatives (students who are at risk who aren’t identified as such)

Brief, easy to use, affordable, and results/reports are accessible almost immediately

MN RtI Center

21

National Center for RTI Review of Screening Tools

Note: Only reviews tests submitted, if it is not on the list it doesn’t mean it is bad, just that it wasn’t reviewed

www.rti4success.org

MN RtI Center22

RTI, General Outcome Measures and Curriculum Based Measurement

Many schools use Curriculum Based Measurement (CBM) general outcome measures for screening and progress monitoring You don’t “have to “ use CBM, but many schools do

Most common CBM tool in Grades 1- 8 is Oral Reading Fluency (ORF) Measure of reading rate (# of words correct per minute on a grade

level passage) and a strong indicator of overall reading skill, including comprehension

Early Literacy Measures are also available such as Nonsense Word Fluency (NWF), Phoneme Segmentation Fluency (PSF), Letter Name Fluency (LNF) and Letter Sound Fluency (LSF)

MN RtI Center23

Why GOMs/CBM? Typically meet the criteria needed for RTI screening and

progress monitoring Reliable, valid, specific, sensitive, practical Also, some utility for instructional planning (e.g., grouping)

They are INDICATORS of whether there might be a problem, not diagnostic! Like taking your temperature or sticking a toothpick into a cake Oral reading fluency is a great INDICATOR of reading decoding,

fluency and reading comprehension Fluency based because automaticity helps discriminate between

students at different points of learning a skill


GOM…CBM… DIBELS… AIMSweb…

MN RtI Center25

CBM Oral Reading Fluency Give 3 grade-level passages using standardized admin and

scoring; use median (middle) score 3-second rule (Tell the student the word & point to next word) Discontinue rule (after 0 correct in first row, if <10 correct on 1st

passage do not give other passages)

Errors Not Errors

Hesitation for >3 seconds

Incorrect pronunciation for context

Omitted Words

Words out of order

Repeated Sounds

Self-Corrects

Skipped Row

Insertions

Dialect/Articulation

MN RtI Center

Fluency and Comprehension

A good measures of overall reading proficiency is reading fluency because of its strong correlation to measures

of comprehension.

The purpose of reading is comprehension

MN RtI Center27

Screening Logistics

What materials?

When to collect?

Who collects it?

How to enter and report the data?

MN RtI Center28

What Materials? Use computer or PDA-based testing system

-OR- Download reading passages, early literacy

probes, etc. from the internet Many sources of CBM materials available free or low

cost: Aimsweb, DIBELS, edcheckup, etc. Often organized as “booklets” for ease of use Can use plastic cover and markers for scoring to save

copy costs

MN RtI Center29

Screening Materials in K and Gr 1

Screening Measures will change from Fall to Winter to Spring slightly

Early literacy “subskill” measurement is dropped as reading develops

Downloaded materials and booklets

MN RtI Center

30

K and Gr 1 MeasuresAIMSweb Early Literacy and R-CBM(ORF)

Kindergarten Grade 1 Fall Winter Spring Fall Winter Spring

Letter Naming

Letter Naming

Letter Naming

Letter Naming

Letter Sounds

Letter Sounds

Letter Sounds

Letter Sounds

Nonsense Words

Nonsense Words

Nonsense Words

Nonsense Words

Nonsense Words

R-CBM R-CBM Rhyming

Alliteration Phoneme

Segmentation Phoneme




Segmentation Picture Naming

Picture Naming

Picture Naming

Word Use Fluency

(optional)

Word Use Fluency

(optional)

Word Use Fluency

(optional)

General Literacy Risk Factor= Black, Alphabetic Principle = GreenPhonemic Awareness = Purple, Vocabulary = BlueFluency with Connected Text & Comprehension= Red

MN RtI Center

31

Gr 2 to 12: AIMSweb Early Literacy and CBM Measures

Grade 2 Grade 3 Fall Winter Spring Fall Winter Spring

Nonsense Word

Fluency

R-CBM R-CBM R-CBM R-CBM R-CBM R-CBM R-Maze

(optional) R-Maze

(optional) R-Maze

(optional) R-Maze

(optional) R-Maze

(optional) R-Maze

(optional) Word Use Fluency

(optional)

Word Use Fluency

(optional)

Word Use Fluency

(optional)

Word Use Fluency

(optional)

Word Use Fluency

(optional)

Word Use Fluency

(optional)

Grade 4-12+ Fall Winter Spring

R-CBM R-CBM R-CBM R-Maze

(optional) R-Maze

(optional) R-Maze

(optional)

MN RtI Center32

Screening Logistics: Timing Typically 3x per year: Fall, Winter, Spring

Have a district-wide testing window! (all grades and schools collect data within the same 2 week period)

In Fall K sometimes either test right away and again a month later or wait a little while to test

Benchmark testing: about 5 minutes per student (individually administered) In the classroom In stations in a commons area, lunchroom, etc.

MN RtI Center33

Screening Logistics: People Administered by trained staff

paras, special ed teachers, reading teachers, general ed teachers, school psychologists, speech language, etc.

Good training is essential!

Measurement person assigned in each building to organize data collection Either collected electronically or entered into a web-

based data management tool by clerical staff

MN RtI Center34

Screening Logistics Math Quiz

If you have a classroom with 25 students and to administer the screening measures takes approx. 5 min. per student (individual assessment time)…

How long would it take 5 people to “screen” the entire classroom?

MN RtI Center35

Remember: Garbage IN…. Garbage OUT….

Make sure your data are reliable and valid indicators or they won’t be good for nuthin… Training Assessment Integrity

checks/refreshers Well chosen tasks/indicators

MN RtI Center36

Use Technology to Facilitate Screening

MN RtI Center37

Using Technology to Capture Data Collect the data using technology such as a PDA

Example: http://www.wirelessgeneration.com/

http://www.aimsweb.com

Students take the test on a computer

Example: STAR Reading

http://www.renlearn.com/sr/

MN RtI Center38

Using Technology to Organize and Report Data Enter data into web-based data management system

Data gets back into the hands of the teachers and teams quickly and in meaningful reports for problem solving

Examples http://dibels.uoregon.edu http://www.aimsweb.com http://www.edcheckup.com

MN RtI Center

39

Screening is just one part of an overall assessment system for making decisions

*Decision-Tree for Screening, Instructional Decision-Making, & Progress Monitoring with DIBELS

*Note: The concept and content of this model was provided by Dr. Lisa Stewart of MSUM

Did the student meet or exceed the Low Risk/Benchmark goals on the most recent DIBELS testing?

YES

Next progress check is regularly scheduled DIBELS testing for all students

NO

Did the student fall into the "Some risk" category or the "At-risk" category? An intervention plan may be needed.

Somerisk

Do other data (e.g., OS, BMRR, DRA) indicate some concern?

NO

Make sure a good curricula is in place in the classroom and consider monitoring monthly.

Put the student in strategic instruction (e.g., small group with supplemental curricula). Be SURE TO CONTINUE TO USE DATA to make changes as needed. Monitor monthly.

YES

Atrisk

Do other data (e.g., OS, BMRR, DRA) indicate high level of concern? (important here to get good info)

Put the student in intensive instruction (e.g., 1:1 or very small group with supplemental and direct instruction curricula). Be SURE TO CONTINUE TO USE DATA to make changes as needed. Monitor weekly!

YESNO

MN RtI Center40

Remember: Screening is part of a problem-solving system

Helps identify students at-risk in a PROACTIVE way Gives feedback to the system about how students progress

throughout the year at a gross (3x per year) level If students are on track in the fall are they still on track in the

winter? What is happening with students who started the year below

target, are they catching up? Gives feedback to the system about changes from year to

year Is our new reading curriculum having the impact we were

expecting?

MN RtI Center41

Build in Time to USE the Data!

Schedule data “retreats” or grade level meeting times immediately after screening so you can look at and USE the data for planning.

MN RtI Center42

Common Mistakes Not enough professional development and communication

about why these measures were picked, what the scores do and don’t mean, the rationale for screening, etc

Low or questionable quality of administration and scoring Too much reliance on a small group of people for data

collection Teaching to the test Limited sample of students tested (e.g., only Title students! )

Slow turn around on reports Data are not used

MN RtI Center43

Using Screening Data: See Module 2!

MN RtI Center44

Articles available with this module Stewart & Silberglitt. (2008). Best practices in developing academic local

norms. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP Publications.(pp. 225-242).

NCRLD RTI Manual (2006). Chapter 1: School-wide screening Retrieved from http://www.nrcld.org/rti_manual/pages/RTIManualSection1.pdf 6/26/09

Jenkins & Johnson. Universal screening for reading problems: Why and how should we do this? Retrieved 6/23/09, from RTI Action Network site: http://www.rtinetwork.org/Essential/Assessment/Universal/ar/ReadingProblems

Kovaleski & Pederson (2008) Best practices in data analysis teaming. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP

Ikeda, Neessen, & Witt. (2008). Best practices in universal screening. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP Publications.(pp. 103-114).

Gibbons, K (2008). Necessary Assessments in RTI. Retrieved from http://www.tqsource.org/forum/documents/GibbonsPaper.doc on 6/26/09

MN RtI Center45

RTI Related Resources National Center on RTI

http://www.rti4success.org/

RTI Action Network – links for Assessment and Universal Screening http://www.rtinetwork.org

MN RTI Center http://www.scred.k12.mn.us/ and click on link

National Center on Student Progress Monitoring http://www.studentprogress.org/

Research Institute on Progress Monitoring http://progressmonitoring.net/

MN RtI Center46

RTI Related Resources (Cont’d) National Association of School Psychologists

www.nasponline.org

National Association of State Directors of Special Education (NADSE) www.nasdse.org

Council of Administrators of Special Education www.casecec.org

Office of Special Education Programs (OSEP) toolkit and RTI materials http://www.osepideasthatwork.org/toolkit/

ta_responsiveness_intervention.asp

MN RtI Center47

Key Sources for Reading Research, Assessment and Intervention…

University of Oregon IDEA (Institute for the Development of Educational Achievement) Big Ideas of Reading Site http://reading.uoregon.edu/

Florida Center for Reading Research http://www.fcrr.org/

Texas Vaughn Gross Center for Reading and Language Arts http://www.texasreading.org/utcrla/

American Federation of Teachers Reading resources (what works 1999 publications) http://www.aft.org/teachers/pubs-reports/index.htm#reading

National Reading Panel http://www.nationalreadingpanel.org/

MN RtI Center48

Recommended Sites with Multiple Resources

Intervention Central- by Jim Wright (school psych from central NY) http://www.interventioncentral.org

Center on Instruction http://www.centeroninstruction.org

St. Croix River Education District http://scred.k12.mn.us

MN RtI Center49

Quiz

1.) A core feature of RTI is identifying a(n) _________ system.

2.) Collecting data on all or a targeted group of students in a grade level or in the school is called what? A.) Curriculum B.) Screening C.) Intervention D.) Review

MN RtI Center50

Quiz (Cont’d)

3.) What is a characteristic of an efficient measurement system for RTI? A.) Valid B.) Reliable C.) Simple D.) Quick E.) All of the above

MN RtI Center51

Quiz (Cont’d)

4) Why screen students?

5) Why would general education teachers need to be trained on the measures used if they aren’t part of the data collection?

MN RtI Center

Quiz (Cont’d)

6) True or False? If possible the same tools should be used for screening and progress monitoring.

7.) List at least 3 common mistakes when doing screening and how they can be avoided.

MN RtI Center

The End

Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only.

Special Thanks: Thank you to Dr. Ann Casey, director of the MN RTI Center, for

her leadership Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan

Rowe, Minnesota State University Moorhead graduate students, for editing work, writing quizzes, and enhancing the quality of these training materials

benchmark screening: what, why and how

Documents