david tudor ospi special education program supervisor – learning improvement, institutional...

David TudorOSPI Special EducationProgram Supervisor – Learning Improvement, Institutional Education Mini-ConferenceMarch 20, 2008

• Discuss different types of assessments and structure of an assessment system

• Evaluate data to make program/instructional decisions

• Model effective reading and mathematics strategies

• Assessment – process of collecting a behavior sample (data).

• Evaluation – process of using information to make a reasoned decision.

Howell & Nolet (2000). Curriculum Based Evaluation (3rd edition). Belmont, CA: Wadsworth/Thomas Learning

1. Define your purpose2. Define the thing to be measured3. Devise a way to make the

measured item observable4. Conduct assessment5. Evaluate results6. Summarize7. Make decisions

• Assessments are an essential element of education used to inform instruction. (Wren 2000)

• Assessments provide teachers with information needed to develop appropriate lessons and improve instruction for all students, including students with disabilities (Rhodes and Shanklin,1993)

• Frequent assessment of developing readers, and the use of that information for planning instruction, is the most reliable way of preventing children from falling behind and staying behind (Learning First Alliance 2000, Torgesen 1998 )

• In order to meet the higher expectations of current standards-based systems, educators need information that can be used to project how students are doing against the grade-level standards throughout the course of the year so they can determine what needs to be done to accelerate student progress toward proficiency standards. Progress monitoring techniques can provide that information (Quenomoen,R., Thurlow,M., Moen, R., Thompson ,S. &Morse, A.B. 2003)

• Educators in effective schools consistently use reading assessments that inform and shape the instructional programs designed to help students meet established benchmarks ( Denton, Foorman, et al., 2003)

• Reliable: A reliable measure is one that produces dependable scores. Scores are similar no matter who tests the student. Assessments close in time yield similar results. An assessment is determined to be reliable if there is a high correlation between the scores on the two times the student was tested

• Valid: A valid measure is an assessment that measures the trait or ability that it says it will measure

• Fidelity refers to the degree to which components are implemented as designed, intended, and planned.

• Fidelity is vital in universal screening, instructional delivery and progress monitoring.

• Formative: On-going assessments, reviews, observations in a classroom. Teachers use formative assessments to improve instructional methods and student feed back throughout the teaching and learning process.

• Summative: Used to evaluate the effectiveness of instructional programs and services at the end of an academic year or at a predetermined time. Measure student competency after an instructional phase is complete (example-- WASL). Summative evaluations are used to determine if students have mastered specific competencies and identify instructional areas where additional help is needed.(Florida Center on Reading Research, 2005)

• Outcome – measure effectiveness of overall instructional program

• Screening / Benchmark – quick and efficient. Measure a student’s current achievement based on grade-level performance standards

• Diagnostic – define a student’s weaknesses and strengths with critical skills

• Progress Monitoring – measure of progress towards academic goals related to current instruction

• Informal – provide student level information, but are not standardized or normed

• Discuss the key elements of valid and reliable assessments.

• What are the benefits of having an assessment system?

• Primary users are teachers, school administrators, district level administrators, state education agencies, and policy decision makers.

• Measures what content and skills have been mastered.

• High stakes summative assessments measure program effectiveness and normative student achievement.

• Usually administered once a year but could also be used to measure an end of semester study unit.

(K- 12 Reading Model pg. 39)

• School staff conduct universal screening in all academic areas and behavior to all students three times/year

• Purpose of universal screening is to identify students “at risk” for academic or behavior failure

• Universal screening data tells us whether a child is on track compared to peer group and/or state standards

• The student’s data at benchmark testing periods can be utilized to validate the effectiveness of intervention. Is the gap closing?

• All students are screened to determine which students are suspected to be at risk for academic and/or behavioral difficulty. • Some students suspected to be at risk

continue in Tier I with progress monitoring;

• Progress monitoring:• Confirms or disproves risk (false positive)

0

10

20

30

40

50

60

70

Dig

its

Co

rrec

t in

on

e m

inu

te

3rd Grade Math Addition and Subtraction 0-18

3rd Grade Addition and Subtraction 0-18

010203040506070

John Su

eSteve

Sherry

BridgetMike

Larisa

CarleyTomEliasTylerEvanZane

Kadon

Jasmine

NicoleGina

Cardenas

TiffanyKris

ShermanAaronSkye

Graham

Roxann

Dig

its

Co

rrec

t in

On

e M

inu

te

• http://kmnet.renlearn.com/Library/R003979401GG1107.pdf

• http://kmnet.renlearn.com/Library/R003979501GGAA93.pdf

• Why is it essential to administer screening assessments?

• What are the unique challenges faced by institutional education centers?

• How can institutional education centers mitigate unique challenges?

• Primary users - teachers, students and parents

• Promote academic growth while students are progressing to meet standards.

• Measure for ongoing, specific instructional decisions based on data outcome.

• Includes curriculum-based assessments (CBAs), curriculum-based measures (CBMs), daily, informal observations, curriculum tasks, informal or formal tests (K- 12 Reading Model pg. 39)

• Progress monitoring is a scientifically based practice that uses a set of standardized assessment procedures for determining the extent to which students are benefiting from classroom instruction and for monitoring effectiveness of curriculum on a regular basis.

• Progress monitoring can be implemented with individual students or an entire class.

• National Center on Student Progress Monitoring• Formative assessment – process of monitoring

student achievement during instruction.• CBMs are primarily used as a method for progress

monitoring because they are brief, easy to administer and score, and are good predictors of student ability

• Standard assessment tools… • Reliable• Valid• Sensitive• Capacity to model growth• Independent• Feasible

• That are the same difficulty and that are… • Given the same way each time.

• Curriculum-based measurements (CBM)• Original work from Deno, 1985• Establish entry-level performance relative to

long term goal and note trends (How does the individual compare to the whole? Dual discrepancy)

• Good predictors of student achievement• Have a prescribed methodology which

increases validity and reliability• Include systematic procedures for analyzing

and summarizing information

• Monitoring Basic Skills Progress (Pro-Ed Online, 2006);

• DIBELS (2006);• Intervention CBM probes (Interventioncentral.org) FREE

• AIMSweb (2006)

• The same kind of evaluation technology as other The same kind of evaluation technology as other professions…professions…• DoctorsDoctors• Stock brokersStock brokers

• PowerfulPowerful measures that are:

• SimpleSimple

• AccurateAccurate

• EfficientEfficient indicators of student achievement that guide/inform a variety of decisions decisions

• Are designed to serve as “Are designed to serve as “indicatorsindicators” ”

of general math achievement. of general math achievement.

• Are Are standardized testsstandardized tests to be given, to be given,

scored, and interpreted in a standard scored, and interpreted in a standard

way.way.

• Are Are researchedresearched with respect to with respect to

psychometric properties to ensure psychometric properties to ensure

accurate measures of learning.accurate measures of learning.

27

• Math probes are administered 2 to 4 minutes Math probes are administered 2 to 4 minutes

depending upon the Grade Level of the assessment depending upon the Grade Level of the assessment

material.material.

• Emphasis is placed on Emphasis is placed on what students do correctlywhat students do correctly..Grade Level

Timing Test Arrangement What is Scored

1 - 3 2 minutes

Individual, small, or large group

Correct digits in answer

3 4 minutes

Individual, small, or large group

Correct digits in answer

5 -6 4 minutes

Individual, small, or large group

Correct digits in answer or

Correct digits in answer & critical process

• Oral reading fluency (ORF)

• 3 minute cloze comprehension

• Definitions

• Using aggregate tables activity• Using the aggregate table (green sheets) in your

handouts, with a partner, for Grade 5 and 6 identify: • Winter benchmark levels within the 50th-75th %• Number of students in the spring sample• The ROI for the 25th percentile

• ROI Formula=ROI is the spring score – fall score (or winter – fall) divided by 36 weeks (or 18 weeks, depending on the number of instructional weeks).

• Num = Number of students • WRC = words read correctly• ROI = Rate of improvement.

Fall Winter SpringGrade

%ile Num CD Num CD Num CD ROI

5

90

6944

51

7603

60

8023

73 0.6

75 40 48 61 0.6

50 30 38 49 0.5

25 23 28 36 0.4

10 15 18 25 0.3

Mean

32 39 49

StDv 15 17 20

Fall Winter SpringGrade

%ile

Num CD Num CD Num CD ROI

6

90

2692

53

3654

69

3318

65 0.3

75 39 50 47 0.2

50 27 37 33 0.2

25 20 27 25 0.1

10 14 21 21 0.2Mea

n32 42 39

StDv

21 23 23

100

95

90

85

80

75

70

65

60

55

50

45

40

35

30

25

20

15

10

5

B 1 2 3 4 5 6 7 8 9 10

11

12

13

Word

s R

ead C

orr

ect

X

XX

100

95

90

85

80

75

70

65

60

55

50

45

40

35

30

25

20

15

10

5

B 1 2 3 4 5 6 7 8 9 10

11

12

13

Word

s R

ead C

orr

ect

XX

X XX X

XX

XX

XX

X

XX

X

• New Practitioner Briefs Added to NCSPM Library!Two new practitioner briefs on website; the first is What Is Student Progress Monitoring and How Will It Help Me?

• The second is Getting Started:  How Do I Implement CBM in My School?

• Both briefs are geared toward administrators, but are beneficial for teachers and families as well.

• http://www.studentprogress.org

34

• Would progress monitoring be effective within institutional education centers?

• Why or why not?• What is currently in place?• What needs to be added?

• Targeted assessment means shifting to evaluations that are designed around the specific “targeted” concerns of the student.

• In other words, we select assessments that measure the area of concern rather than administering an assessment and then trying to determine what it means.

• http://www.k12.wa.us/CurriculumInstruct/Reading/default.aspx

The questions that p_____ face as the raise ch________ from in_______ to adult life are not easy to an________. Both fa______ and m________ can become concerned when health problems such as co_______ arise any time after the e______ stage to later life. Experts recommend that young ch______ should have plenty of s______ and nutritious food for healthy growth. B______ and g______ should not share the same b_______ or even sleep in the same r______. They may be afraid of the d_____.

- Teaching Reading in the Content Areas. 2nd edition. Billmeyer and Barton, 1998.

• Reading• Decoding/Phonemic Awareness• Fluency• Vocabulary• Comprehension

• Mathematics• Low stress strategies

• Research related to adjudicated youth is limited.

• DeBoer, Malmgren, Glass (November, 2006) Reading Instruction for Youth with EBD in a Juvenile Correctional Facility

For students with low comprehension...

For students with high comprehension...

For students with low fluency...

•Assess phonemic awareness and teach if necessary.•Assess decoding and teach if necessary.•Teach fluency explicitly.•Teach comprehension.•Work on vocabulary.

•Assess decoding and teach if necessary.•Teach fluency explicitly.•Challenge with high-level comprehension and vocabulary.

For students with high fluency...

•Work on vocabulary.•Teach comprehension.

•Challenge with high-level comprehension and vocabulary.Adapted from Hasbrouck & Tindal (2006)

http://www.readnaturally.com/howto/whoNeeds.htm

• Choral reading• Cloze reading• Speedy Syllables• Assisted Reading Practice• 6 Minute Solution• Paired Reading

• Morphemic Elements•Affix concentration•Meaningful Affixes•Graphic organizers

• Collaborative Strategic Reading (Klingner & Vaughn, 1998)

• Strategic Instruction Model (Deschler & Schumaker, 1988)

• Lattice multiplication• Lattice addition• Skip zeroes• Duplation• Thoughts about Presentation

• Think Alouds

• www.edhelper.com• http://www.khake.com/page3.html

• Skills for Today website• http://www.alaskachd.org/toolkit/index.html

• Self-determination toolkit• WSECU – Vault program• http://www.handsonbanking.org/• http://www.fcrr.org/

• Florida Center for Reading Research• http://www.k12.wa.us/assessment/WASL/SocialStudies/

default.aspx• OSPI reading assessment – 4 week summer school course.

For further information, please contact:

David Tudordavid.tudor@k12.wa.us(360)725-6075