curriculum based measurement - part ii

7/31/2019 Curriculum Based Measurement - Part II

1/116


2/116

Overview

AssessmenttoolsforRtI sortingthroughthe

options EssentialfeaturesofCurriculumBased

Measurement(CBM)

ExamplesofCBMandwheretogetmaterials

Interpretingperformance,

norms,

benchmarks,andstandards oh,my!

SettinggoalsandmonitoringprogressusingvariousCBMtools


3/116

AMulti

Tiered

Model

Core Program

Supplemental SupportSupplemental Support

Intensive Support


4/116

Dontforget

the

purpose

Outcome

Provideabottomline

evaluationofthe

effectivenessof

the

program.

Screening

Administeredtodeterminewhichchildrenareat

riskfordifficultyandwhowillneedadditionalintervention.

Diagnosis Helpteachersplaninstruction byprovidingindepthinformationaboutstudents skillsandinstructionalneeds.

ProgressMonitoring

Determinewhetherstudentsaremaking

adequateprogress orneedmoreinterventiontoachievegradeleveloutcomes.

Source:ReadingFirstInitiative:SecretarysLeadershipAcademy


5/116

AdministrationandScoring:

MathematicsComputation

MaterialsNeeded:

Directions

Mathprobe

Scoringtemplate

Stopwatch


6/116


7/116

DirectionsforAdministration

ofMathematics

Probes

Providethestudentwithapencilandthemathprobe. Placetheprobefacedownonthedeskinfront

of

the

students.

Readthesespecificdirectionstothestudent(s):

Directions

for

Multiple

Skill

Probes:

The

sheet

on

your

desk

contains

anumberofmathproblems. Thereareseveraltypesofproblemsonthesheet. Someare

(listallproblemtypesforthatprobe).

Lookat

eachproblemcarefullybeforeyouanswerit. WhenIsaybegin,

start

answeringthe

problems.

Start

with

the

first

problem

and

work

across

thepage(demonstratebypointing).

Thengotothenextrow.Tryto

workeachproblem. Ifyoucannotansweraproblem,markanX

throughitandgotothenextone.(Ifthereismorethanonepage,readthefollowing:

Ifyoufinishapage,turnthepageandcontinueworking.)

Are

there

any

questions?

(pause)


8/116


ofMathematics

Probes

DirectionsforSingleSkillProbes:Thesheetonyourdeskcontains

anumber

of

math

problems.

All

the

problems

are(+,

, x,

).

Lookateachproblemcarefullybeforeyou

answerit. WhenIsaybegin,

startansweringtheproblems.

Start

with

the

first

problem

and

work

across

the

page

(demonstratebypointing).

Thengotothenextrow.Tryto

workeachproblem. Ifyoucannotansweraproblem,markanX

throughitandgotothenextone.(Ifthereismorethanone

page,read

the

following: Ifyoufinishapage,turnthepageandcontinueworking.)Arethereanyquestions?

(pause)


9/116


ofMathematics

Probes

SayBegin andstartyourstopwatch

Monitorstudents

to

ensure

that

they

work

acrossthepageanddonotskiparoundoransweronlyspecificproblems. Ifthe

student(s)

skip

around

on

the

page,

say,

Try

toworkoneachproblem. Ifastudentworkstoolongononeproblem,say,MarkanXthroughtheproblemandgotothenextone.Attheendof2minutes,sayStop. Putyour

pencilsdown.


10/116

EfficientMathScoring

Procedures Comparethestudentsanswerwiththescoringtemplate. Iftheansweriscorrect,givethestudentfullcredit.

Iftheanswerisincorrect,scorethecorrectdigits(CD).

RecordthetotalCDsearnedforeachprobleminparentheses.

Afteralltheproblemshavebeenscored,sumthe

numberof

CDs

per

row,

and

record

this

number

in

bracketsattherighthandmargin.

Sumtherowtotalsandrecordthisnumberinbrackets

atthe

top

of

each

page.


11/116


12/116


13/116

WhatisaCorrectDigit?

Acorrectdigit(CD)isadigitthatiswritten

correctlybelow

the

line.

Digits

written

above

theline arenotcountedascorrectdigits.Simple

Case

Complex

Case

30

351

21

x 25

9 1755

7020

8775

ProblemValue

(1CD)

(12CD)


14/116

WhatisaCorrectDigit?

Problemvaluesonthescoringtemplateareassigned

accordingto

the

longest

method

taught

to

solve

the

problem(answerandprocess).

Ifastudents

answer

is

correct,

the

student

receives

the

fullproblemvalue,eveniftheworkisnotshown

Ex: 350

350

x20 x 207000 (11CD)

000

7000

7000 (11CD)


15/116

AdditionalFeaturesof

CorrectDigits

Place HoldersPlace Holders::

If the problem is correct, anIf the problem is correct, an XX,, OO, or an explicit, or an explicit

space that counts as a place holder is scored as a correct digitspace that counts as a place holder is scored as a correct digit..

EX:EX: Template (a) (b) (c)Template (a) (b) (c)

347347

x 19x 19

31233123

347X347X

65936593

(12 CD)(12 CD)

347347

x 19x 19

31233123

347347

35703570

(8 CD)(8 CD)

347347

x 19x 19

31233123

34703470

65936593

(12 CD)(12 CD)

347347

x 19x 19

3123312334703470

65936593

(12 CD)(12 CD)


16/116


CorrectDigits

CrossedOutProblems: Ifthestudenthascrossedoutaproblem,creditisearnedforthecorrectdigitswritten.

EX:Template

(a) (b) (c)

1212

x 14x 14

4848

(2 CD)(2 CD)

1212

x 14x 14

4848

12x12x

(5 CD)(5 CD)

1212

x 14x 14

4848

120120

168168

(8 CD)(8 CD)

1212

x 14x 14

4848

120120

168168

(8 CD)(8 CD)


17/116


CorrectDigits

IncompleteProblems: Whenastudenthasnotcompletedaproblem,creditisearnedforthecorrectdigitswritten.

EX:Template

(a) (b) (c)

3232x15x15

00

(1 CD)(1 CD)

3232x15x15

160160

32032000

(7 CD)(7 CD)

3232x15x15

160160

320320

(6 CD)(6 CD)

3232x15x15

160160

320320480480

(9 CD)(9 CD)


18/116

AdditionalFeaturesofCorrectDigits

ReversedDigits:

Reverseddigitsarecountedascorrect. Ifthedigit

lookslikeanothernumber,countitaswrong.

EX:Template

(a) (b) (c)

3232+32+32

9494

(1 CD)(1 CD)

3232+32+32

44

(2 CD)(2 CD)

3232+32+32

66

(2 CD)(2 CD)

3232+32+32

6464

(2 CD)(2 CD)


19/116

AdditionalFeaturesof Correct

Digits

Remainders: Indivision,remaindersarescoredascorrectdigits.Zeroremainders(r0)arenotscoredascorrectdigits.

IntermediateSteps

of

Multiplication

and

Division

Basic

Facts:

Onlycountthecorrectdigitsintheanswerofthemultiplicationanddivisionbasicfactsupto12. Donotcounttheworkshown

as

correct

digits.Decimals:Adecimalwrittenintheappropriateplaceiscounted

asacorrectdigit. Decimalproblemsarescoredfirstforcorrectdigitsandthenanextrapointisgivenforcorrectplacementof

thedecimalpoint. Ifthedigitsareincorrectlyplaced,butthedecimalisintherightplace,thentheygetonepointforplacementofthedecimal.


20/116


21/116


22/116


ofWritten

Expression

Selectanappropriatestorystarter.

Providethe

student

with

apencil

and

asheet

of

lined

paper.

Saythesespecificdirectionstothestudents:

Youaregoingtowriteastory. First,Iwillreadasentence,andthenyouwillwriteastoryaboutwhathappensnext. Youwillhave1minutetothinkaboutwhatyouwillwrite,and3minutestowriteyourstory. Remembertodoyourbestwork. Ifyou

dontknow

how

to

spell

aword,

you

should

guess.

Are

there

any

questions?(pause).

Putyourpencilsdownandlisten.Forthe

nextminute,thinkabout(insertstorystarter).


23/116


ofWritten

Expression

Afterreadingthestorystarter,beginyourstopwatchandallow1minute

forstudentstothink.

(Monitorstudentssothattheydo

notbeginwriting).

After30seconds

say:Youshouldbethinkingabout(insertstory

starter).

Attheendof1minute

say:Nowbeginwriting.

Restartyour

stopwatch.

Monitorstudents

attentiontothetask. Encouragestudentsto

workonlyiftheyarelookingaroundortalking.

After90seconds

say:

Youshouldbewritingabout(insertstory

starter).

Attheendof3minutes

say:Stop.Putyourpencilsdown.


24/116


25/116


26/116

TotalWordsWritten

WhenscoringTWW,underline eachwordwritten Awordisanyletterorgroupoflettersseparatedbyaspace,

even

ifthe

word

is

misspelled orisanonsenseword.

Examples:

The

sky

was

blue

TWW=4

The sky was blew TWW=4I tuk a baf TWW=4I

tuka

baf

TWW=3


27/116


28/116

TotalWordsWritten

Rule3: Abbreviations Commonlyusedabbreviationsarecountedaswords.Example:Chris

watched

T.V.(TWW=3)

Rule

4: StoryTitlesorEndings Wordswrittenasastorytitleorendingarecountedaswordswritten.

Example:The

Big

Run

On

the

fourth

of

July,I

ran

the

Boston

Marathon.

The End.(TWW=15)


29/116


30/116

ScoringTotalWordsWritten

(TWW)

WrittenExpressionExample1

Scott,Grade2

Yesterday,amonkeyclimbedthroughthewindowatschooland

and he was jumping on descs and when we tride to gethim he would cLimb up on top of the cupberds and wecould not reach him. When we went up their on a Ladder

he would jump on a Light.

TWW=________


31/116

ScoringTotalWordsWritten

(TWW)WrittenExpressionExample1(Key)

Scott,

Grade

2


and he was jumping on descs and when we tride to gethim he would cLimb up on top of the cupberds and wecould not reach him. When we went up their on a Ladderhe would jump on a Light.

TWW=__42____


32/116

WordsSpelledCorrectly(WSC)

WhenscoringWSC,circleincorrectlyspelledwords

WhatisaCorrectlySpelledWord?

AwordisspelledcorrectlyifitcanstandaloneasacommonwordintheEnglishlanguage. Contextualclarityisnotanissue.

Ex:Billwillreedthebook.(WSC=5)

andcanratpake(WSC=3)

dzqran

down

the

rode

(WSC=4)


33/116


34/116

WordsSpelledCorrectly

Rule2:Abbreviations

Commonlyusedabbreviations

(Dr.,Mrs.,Blvd.)arecountedascorrectlyspelledwords.

Ex:JanlivesonSunsetBlvd.(WSC=5)

Rule3:

Story

Titles

or

Endings Wordswrittenaspartofastorytitleorending,ifspelledcorrectly,areincluded

inthewordsspelledcorrectlycount.

Ex:

TheBigRun

OnthefourthofJuly,IrantheBostonMarathon.

TheEnd.(WSC=15)


35/116


Rule4:CapitalizedWords

Propernounsmustbe

capitalizedtobeconsideredascorrect. Capitalizationofthefirstwordinasentenceisnotrequiredforthewordtobe

spelled

correctly.

If

aname

can

also

be

aword

(i.e.,

bill)

itdoesnothavetobecapitalized.

Ex:Marysawthebook.(WSC=4)

wecrossedtheMississippi(WSC=4)

vickisat

with

bill.

(WSC=3)

vickisatwiththebill.(WSC=4)

vickisat

with

mary.

(WSC=2)


36/116


Rule5:WordswithReversedLetters Wordscontainingreversedlettersarecountedascorrectlyspelledwordsunless

the

reversed

letter

caused

aword

to

be

spelled

incorrectly. Thisruleappliesmostfrequentlytothelettersp,b,q,d,n,andu.

Ex:Therewasabadstorm. (WSC=5)

Therewasabadstorm. (WSC=5)

Thedolphin

swam

in

the

sea.

(WSC=6)

Thebolphinswaminthesea. (WSC=5)

Joespigwonfirstplaceatthefair.(WSC=8)

Joesqig

won

first

place

at

the

fair.

(WSC=7)


37/116


Rule6:Contractions Foracontractiontobecountedasacorrectlyspelledword,anapostrophe

inthecorrectplaceinthewordisrequiredifthewordcannotstandalone.Ex:

Iwont

go.

(WSC=3)

Iwontgo. (WSC=2)

Rickshungry

(WSC=2)

Rickshungry (WSC=2)


38/116

ScoringWordsSpelledCorrectly

WrittenExpressionExample2a

Scott,Grade2


and he was jumping on descs and when we tride to get him he

would cLimb up on top of the cupberds and we could notreach him. When we went up their on a Ladder he would jumpon a Light.

TWW=

42

WSC=__


39/116

ScoringWordsSpelledCorrectly

WrittenExpressionExample2b

Mike,Grade2


how ere thay. I dont kno I wont sum Frit. Sed The mucy.

Were is The Frit I dont no. Litts tllkTWW=22

WSC=__


40/116

AnswerKeyWrittenExpression

Example2a

Scott,Grade2

Yesterday,amonkey

climbed

through

the

window

at

school

and

and he was jumping on descs and when we tride to get him hewould cLimb up on top of the cupberds and we could not reach

him. When we went up their on a Ladder he would jump on a Light.TWW=42

WSC=39


41/116


Example2b

Mike,Grade

2


how ere thay. I dont kno I wont sum Frit. Sed The mucy.

Were is The Frit I dont no. Litts tllkTWW=22

WSC=10


42/116

CorrectWritingSequences(CWS)

Acaret(^)isusedtomarkeachunitofthecorrectwritingsequence. Useanupsidedown

caret(v)

to

mark

each

unit

that

is

not

acorrect

writingsequence. Thereisanimpliedspaceatthebeginningofthefirstsentence.

WhatisaCorrectWritingSequence?

Twoadjacentwritingunits(word/wordorword/punctuation)thatareacceptablewithinthecontextofwhatiswritten.


43/116


Rule1:Capitalizationatthebeginningofa

sentence Thefirstwordofasentencemustbecapitalizedforacorrectwritingsequencetobescored.

EX:

^The^sky^was^blue.^ (CWS=5)

EX:^The

^sky^was

blew.

(CWS=3)


44/116


45/116


Rule2b:Capitalizationwithinasentence:Words

thatshould

not

be

capitalized

Wordsthatshouldnotbecapitalizedmustbeginwithalowercase

letterforacorrectwritingsequencetobescored.

Examples:

^The^monkeyâteâ^banana^while^swinging^from^the^tree.^ (CWS=11)

^The

Monkey

ateâ

Banana

while^swinging^from^the^tree.^

(CWS=7)


46/116


47/116


Rule4a:Punctuationwithinasentence:Commas

Commas

arenotcountedaspartofthecorrectwritingsequencetotalexceptwhenincludedinaseriesofnouns. Whenpartofaseries,

commas

must

be

used

correctly

for

correct

writing

sequencetobescored

Examples:

^Next,^takeôut^the^garbage.^ (CWS=6)

^Next^takeôut^the^garbage.^ (CWS=6)

^Max^went^to^the^store^to^buy^bread^,^milk^,

ând^cheese.^

(CWS=14)

^Max^went^to^the^store^to^buy^bread^,^milkând^cheese.^(CWS=13)

^Max^went^to^the^store^to^buy^bread

milkând^

cheese.^

(CWS=11)


48/116


Rule4b:Punctuationwithinasentence:Other

punctuationmarks

Otherpunctuationmarkssuchasquotes,colons,andsemicolonsarenotcountedas

partofthecorrectwritingsequencetotal.

EX:^He^yelled,^Watch^out^for^that^dinosaur.^ (CWS=8)

EX:^He^yelled,^Watch^out^for^that^dinosaur.^ (CWS=8)


49/116


Rule5:

Spelling

Wordsmustbespelledcorrectlyfor

acorrect

writing

sequence

to

be

scored.

See

Words

SpelledCorrectly(WSC)sectionofCBMAdministrationandScoringWrittenExpressionfor

specific

spelling

rules.

EX:Âllôf^the^kids^started^to^laugh.^ (CWS=8)

EX:Âllôf^the^kids^started^to

laghf.

(CWS=6)


50/116


Rule6:Syntax

Wordsmustbesyntacticallycorrect

foracorrect

writing

sequence

to

be

scored.

Sentencesthatbeginwithconjunctionsareconsideredsyntacticallycorrect.

Î^had^never^seen^the^wolves^before.^ (CWS=8)

Î^never

seen^the^wolves

never.^ (CWS=5)

Ând^then^the^boy^gave^the^duck^some^bread.^

(CWS=10)


51/116


Rule7:Semantics Wordsmustbesemanticallycorrectforacorrectwritingsequencetobescored.^Jamaal^went^to^the^library.^ (CWS=6)

^Jamaal^went

too

the^library.^

(CWS=4)

^My^dad^made^the^treehouse^especially^for^me.^

(CWS=9)

^My^dad^made^the^treehouse

specially

for^me.^

(CWS=7)


52/116


Rule8:StoryTitlesandEndings Wordswrittenaspartofastorytitleorendingareincludedinthecorrectwritingsequencetotal.

Example:

^The^Big^Run^Ôn^the^fourthôf^July,Î^

ran^the^Boston^Marathon.^

^TheÊnd.^

(CWS=18)


53/116

ScoringCorrectWritingSequences

(CWS)WrittenExpressionExample3a

Scott,Grade2


and he was jumping on descs and when we tride to get him he

would climb up on top of the cupberds and we could notreach him. When we went up their on a Ladder he would jumpon a Light.

TWW=

42

WSC=39

CWS=__

i i i


54/116

ScoringCorrectWritingSequences

(CWS)WrittenExpressionExample3b

Mike,Grade

2


how ere thay. I dont kno I wont sum Frit Sed The mucy.

Were is The Frit I dont no. Litts tllk

TWW=22

WSC=10

CWS=__

A K W itt E i


55/116


Example3a

Scott,Grade2


ând^he^was^jumpingôn_

descs

and^when^we

tride

to^ get^him^he^would^climbûpôn^topôf^the

cupberds

and^we^could^not^reach^him.^^When^we^wentûp

their

onâ

Ladder

he^would^jumpônâ

Light.

TWW=42

WSC=39

CWS=32


56/116


Example3b

Mike,Grade2


how

ere

thay.

^ I

dont

kno

^ I

wont

sum

Frit

Sed

The

mucy.

Were

is

The

Frit

^I

dont

no.

Litts

tllk

TWW=22

WSC= 10

CWS= 3


57/116

W itt E i


58/116

WrittenExpression Example

4b

Andy,Grade

8

Ifyouweretheteacherofthisclass,write

aboutwhatitwouldbelike.

Their wood be alot of partys, no work games Idont no about t.v but free time all the Time. We wouldhave aLot fun. pop candy enthy you want It wood be

Like Never LandTWW=____

WSC=____

CWS=____

i i


59/116


Example4a

Charlie,Grade8

Ifyou

were

the

teacher

of

this

class,

write

aboutwhatitwouldbelike.

ÎfÎ^were^the^teacherôf^this^class,Î^

would^not^give^homeworkând^the^students^would^b eâble^toêatîn^class.^^ThenÎ^would^give^themâ^ choiceôf

assiments

they

did,^but^the

would^have^to

chose

one^to^turnîn.^Î^would^give^themTWW=48

WSC=47

CWS=44

A K W i E i


60/116


Example4b

Andy,Grade8

Their

wood

be

alot

of

partys,

no^work

games

I^dontnoaboutt.v

but^free^time^all^theTime

^We^would^haveaLot

fun.^popcandyenthy

you^

want

Itwood

be

Like^Never

Land

TWW=36

WSC=31CWS=13


61/116

Wheretogetstarted?

CommercialProducts: AIMSweb

EdCheckup

STEEP YearlyProgressPro DIBELS

DIYApproach InterventionCentral

Booksand

publications:

Hosp,M.K.,Hosp,J.L.,&Howell,K.W.(2007).TheABCsofCBM:APracticalGuidetoCurriculumBasedMeasurement.NewYork:Guilford
http://www.interventioncentral.org/htmdocs/interventions/cbmwarehouse.phphttp://www.interventioncentral.org/htmdocs/interventions/cbmwarehouse.phphttp://www.interventioncentral.org/htmdocs/interventions/cbmwarehouse.phphttp://www.interventioncentral.org/htmdocs/interventions/cbmwarehouse.php


62/116

InterpretingPerformance

Howdowedeterminewhatscoresmean?

1.

Expert

Opinion/Instructional

Placement

Standards

2.

NationalNorms

3. LocalNorms Classroom Building District

4.

LinkagestoLaterOutcomes


63/116

InstructionalPlacementStandards

Thelevelatwhichastudentcouldbeexpectedtobenefitfromteacherled instruction,basedonexpertopinion

InstructionalPlacementStandardsinReading:

For1st

2nd

grade

materials:

40

60

wpm,

with

4or

fewer

errors

For3rd6th

gradematerials:70100wpm,with6orfewererrors

Theabove

ranges

assume

the

child

is

receiving

teacher

led

instruction

inthematerial

Theseareapproximate

ranges. Thetypeofinstructionmustbetaken

intoaccount(e.g.,teacherledvs.independent),aswellasthequality

ofthe

errors

Expert Opinion/Instructional


64/116

ExpertOpinion/Instructional

PlacementStandards

MCBM(MathematicsComputation)InstructionalPlacementStandards

Grades13:

10

19

CD

with

37errors

Grades4andup:2039CDwith37errors

ExpectancyCriteriaforCBMWE Grade1=15ormoreTWW

Grade2=28ormoreTWW

Grade3=37ormoreTWW

Grade4=41ormoreTWW

Grade5=49ormoreTWW

Grade6=52

or

more

TWW


65/116

NationalNorms

CBM

E.g.,AIMSweb,HasbrouckandTindalpublications(seeinterventioncentral.com)orgostraighttothe

source.http://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNor

ms.pdf

Potentiallyusefulbutwithcaveats:Aggregatesofdatacollectedinvariouslocales

Sometimesbasedonvaryingmaterials

Biasedsamples

NotcontrolledforrepresentativenessofU.S.population

Beextra

cautious

in

areas

other

than

reading
http://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.brtprojects.org/publications/tech_reports/TechRpt33_FluencyNorms.pdfhttp://www.gifted.uconn.edu/SEMR/oralreadingfluency%20norms.2005.pdf


66/116

LocalNorms

WhyLocalNorms?

Localnorms

operationalize

academic

expectations

Localnormsmaydecreasebiasinthedecision makingprocessLocal

norms

can

be

used

to

make

arange

of

educationaldecisions


67/116

LevelsofLocalNorms

1. ClassroomNorms2. BuildingNorms3. DistrictNorms Considerations

in

Selecting

alevel:

Characteristicsoflocalpopulations(homogeneousvs.heterogeneous)

Curriculum

Variability Schoolautonomyindecisionmaking

Decisionsforwhichthedatawillbeused

Economics


68/116


69/116

BuildingNorms1.

Advantages

Reasonableestimateoflevelandvariabilitywithinabuilding

Morestablethanclassroomnorms

DefensibleestimateofLRE

Usefulwhenschoolswithinadistrictdifferoncurriculaorpopulation

Can

aggregate

into

district

norms

2.

Disadvantages:

Samplemaybetoosmalltopermituseofpercentileranks

Mayresultindifferentoutcomesinproblemidentificationandcertificationifchildmoveswithinthedistrict3.

Bestuses:

Problemidentification,goalsetting,periodicandannualreviews

Discussion

tool

for

faculty

and

administration


70/116

DistrictNorms

Advantages Consistentestimateofachievementandvariabilityacrossthedistrict

ConsistentestimateofLRE

Reducedcost

Canusepercentileranks

Canbeusedforcontinuoussetofeducationaldecisions


71/116

DistrictNorms

Disadvantages:

LesspreciseestimateofLRE

Requiresrelativehomogeneityofcurriculaandpopulation

acrossschools

Best

uses:

ProblemCertification

Prioritizingstudentneeds

Goals,objectives,progressmonitoring

Periodicandannualreviews


72/116

LinkagestoLaterOutcomes

Based

on

correlations

with

performance

on

latermeasures oftenwithahighstakestest

Usedforscreeningandgoalsetting

Distinguishbetweenscreeningcutscoresandbenchmarkgoals


73/116

ScreeningCutScoresvs.BenchmarkGoals

ScreeningCutScores:

Identifystudents

at

risk

of

failureonanoutcome

measure

PredictFailure

Usuallyhavelowervaluesthanbenchmarkgoals

Usedtoidentifystudentsinneed

of

intervention

services

73

BenchmarkGoals:

Goalsfordesiredperformance

Predictsuccess

Usuallyhavehighervaluesthanscreeningcutscores

Usedtosetappropriategoals

forstudentsreceiving

interventionservices

l


74/116

Example:DIBELS

74

Various Classification Categories

Low Risk Established

Benchm ark #

Some Risk Emerging

Cut - score #

At Risk Deficit

(Good, Kaminski, Simmons, & Kameenui, 2001)


75/116

ProgressMonitoringProcedures

1. SurveyLevelAssessment2. Settingdatabasedgoals3. Collectingandgraphingprogressmonitoring

data4. Makingdecisions


76/116

CBMSurvey

Level

Assessment

Purpose

todeterminethelevelofthecurriculumin

whichthe

student

is

successful.

This

information

canhelpusto:

Determineastudents

instructional

level

Selectalevelforprogressmonitoring

Determinetheseverityandmagnitudeofastudents

academicproblem

Setindividualgoalsforstudents


77/116


78/116

DefiningSuccess

BenchmarksorTargets childperformsinarangethat

predicts

successful

performance

on

later

outcomes(e.g.,DIBELSbenchmarkgoals)

LocalorNationalNorms

childperformswithin

theaverage

range

of

students

at

that

grade

ExpertJudgment;e.g.,InstructionalPlacement

Standards thelevelatwhichastudentcouldbeexpectedtobenefitfromteacherled instruction(basedonexpertjudgment)


79/116


80/116

SurveyLevelAssessmentofReading

LocalNormsExample


81/116


82/116

SurveyLevel

Assessment

Outcomes

Determineinstructional

and

monitoring

levels

(typicallynotthesamelevel)

Setindividualgoalsforstudents

Helpdeterminewhichspecificskillstoteach(e.g.,PA,AP,Fluency,etc?)


83/116

Activity:SurveyLevelAssessment

ReviewSarahsSLAdata

Answer

questions

a

through

c Discussinsmallgroups

5minutes

N d f Cl G l


84/116

NeedforClearGoals

C t f G l


85/116

ComponentsofaGoal

Individual Reading Progress Graph

Weeks of School Year

Minute(WCM)

Expecte

dRate

ofProgr

ess:2W

ordsper

Week

Current Performance: 15 WCM

Criterion forSuccess:

75 WCM

Time Frame: 30 Weeks

Behavior: Number of Words Read

Correctly per Minute

3Step Process for


86/116

3StepProcessfor

WritingIndividualized

Goals

Collectcurrent

performance

data

UseSurveyLevelAssessmentinformation

Specifymeasurement

conditions

Timeframeandmeasurementmaterialrepresentingexpectedoutcome

SpecifyCriterionforSuccess

Canuseexpectedratesofprogressor

benchmarkstandards

S l ti M t M t i l


87/116

SelectingMeasurementMaterial

forGoal

IssuestoConsider:

Degreeto

which

goal

reflects

reduced

discrepancy

betweenperformanceofstudentandgradelevel

peers

Studentsshould

do

more

in

more

difficult

material inoneyear

SensitivitytoInstructionalEffects

Ifthestudentreadslessthan1520WRCinagivenlevelofthecurriculum,thematerialmightnotdetectsmallimprovementsinreadingskills

Individualizedorbasedonbenchmark?

O G l S tti


88/116

OnGoalSetting

The greatest danger for most of usis not that our aim is too high and

we miss it, but that it is too lowand we reach it.~Michelangelo

Considerations for Setting Criteria for


89/116

ConsiderationsforSettingCriteriafor

IndividualizedGoals

BalanceRealisticandAmbitious

Studentswith

more

ambitious

goals

show

greater

achievement(Fuchs,1993)

Theultimategoalofinterventionsshouldbetocatch

studentsup

to

their

grade

level

peers

(i.e.,

reduce

the

discrepancy,solve

theproblem)

Expectedratesofprogressareavailableforstudentsin

general

education

(Fuchs

et

al.,

1993)

and

those

with

learningdisabilities(Denoetal.,2001)

Learningneedstobeacceleratedforstudentswithdisabilitiesorthosebehindtheirpeersifthe

discrepancyistobereduced


90/116

ResearchonExpectedRatesof

Progress:General

Education

WeeklyGrowth

Rates

in

Reading

(Fuchset

al.,

1993)

GradeRealistic(Mean)

Ambitious(Mean + 1 SEM)

1 2 WRC/week 2.5 WRC/week

2 1.5 2

3+ 1 1.5

R h E d R f P


91/116

ResearchonExpectedRatesofProgress:

Studentswith

Learning

Disabilities

Giveneffective

instruction,

students

with

learningdisabilitiesshouldbeabletoachieve

thefollowinggrowthrates(Denoetal.,2001):

Initial Perf. Growth Rate (WRC per week)

Up to 30 WRC 2.0

30+ WRC 1.25-1.50

SettingGoalCriteriaUsing


92/116

g g

Expected

Rates

of

Progress Takethestudentscurrentperformanceongoallevel

passages

e.g.,Johnny

currently

reads

40WRC onthirdgradepassages Countthenumberofweeksuntiltheendoftheschool

yearand

multiply

by

the

expected

rate

of

progress

(e.g.,1.5wordsperweek)

e.g.,thereare30weeksleftintheschoolyear

30*1.5

=45

Addtheresultingnumbertothestudentscurrentperformance

40+45

=

85WRC85WRC

l


93/116

SettingGoalCriteria

UsingNormsorBenchmarks Decideatwhichpercentileyouwouldliketo

seethestudentperformingbytheendoftheyear

e.g.,

25th,

50th Findthescorethatcorrespondstothat

percentileinthegoallevelmaterialatthetime

thegoal

is

to

be

achieved

e.g.,Springnormsforfourthgrade,25thpercentile=84WRC(roundto85WRC)

l


94/116

SettingGoalCriteria

UsingNorms

or

Benchmarks

Check

the

weekly

rate

of

progress

and

determineifthegoalisrealisticandambitious

Iftheweeklyrateofprogressistoohigh,the

goalmay

not

be

realistic

Iftheweeklyrateofprogressistoolow,thegoalmaynotbesufficientlyambitious

Activity: Setting goals


95/116

Activity:Settinggoals

ReviewSarahsSLAdata

Set

a

data

based

goal

for

her.

Determine:Timeframeforgoal

Levelofmaterialsforprogressmonitoring

Criterion Discussinsmallgroups

510minutes

Graphing Goals and Progress


96/116

GraphingGoalsandProgress

Whygraph?

Visualaid

makes

goalsandstudent

performancecome

tolife

Facilitatescommunicationwithparentsandamongprofessionals

Facilitatesinstructional

decision

making

Graphicdatadisplayisassociatedwithhigherstudentachievement(Fuchs&Fuchs,1986)

Individual Reading Progress Graph: Grade 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

Weeks of 1999-2000 School Year = WCMx = errors


97/116

WhatGoesonaGraph?

GeneralConventions Labelforthehorizontalaxis(xaxis).

numberof

weeks


Graphing Conventions


98/116

GraphingConventions

Unitsforthehorizontalaxis(xaxis).

Daysin

week

equal

interval

9/4/00

9

/18/00

1

0/2/00

10/16

/00

10/30

/00

11/13

/00

11/27/00

12/11/00

12/25/00

1/8/01

1

/22/01

2/5/01

2

/19/01

3/5/01

3

/19/01

4/2/01

4

/16/01

4

/30/01

5

/14/01

5

/28/01




99/116

GraphingConventions

Labelandunitsfortheverticalaxis(yaxis).

Metricfor

academic

area

(e.g.,

#of

WCM)

0

10

20

30

4050

60

70

80

90

100

110

120

130

140

WordsRe

adCorrect(W

RC)

9/4/00

9/18/00

10/2/00

10/16

/00

10/30

/00

11/13

/00

11/27

/00

12/11

/00

12/25

/00

1/8/01

1/22/01

2/5/01

2/19/01

3/5/01

3/19/01

4/2/01

4/16/01

4/30/01

5/14/01

5/28/01

Weeks of 2000-01 School Year



100/116

GraphingConventions

Longtermgoal

Specifiedby

indicating

median

baseline

score

andlongtermgoalcriterionusingXs.

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

9/6/99

9/20/9

9

10/4/9

9

10/18/9

9

11/1/9

9

11/15/9

9

11/29/9

9

12/13/9

9

12/27/9

9

1/10/0

0

1/24/0

0

2/7/00

2/21/0

0

3/6/003/20/0

0

4/3/00

4/17/0

0

5/1/00

5/15/0

0

5/29/0

0


WordsCo

rrectperMi

nute

x

x



101/116

GraphingConventions

Aimline

Represents

expected

rate

of

progress.

IndicatedbyconnectingtheXs.

0

10

20

30

4050

60

70

80

90

100110

120

130

140

9/6/9

9

9/20/9

9

10/4/9

9

10/18/9

9

11/1/9

9

11/15/9

9

11/29/9

9

12/13/9

9

12/27/9

9

1/10/0

0

1/24/0

0

2/7/0

0

2/21/0

0

3/6/0

0

3/20/0

0

4/3/0

0

4/17/0

0

5/1/0

0

5/15/0

0

5/29/0

0


WordsCorrectperM

inute

x

x



102/116

GraphingConventions

TrendLine

Representsstudents

estimated

rate

of

progressbasedonweeklyperformancedata.

Words

CorrectperM

inute

0

10

20

30

4050

60

70

80

90

100

110

120

130

140

8/23/9

9

9/6/9

9

9/20/9

9

10/4/9

9

10/18/

99

11/1/9

9

11/15/

99

11/29/

99

12/13/

99

12/27/

99

1/10/0

0

1/24/0

0

2/7/0

0

2/21/0

0

3/6/0

0

3/20/0

0

4/3/0

0

4/17/0

0

5/1/0

0

5/15/0

0

5/29/0

0




103/116

GraphingConventions

Phasechangeline.

Darkline

separating

baseline

from

intervention

andusedwheninterventionismodified

0

10

20

30

4050

60

70

80

90

100

110

120

130

140

8/23/9

9

9/6/9

9

9/20/9

9

10/4/9

9

10/18/

99

11/1/9

9

11/15/

99

11/29/

99

12/13/

99

12/27/

99

1/10/0

0

1/24/0

0

2/7/0

0

2/21/0

0

3/6/0

0

3/20/0

0

4/3/0

0

4/17/0

0

5/1/0

0

5/15/0

0

5/29/0

0




104/116

GraphingConventions

Labelforeachphase.

Specifiesdescription

of

each

phase

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

8/23/9

9

9/6/9/20/9

10/4/9

10/18/9 11

/1/911

/15/911

/29/912

/13/912

/27/9 1/10/0

1/24/0 2/7/ /21/0 3/6/ 3/

20/ 4/3/4/17/0 5/1/

5/15/0

5/29/


Regular

Class

Regular Class +

Great Leaps

Graphing Options


105/116

GraphingOptions

Byhand(graphpaper,DIBELSprogressmonitoring

bookletsRequireslittlecomputerskillsCumbersome,difficulttomodify

Mustuse

3data

point

decision

rule

or

draw

trend

lines

usingsplitmiddleapproach

UsingExcel

Easierand

more

efficient,

more

easily

corrected

and

updated

Requiressomecomputerskills

Cancalculatetrendlineusingregressionfunction

Graphing Options


106/116

GraphingOptions

Usewebbasedprogressmonitoringsystem

e.g.,DIBELSDataSystem,AIMSweb,EdCheckup,otherprograms

found

at

http://studentprogress.org

Easiestandmostefficient

Morecostly
http://studentprogress.org/http://studentprogress.org/


107/116


108/116

Data

based

Decision

RulesWhyaredecisionrulesimportant?

Ensureinformation

inclusion

and

use.

Facilitatecommunication,reducesambiguityaboutwhatstepstotakenext.

Standardizedecision

making

ensures

fair

and

equaltreatment,preventsarbitrarydecision making. Reducetimespentindecisionmaking

Databased Decision Rules


109/116

Data basedDecisionRules

Advantages:

Provideguidelinesforteachersindeciding

whetherprogram

changes

are

needed

Leadtoincreasedchangesininstructionalprogramsbyteachers

Useof

decision

rules

is

related

to

greater

student

achievement

Disadvantages:

Reducedecisions

to

yes/no

conceptualizations

Mayoversimplifycomplexandvalueladendecisions

Types of Decision Rules


110/116

TypesofDecisionRules

TrendLineAnalysis

Drawtrend

line,

compare

to

aim

line

DataPointAnalysis

Examine

individual

data

points

relative

to

trend

line

(e.g.,3consecutivebelow=change)

DynamicApproach

Providesfor

goal

adjustment

(esp.

upward)

Dynamic Approach


111/116

DynamicApproach

Collectandgraphthestudentsdata

Drawtrendline

(8

or

more

data

points)

Comparetrendlinetoaimline(goalline)

Iftrendissteeperorthesame

as

theaim,

then

re

set

goal

(raise

it)

Iftrendisflatter

thantheaim,then

makeinstructional

changes

Raising Goals


112/116

RaisingGoals

Followtrendlineouttoendofyearandusethatlevelasthenewgoal


0

10

2030

40

50

60

70

8090

100

110

120

130

140

8/23

/99

9/6/99

9/20

/99

10/4/99

10/18/99

11/1/99

11/15/99

11/29/99

12/13/99

12/27/99

1/10

/00

1/24

/00

2/7/00

2/21

/00

3/6/00

3/20

/00

4/3/00

4/17

/00

5/1/00

5/15

/00

5/29

/00

W

ordsCorrectp

erMinute

New

Goal

Benefits of the Dynamic Approach


113/116

BenefitsoftheDynamicApproach

Increasedstudent

achievement

when

goals

are

increased(Fuchs,Fuchs,&Hamlett,1989).

Correctsfor

goals

set

initially

toolow.

Dictatesthatteacherseither

changegoal

or

instruction

on

continualbasis.

Activity: Decision Rules


114/116

Activity:DecisionRules

ReviewSarahsgraph

Basedonthedatapointanalysisrule,areinterventionchangesneeded?

Discussinsmallgroups

2minutes

Some helpful resources


115/116

Somehelpfulresources.

JimWrightsfamouswebsiteinformation,tools,links,RTI_Wirehttp://www.interventioncentral.org

NationalCenterforResearchonLearningDisabilities2006conferencehttp://www.nrcld.org/sea/index.html

NationalCenterforProgressMonitoringhttp://www.studentprogress.org

Centeron

Instruction

http://www.centeroninstruction.org/

VaughnGrossReadingCenterhttp://www.texasreading.org/utcrla/

Contact Information
http://www.interventioncentral.org/http://www.nrcld.org/sea/index.htmlhttp://www.studentprogress.org/http://www.centeroninstruction.org/http://www.texasreading.org/utcrla/http://www.texasreading.org/utcrla/http://www.centeroninstruction.org/http://www.studentprogress.org/http://www.nrcld.org/sea/index.htmlhttp://www.interventioncentral.org/


116/116

ContactInformation

SuzanneBamonto Graney,Ph.D.SchoolPsychologyProgram

Rochester

Institute

of

[email protected]

585

475

2765
mailto:[email protected]:[email protected]

curriculum based measurement - part ii

Documents