aligning cbm with state assessments and ayp david heistad
TRANSCRIPT
![Page 1: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/1.jpg)
Aligning CBM With State Assessments and AYP
David Heistad
![Page 2: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/2.jpg)
Note About This Presentation
Although we use progress monitoring measures in this presentation to illustrate methods, we are not recommending or endorsing any specific product.
![Page 3: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/3.jpg)
Not All State Standards Are Created Equal
![Page 4: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/4.jpg)
Not All Curriculum-Based Measures Are Created Equal (Grade 1 MPS-CBM vs. DIBELS)
Words Correct Per Minute (wcpms)
DIBELS oral reading
Valid 193 193
Missing 0 0
Mean 58.89637 46.47668
Median 55 37
![Page 5: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/5.jpg)
Grade 1 DIBELS Much Harder Than Minneapolis CBM
Wo
rds
Re
ad
Co
rre
ctl
yp
er
Min
ute
![Page 6: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/6.jpg)
In Order to Set Grade Level Benchmarks You Need: A good state test that actually measures
reading. Reliable curriculum-based measures
which are calibrated to be equally difficult across each passage and time period. (i.e. employing standardized procedures by trained staff)
At least 60 students per grade level (preferably 100) who have taken the CBM measures and the State Test
![Page 7: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/7.jpg)
For Example: Read Naturally Benchmark and Progress Monitoring Passages With the MCA Minnesota Comprehensive Assessments
Does the State Test measure reading?– MCA in Grade 3 measures reading comprehension and
vocabulary:• Correlation with NWEA levels test = .864
– MCA and CBM Correlations (n = 1792 students)• Correlation with 3rd grade CBM fall = .733• Correlation with 3rd grade CBM winter = .746• Correlation with 3rd grade CBM spring = .739
![Page 8: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/8.jpg)
Measuring Within a Fluency Program Show Read Naturally CD
![Page 9: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/9.jpg)
Progress Monitoring Passages Aligned With Benchmark Assessments
Grade Passage Name Difficulty* Benchmark Correlation Other Monitoring Passage Correlation
1 I Like Fall—Revised -2.1 .97 .98
1 Pets 1.8 .95 .95
1 My Big Sister Revised 1 -2.1 .95 .92
1 Jill’s First Bike 0.5 .94 .92
1 Meg’s Race -.9 .99 .99
1 Colors .6 .98 .99
1 My Baby Sister 1.0 .98 .99
1 My New Puppy -1.9 .97 .95
1 Pennies for Pine Cones -4.0 .96 .96
1 My Camera 2.4 .96 .97
1 Art 3.9 .98 .97
1 Pigs -5.1 .91 .94
1 Sugar Cookies -.8 .94 .94
1 Grandpa -4.1 .97 .96
1 Bubble Gum -2.8 .96 .97
1 Making Lunch -2.3 .95 .95
![Page 10: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/10.jpg)
Reliability of Read Naturally Benchmark Assessments (Stability Across Time)
Grade N Fall to Winter Winter to Spring Fall to Spring
1 69 .88 .88 .78
2 70 .86 .92 .82
3 98 .92 .92 .88
4 101 .91 .94 .90
5 110 .90 .83 .85
6 98 .87 .84 .87
7 82 .91 .91 .92
![Page 11: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/11.jpg)
Calibrated Difficulty
Grade Number of
studies
Passage A – Passage B
Passage A – Passage C
Passage B – Passage C
1 3 -1.9 -2.1 +.2
2 3 .4 2.0 -1.6
3 3 .4 -2.3 2.7
4 3 1.4 -.4 1.8
5 3 1.5 2.5 -1.0
6 3 -2.7 0 -2.7
7 4 5.5 5.8 -.3
8 3 1.1 -3.1 4.2
![Page 12: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/12.jpg)
What Does a Correlation Look Like on a Scatter-Plot?
R = 0.64
300
320
340
360
380
400
0 20 40 60 80 100 120 140 160 180 200 220
Grade 3 MCAReading
Linear (Grade 3MCA Reading)
![Page 13: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/13.jpg)
What Score Predicts Proficient on the MCA
300
325
350
375
400
0 20 40 60 80 100 120 140 160 180 200 220
Grade 3 MCAReading
HIT
HIT
![Page 14: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/14.jpg)
What Does the “Hit Rate” Table Look Like?
Classification Results a
343 66 409
361 696 1057
0 3 3
83.9 16.1 100.0
34.2 65.8 100.0
.0 100.0 100.0
Proficient on MCAGrade 3 Reading
.001.00
Ungrouped cases
.00
1.00
Ungrouped cases
%
Original Count
.00 1.00
Predicted Group
Membership
Total
70.9% of original grouped cases correctly classified.a.
![Page 15: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/15.jpg)
Statistics
FallWPM WinterWPM SpringWPM 2005 MCA Scale Scores - Reading N Valid 1526 1526 1526 1526 Missing 0 0 0 0 Mean 45.15 70.10 90.74 1447.01 Median 35.00 63.00 86.50 1440.00 Percentiles 1 .00 4.27 9.00 975.40 2 .00 7.00 15.54 1010.00 19 15.00 29.13 49.00 1240.00 20 15.00 31.00 50.00 1250.00 21 16.00 32.00 52.00 1260.00 22 16.00 33.00 53.94 1260.00 23 17.00 34.00 55.00 1270.00 24 17.48 35.00 56.00 1280.00 25 18.00 36.00 57.00 1280.00 26 19.00 37.00 59.00 1300.00 27 20.00 38.00 60.00 1300.00 28 20.00 38.00 60.56 1310.00 29 21.00 39.00 62.00 1320.00 30 21.00 41.00 63.00 1320.00 31 22.00 42.37 64.00 1340.00 32 22.00 44.00 65.00 1340.00 33 23.00 45.00 66.00 1350.00 34 23.00 46.00 67.00 1350.00 35 24.00 47.00 68.00 1360.00 36 25.00 48.00 69.00 1360.00 37 26.00 49.00 70.00 1370.00 38 26.26 50.00 72.00 1390.00 39 27.00 51.00 72.00 1390.00 40 28.00 53.00 73.00 1390.00 41 28.00 54.00 75.00 1400.00 42 29.00 55.00 76.34 1400.00 43 29.00 56.61 78.00 1420.00 44 30.00 58.00 79.00 1420.00 45 31.00 59.00 79.15 1420.00 46 31.00 60.00 81.42 1430.00 47 32.00 61.00 83.00 1430.00
Equipercentile Linking Example: Grade 3
![Page 16: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/16.jpg)
How to Find the Cut Score
1) Discriminant Function (moving the bar up and down the scale and calculating the hit rate)
2) Equipercentile (use SPSS or do by hand)– Use only student with both CBM and State test
scores– Put the scores in order for both CBM and State
test right next to each other in long columns– Find the state score– CBM score next to it is the cut score
![Page 17: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/17.jpg)
Oral Reading 2001 Validity With Grade 3 Reading MCA 2003
20%
30%
40%
50%
60%
70%
80%
90%
100%
Number of Words Read Correctly per MinuteSpring of First Grade
All Students ELL Students
District Standard= 60 wpm80% make level 2b
Cu
mu
lati
ve
Pe
rce
nt
of
Stu
de
nts
at
14
20
or
Ab
ov
e
![Page 18: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/18.jpg)
Beginning of Kindergarten Assessment (BKA) Alignment to
State Standards
![Page 19: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/19.jpg)
Literacy Items on the BKA Includes:
– Picture vocabulary– Oral comprehension– Letter names– Letter sounds– Rhyming– Alliteration (initial sounds)– Concepts of Print– Total Composite Score
![Page 20: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/20.jpg)
BKA Predicts Reading Well by Grade 3 (3 and ½ Years Later!)
Correlation between BKA composite and NALT Grade 3 Reading= .67
Correlation between BKA composite and MCA Grade 3 Reading= .61
A BKA composite score of 85 or higher predicts with 75% accuracy that students will score at level 3 (1420) on the MCA Reading in 3rd grade
![Page 21: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/21.jpg)
BKA & EKA to 1st Grade OR
Data Utilized: Kindergarten from SY0102
OR from SY0203NALT from SY0304
![Page 22: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/22.jpg)
End of Kindergarten Correlated With First Grade Oral Reading
.80** .66**
.73** .63**
.45** .49**
.40** .42**
.73** .49**
EKA Early LiteracyComposite
EKA Verbal Literacy
EKA Quantitative Literacy
EKA Concepts of Print
EKA Oral Fluency
1st GradeOral Reading
1st GradeReading
Comprehension
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 23: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/23.jpg)
End of Kindergarten Predicting 2nd Grade Computer Adaptive Levels Tests
.66** .60**
.62** .56**
.50** .44**
.44** .40**
.51** .47**
Early Literacy Composite
Verbal Literacy
Quantitative Literacy
Concepts of Print
Oral Fluency
2nd GradeNALT
Reading2nd GradeNALT Math
Correlation is significant at the 0.01 level (2-tailed).**.
![Page 24: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/24.jpg)
Individual Growth and Development Indicators (IGDIs) for Ages 3-5
Developed at the University of Minnesota by Dr. Scott McConnel ([email protected]) and Dr. Mary McEvoy
In the public domain and can be found at http://ici2.coled.umn.edu/ecri/
![Page 25: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/25.jpg)
IGDI Example: Alliteration (2 minutes) Training
– “We’re going to look at some pictures and find the ones that start with the same sound.”
– “Listen to me. I’m going to say the names of these pictures, and find 2 that start with the same sound.” Point to and name d-door, d-dice
Sample Items– “Now lets do one together. First, it’s my turn.”– Hold Sample 2 in front of the child, Point to the one
that starts with the same sound as h-hat. Test Administration
– “Now let’s do some more...”
![Page 26: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/26.jpg)
Letter Identification Distribution - All Students
![Page 27: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/27.jpg)
Letter Identification Distribution for African Americans
![Page 28: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/28.jpg)
Winter Kindergarten Assessment Results
Fall to Winter Letter Sound Growth 02/03: Schools Who Achieved the District Goal of 20 Letter Sounds per Minute
0
5
10
15
20
25
30
35
HALL
**
LAKE
HAR
RIET
WHI
TTIE
R**
KEEW
AYDI
N
RAM
SEY
NELL
IE S
TONE
DOW
LING
WEB
STER
PRAT
T
JEFF
ERSO
N
LUCY
LAN
EY
HALE
HOW
E
POW
DERH
ORN
ANDE
RSEN
ELE
M
PAM
at K
enwo
od**
WIL
LARD
**
LYND
ALE
BETH
UNE*
*
NORT
HROP
**
LORI
NG**
BOTT
INEA
U**
LINC
OLN*
*
SHIN
GLE
CREE
K**
So
un
ds
per
min
ute
.
Fall LS
Winter LS
![Page 29: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/29.jpg)
District Level Reports on Early Literacy Gaps and Progress
Go live to: http://rea.mpls.k12.mn.us/
![Page 30: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/30.jpg)
Beginning of Kindergarten Phonemic Awareness Proficiency Trends
0%
20%
40%
60%
80%
100%
Per
cent
of Stu
den
ts P
rofici
ent
2004 52% 45% 29% 32% 82% 53%
2005 44% 45% 37% 26% 83% 52%
2006 46% 46% 39% 27% 85% 53%
American Indian
African American
Asian Hispanic WhiteAll
Students
![Page 31: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/31.jpg)
Beginning of Kindergarten Alphabetic Principle Proficiency Trends
0%
20%
40%
60%
80%
100%
Per
cent
of Stu
den
ts
2004 51% 57% 55% 34% 83% 61%
2005 38% 56% 60% 34% 84% 60%
2006 46% 57% 62% 32% 88% 61%
American Indian
African American
Asian Hispanic WhiteAll
Students
![Page 32: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/32.jpg)
Beginning of Kindergarten Total Literacy Proficiency Trends
0%
20%
40%
60%
80%
100%
Perc
ent
of Stu
dent
"On C
ours
e"
for
Gra
de 3
Readin
g P
rofici
ency
2004 45% 50% 46% 24% 83% 55%
2005 36% 52% 49% 25% 84% 56%
2006 45% 52% 51% 23% 88% 56%
American Indian
African American
Asian Hispanic WhiteAll
Students
![Page 33: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/33.jpg)
End of Kindergarten Percent of Students Reading at Least 10 Words per Minute
0%
20%
40%
60%
80%
100%
Perc
ent
of
Stu
dents
'02-03 40% 46% 59% 35% 71% 53%
'03-04 70% 67% 77% 54% 83% 71%
'04-05 68% 69% 77% 54% 84% 72%
'05-06 58% 68% 72% 43% 85% 68%
American Indian
African American
Asian Hispanic White All
Students
![Page 34: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/34.jpg)
Three Main Purposes for Value-Added Analysis in Minneapolis Public Schools
Rewarding schools, teams and individual teachers that show exceptional improvement in achievement
Identifying instructional strategies employed by “beat the odds teachers” to inform staff development
Assigning “beat the odds teachers” to high-need schools (not yet)
![Page 35: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/35.jpg)
The MPS Value-Added Model End of Kindergarten Reading Fluency =
– Beginning of Kindergarten Total Literacy +– Special Education status +– English Language Learner status +– Age group +– Gender +– Racial/Ethnic Status +– Free or reduced lunch status +– Kindergarten Teacher effects
![Page 36: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/36.jpg)
Kindergarten Teachers Who Beat the Odds
Teachers were identified empirically using value-added analysis– End of Kindergarten Assessment Results as
predicted from Beginning of Kindergarten, Poverty, ELL, Special Education, Gender, Age, and Racial/Ethnic background.
– Ten top teachers were interviewed and video taped
– These teachers worked last summer to produce an early literacy instruction video tape
![Page 37: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/37.jpg)
See Video Clips
![Page 38: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/38.jpg)
Reading Fluency One Year Follow-up (n = 96 Students Per Group)
0
10
20
30
40
50
60
70
Kindergarten First Grade
Matched Control Group Beat the Odds Teachers
Nu
mb
er o
f W
ord
s R
ead
C
orr
ectl
y p
er M
inu
te
![Page 39: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/39.jpg)
End of Grade 1 Oral Reading Scores for “Beat the Odds” Teachers
0
20
40
60
80
Number of Words Read
Correctly per minute
Posttest 62.4 51.3
BTO Teachers Control Group
![Page 40: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/40.jpg)
End of Grade 1 Percent of Students “On Course” for Grade 3 MCA Proficiency
0%
10%
20%
30%
40%
50%
Percent of Students
"on course" for Grade 3 Proficiency
Pretest 35% 35%
Posttest 47% 37%
BTO Teachers Control Group
![Page 41: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/41.jpg)
Teacher Videos Are Online at http://rea.mpls.k12.mn.us/
Beat the Odds Teachers!
Marie Olson: Lyndale
Michele Fisher: Hall
Carolyn Bergstrom: Shingle Creek
Anna Willams: Shingle Creek
Tim Yurecko: Lucy Laney
Monica Trent: Lyndale and
Whittier
Melissa (Schroeder) Burns:
Kenwood
Mary Ann Theisen: Lincoln
Penny Helvey: Lincoln
![Page 42: Aligning CBM With State Assessments and AYP David Heistad](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649e115503460f94afd1fd/html5/thumbnails/42.jpg)
Using CBM Benchmark Assessments to Identify Exceptional Teachers/ Exceptional Instruction
See instructional CD