determining informative student growth on the florida assessments for instruction in reading
DESCRIPTION
Determining Informative Student Growth on the Florida Assessments for Instruction in Reading. Yaacov Petscher, Ph.D. Director of Research Florida Center for Reading Research. Common Questions. How much growth should occur? What score type should be used for growth? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/1.jpg)
Determining Informative Student Growth on the Florida Assessments for Instruction in Reading
Yaacov Petscher, Ph.D.Director of ResearchFlorida Center for Reading Research
![Page 2: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/2.jpg)
Common QuestionsHow much growth should occur?What score type should be used
for growth?Why are students’ PRS scores in
Grade 1 decreasing?What do the Reading
Comprehension and Word Analysis Ability scores really tell us?
![Page 3: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/3.jpg)
Things to DiscussReview Goals and FAIR
Psychometrics Review of FAIR & Score Types
◦How they were derived◦2010-2011 changes to FAIR
Growth in FAIR subtestsComparison of Growth
![Page 4: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/4.jpg)
Goals in Assessment – K2What do we want to maximize in
a screen?◦Correct classification or base rates
Focus on correct classification◦Cost = higher false positives/false
negativesFocus on predictive power
◦Cost = Lower sensitivity
![Page 5: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/5.jpg)
Reliability of Broad ScreenIRT Precision Estimates for Kindergarten and Grade 1 Broad Screen Tasks by Time of AssessmentGrade Task AP 1 AP 2 AP 3
Kindergarten Letter Names 0.51* - -
Letter Sounds 0.87 0.86 -
Phonological Awareness - 0.86 0.87
Word Reading - - 0.86
Grade 1 Word Reading 0.86 0.86 0.86
*The Letter Sounds task was more reliable than Letter Names at AP 1; however, due to the restricted range for high risk, a policy decision was made to use the Letter Name task in order to better capture the floor of the distribution. Because the Broad Screen in G2 is a timed task, precision estimates are not reported; however, test-retest reliability was strong (.79-.84).
![Page 6: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/6.jpg)
Predictive Validity of Broad Screen
K 1st 2nd K 1st 2nd K 1st 2ndAP1 AP2 AP3
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
NPPPPP
![Page 7: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/7.jpg)
Goals of Assessment – 3-12What do we want to maximize?
◦Reliable estimate of student abilityComputer adaptive test (CAT)
◦Allows for individual test creation◦Limits form effects◦Difficult to teach to the test
Starting students in the CAT
![Page 8: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/8.jpg)
![Page 9: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/9.jpg)
FAIR-FCAT RelationshipFall Winter Spring
Grade RC Screen FSP RC Screen FSP RC Screen FSP3 0.72 0.71 0.74 0.72 0.75 0.734 0.69 0.74 0.73 0.74 0.74 0.745 0.70 0.75 0.73 0.76 0.73 0.766 0.73 0.74 0.72 0.74 0.72 0.747 0.71 0.72 0.69 0.72 0.69 0.728 0.71 0.76 0.71 0.76 0.71 0.769 0.69 0.73 0.67 0.73 0.67 0.73
10 0.69 0.75 0.67 0.74 0.66 0.74
![Page 10: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/10.jpg)
FAIR K-2 Broad ScreenEach grade has different tasks
◦Kindergarten◦Grade 1◦Grade 2
Probability of Reading Success (PRS)◦What does this mean?◦How can it be used?
![Page 11: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/11.jpg)
“The Probability of Reading Success (PRS) score predicts the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance. The PRS can be used descriptively to compare class, school, or district level performance from one AP to the next. “
![Page 12: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/12.jpg)
State Median PRS K-2
Grade AP 1 AP 2 AP3K 76 87 911 80 85 882 53 60 71
![Page 13: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/13.jpg)
Analyzing Student ProgressMake descriptive comparisons
◦<85% PRS Was there a change in PRS?
Yes – YAY! Did they shift zones?
No – Look at TDI information and examine progress
◦>=85% PRS Did the student remain the “green zone”? Grade 1 question…
![Page 14: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/14.jpg)
Previous Grade 1 PRSAP1 PRS AP2 PRS AP3 PRS0 .11 0 0.02 0 .011 .15 1 0.04 1 .022 .21 2 0.06 2 .033 .29 3 0.10 3 .064 .37 4 0.16 4 .125 .46 5 0.25 5 .206 .56 6 0.37 6 .337 .65 7 0.50 7 .498 .74 8 0.64 8 .659 .80 9 0.75 9 .7910 .86 10 0.85 10 .8811 .9012 .93
![Page 15: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/15.jpg)
New Grade 1 PRSAP1 PRS AP2 PRS AP3 PRS0 0.11 0 0.01 0 .011 0.14 1 0.02 1 .022 0.20 2 0.03 2 .033 0.27 3 0.05 3 .064 0.36 4 0.09 4 .125 0.46 5 0.17 5 .206 0.56 6 0.28 6 .337 0.66 7 0.44 7 .498 0.75 8 0.61 8 .659 0.82 9 0.76 9 .7910 0.86 10 0.86 10 .88
![Page 16: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/16.jpg)
RC Screen◦ Helps us identify students who may not be able
to meet the grade level literacy standards at the end of the year as assessed by the FCAT without additional targeted literacy instruction.
Mazes◦ Helps us determine whether a student has more
fundamental problems in the area of text reading efficiency and low level reading comprehension.
Word Analysis ◦ Helps us learn more about a student's
fundamental literacy skills--particularly those required to decode unfamiliar words and read accurately.
Purpose of Each 3-12 Assessment
![Page 17: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/17.jpg)
FAIR 3-12 Score TypesReading Comprehension - BS/PMT
FCAT Success Probability (FSP) Percentile Standard Score Lexile Ability Score and Ability Range FCAT Cluster Area Scores Ability Score (RCAS)
Maze - TDI Percentile Standard Score Adjusted Maze Score
Word Analysis - TDI Percentile Standard Score Ability Score (WAAS)
![Page 18: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/18.jpg)
PercentilesRaw score transformation that
indicates the rank of the student compared to others of the same grade◦Does not denote mastery (criteron-
referencing) but relative performance
![Page 19: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/19.jpg)
Standard ScoresStandardized scores are derived
from raw scores to compare one student’s performance on a test to the mean of all other students at that grade
Mean = 100, SD = 15, Range = 55-145
![Page 20: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/20.jpg)
Ability ScoresSimilar to standard scores!
......but different!Mean = 500, SD = 100, Range =
200-800
![Page 21: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/21.jpg)
Why we use Ability Scores
Tom Brady Raw Standard Percentile AbilityAP1 12 100 50th 500AP2 17 100 50th 550AP3 20 100 50th 600
![Page 22: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/22.jpg)
Reading Comprehension Mazes Word Analysis
AP Score PM score AP Score PM score AP Score PM score
student lexile score
student lexile score
%ile & SS
RCAS
Percentilerank
Percentilerank
FSP
Adj. Maze
SS SS
WAAS
![Page 23: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/23.jpg)
Why not FSP?FSP includes previous FCATDifferential calculation of FSPStudent may gain in FAIR
Reading but not change FSP
![Page 24: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/24.jpg)
Gain Score AnalysesSimple difference scoresOnly students who were in all APsOnly within the testing windowOnly for consistent grade
students
![Page 25: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/25.jpg)
Analyzing Student ProgressUnlike K-2 we have Ability Scores
◦Determine if score type is AP or PM scoreAP score
◦<85% FSP Was there a change in FSP?
◦>=85% FSP Did the student remain the “green zone”?
PM score◦Examine AS in light of state results◦Did ability score increase for the student?
◦
![Page 26: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/26.jpg)
Changes to RCASNew passages and linking
◦Range of scores for RCAS & Lexile changingRCAS 2009-2010
◦200-800RCAS 2010-2011
◦150-1000Uncapped Lexile 2009-2010
◦220L – 1735LUncapped Lexile 2010-2011
◦225L – 2105L
![Page 27: Determining Informative Student Growth on the Florida Assessments for Instruction in Reading](https://reader033.vdocuments.us/reader033/viewer/2022051518/5681628c550346895dd2f84d/html5/thumbnails/27.jpg)
Next StepsAnalyzing specific growth targetsIs there merit in knowing the gain
scores?Working with JRF to provide
guidelines to districts/schools