including a detailed description of the colorado growth model 1
TRANSCRIPT
![Page 1: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/1.jpg)
Including a detailed description of the Colorado Growth Model
1
![Page 2: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/2.jpg)
OverviewReview Growth Model OptionsDetails on Colorado Growth ModelMajor technical questions
2
![Page 3: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/3.jpg)
Basic DefinitionsStatus describes the academic performance
of a student or group (a collection of students) at a single point in time.
Growth describes the academic performance of a student or group (a collection of students) over two or more time points.
A growth model is a collection of definitions, calculations, and rules that quantifies student performance over two or more time points and supports interpretations about students, their classrooms, their educators, and their schools.
3
![Page 4: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/4.jpg)
School-level “Growth” MeasuresYear to Year changes in percent met
safe harborGrade to Grade changes in percent met or
average scoresHow are my fifth graders doing compared to
my fourth gradersCohort growth over time in percent met or
average scoresHow has my 2009-10 third grade class
performed overtime
4
![Page 5: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/5.jpg)
Growth ModelsA growth model is a collection of definitions,
calculations, and rules that quantifies student performance over two or more time points and supports interpretations about students, their classrooms, their educators, and their schools.
Most growth models follow individual student growth, and these are the subject of discussion today.
5
![Page 6: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/6.jpg)
Common Growth ModelsGain ScoresTrajectoryCategoricalResidual GainProjectionStudent Growth PercentilesMultivariate
6
![Page 7: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/7.jpg)
Key Questions Growth ModelsWhat interpretation does the growth model
support Formative? School accountability? Teacher evaluation?
Program Evaluation?What is the statistical foundation?
Gain-based, enhanced status, multivariateWhat are required data features?
Vertical scale? Articulated cut scores? Large datasets? Factors beyond test scores?
What interpretations does the model support? Does aggregation at the school level make sense?
Does the model include growth to standard?What are common misinterpretations and threats
to validity?7
![Page 8: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/8.jpg)
Gain ScoresDefinition: Calculates the gain in a student’s
test score from year to year. Example: 5th grade math score – 4th grade math score
Requires: vertical scale.Strengths: very easy to calculate, very easy to
aggregatedWeaknesses: may not be very reliable, easier to
manipulate, poor vertical scale lessens value.School measure: average gainsOregon data: Gains vary by previous
performance levels, see next slide.8
![Page 9: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/9.jpg)
Gain Scores – Oregon Data
9
Growth From Grade 5
-10
-5
0
5
10
15
20
25
200 210 220 230 240 250
Growth to Grade 6 Growth to Grade 7 Growth to Grade 8
![Page 10: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/10.jpg)
Trajectory ModelDefinition: If a student continues on this path,
where will she be in the future (Oregon’s current model)
Requires: vertical scale useful, but not requiredStrengths: Simple to computeWeaknesses: assumes students make same
growth year after year, targets for student already meeting may be difficult to determine
School measure: percent of students “on track”Oregon data: see next slide.
10
![Page 11: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/11.jpg)
Percent of students “On Track”
11
![Page 12: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/12.jpg)
Categorical ModelDefinition: evaluates students transitioning
between performance levelsRequires: cut scores, benefits from articulated
cut scoresStrengths: useful when vertical scale not
available Weaknesses: coarse measure of student progressSchool measure: percent on track or weighted
index of students improving their performance levels
Oregon data: none12
![Page 13: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/13.jpg)
Residual Gain ModelDefinition: student gains are measured against
“expected gains.” This is an “enhanced status model.”
Requires: state level data, model to predict expected gains, does NOT require vertical scaling
Strengths: value added measure, statistically robust - many statistics can be derived from the measure (median, standard deviation, school variability)
Weaknesses: does not directly measure growth, growth to standard is an add-on
School measure: average residual gains.Oregon data: gains are based on the mean gain,
conditioned on the prior test score. See next slide.
13
![Page 14: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/14.jpg)
Residual Growth versus Status
14
![Page 15: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/15.jpg)
Projection ModelDefinition: uses a regression model to predict future
scores. (Oregon’s federal growth model pilot submission was a hybrid of this model and a trajectory model)
Requires: linear regression model, state level data. Strengths: empirically grounded: uses historic data
to help predict current student’s performance, can use data from multiple subjects, supports growth to standard
Weaknesses: assumptions, such a linear growth, are often untenable.
School measure: average predicted score, percent of students “on track”
Oregon data: I’ve looked at versions of this, but don’t have pretty data to present.
15
![Page 16: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/16.jpg)
Student Growth PercentilesDefinition: student growth compared to other
students with similar score histories, expressed as a percentile. It is an “enhanced status model”.
Requires: state data, large data sets, does NOT require vertical scale
Strengths: includes growth to standard algorithm, is value added
Weaknesses: complex modelSchool measure: median or mean growth percentile.Oregon data: see next slide and later in this
presentation.
16
![Page 17: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/17.jpg)
Growth Percentiles in Oregon
17
![Page 18: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/18.jpg)
The Colorado Growth ModelBasic AssumptionsSample DataAdequate GrowthMajor technical decisions
18
![Page 19: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/19.jpg)
Quick DescriptionStudents with two consecutive years of test
scores are provided with a growth percentile.Student growth is compared to “academic
peers”, who are students with similar score histories.
A student growth percentile of 60 means the student “grew” as much or more than 60 percent of students with similar test score histories.
Various percentiles of growth can be projected into the future to determine “Adequate Growth Percentiles”
19
![Page 20: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/20.jpg)
GlossaryAcademic Peers – students with the same or
similar test scores in the past.Student Growth Percentile (SGP) – the
growth of the student as compared to her academic peers
Adequate Growth Percentile (AGP) – the growth percentile that needs to be sustained in order for the student to meet or still meet standard in 3 years.
20
![Page 21: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/21.jpg)
Growth Percentiles in Oregon
21
![Page 22: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/22.jpg)
Sample SGP Output
22
Prior Grade
Current
Grade
Prior RIT
Current RIT
Growth
SGP AGP
5th 6th 195 200 5 16 80
5th 6th 209 208 -1 8 72
5th 6th 209 214 5 34 64
5th 6th 209 218 9 61 58
5th 6th 209 222 13 85 53
5th 6th 209 226 17 96 48
5th 6th 227 232 5 57 27
![Page 23: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/23.jpg)
Student Growth ReportStudent reports can include the following:Student test historyCurrent Growth percentile – is the growth
low, average, or highScores that correspond to 1st,35th, 65th and
99th percentile (for comparison)Projections into the future for various growth
percentiles
23
![Page 24: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/24.jpg)
Sample Student Report
24
![Page 25: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/25.jpg)
Aggregations at School LevelMedian SGP by subgroup
This is the recommendation by the SGP author, though others recommend mean
Median AGP by subgroupThis is the recommendation of the SGP author,
though others recommend meanSubgroups Making Adequate Growth
Comparison of the median SGP and the median AGP.
Percent meeting AGPThis is not in the proposed school ratings model.
25
![Page 26: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/26.jpg)
Draft School Level Growth Report
26
Median SGP Median AGP
Subject
Subgroup2011-
122012-
13Combine
d2011-
122012-
13Combine
d
MathAll
Students45 51 47 34 37 36
Math LEP 51 54 53 56 61 58
Reading
All Students
55 52 53 26 28 27
Reading
LEP 45 48 47 65 63 64
![Page 27: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/27.jpg)
Web-based reporting
27
![Page 28: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/28.jpg)
Now the ComplicationsSGPs -- The model can take into account
more than two years of data for each student
By using more than two years of data we may get a better estimate of the student’s growth.
See the next slide for examples
28
![Page 29: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/29.jpg)
SGPs – Higher Order Growth
29
Current Grade
Back 2 years RIT
Prior RIT
Current RIT
Growth
SGP Adjustment
7th 225 231 6 47
7th 206 225 231 6 77 +30
7th 211 225 231 6 69 +22
7th 216 225 231 6 57 +10
7th 221 225 231 6 45 -2
7th 226 225 231 6 33 -14
7th 231 225 231 6 17 -30
This data show how 3 years of test scores can affect growth percentiles.
![Page 30: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/30.jpg)
Focus and Priority SchoolsNew “rating” system that will identify focus
and priority schools. Growth is a very strong factor in the rating.
30
Draft Focus/Priority Identification Weighting
Elem/Middle
High
Achievement 25% 20%
Growth 50% 20%
Subgroup Growth 25% 10%
Graduation -- 35%
Subgroup Graduation
-- 15%
![Page 31: Including a detailed description of the Colorado Growth Model 1](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e215503460f94b0d53e/html5/thumbnails/31.jpg)
Details on the waiver: http://www.ode.state.or.us/go/nextgen
Details on priority, focus, and model schools: http://www.ode.state.or.us/search/page/?id=3742
Priority, Focus and Model School Detail Sheets: http://www.ode.state.or.us/search/page/?id=3749
Policy and Technical Manual: http://www.ode.state.or.us/wma/policy/accountability/nextgen2012/nextgenaccountabilitymanual2012.pdf
Jon WiensEmail: [email protected]
Contact Information and Links
31