11

Student Longitudinal Student Longitudinal Growth ProjectGrowth Project

Jonathan WiensJonathan Wiens

Office of Assessment and Information Office of Assessment and Information ServicesServices

Oregon Department of EducationOregon Department of Education

22

Model DevelopmentModel Development

1/08 to 8/081/08 to 8/08 – ODE researches growth models, and – ODE researches growth models, and tests them with historic test datatests them with historic test data

3/08-9/08 3/08-9/08 – ODE shares models with internal and – ODE shares models with internal and external advisory committeesexternal advisory committees

9/08-11/089/08-11/08 – ODE, stakeholders, and technical – ODE, stakeholders, and technical consultant refine details of leading consultant refine details of leading

candidates, with stakeholder inputcandidates, with stakeholder input11/08-12/08 11/08-12/08 – ODE, advisors recommend the – ODE, advisors recommend the

“Student Centered” model “Student Centered” model12/08- 4/0912/08- 4/09 – ODE presents Student Centered – ODE presents Student Centered Growth model for wider review.Growth model for wider review.

33

Growth ModelsGrowth Models A Growth Model is a method of A Growth Model is a method of

measuring student learning over time. measuring student learning over time. Growth models can provide a more Growth models can provide a more

dynamic view of school effectiveness.dynamic view of school effectiveness. Growth models can allow schools receive Growth models can allow schools receive

credit for: credit for: – closing the Achievement Gapclosing the Achievement Gap– accelerating student learningaccelerating student learning– moving students closer to standardmoving students closer to standard

44

Individual Individual StudentStudent Growth Growth

The most detailed view of growth occurs at theThe most detailed view of growth occurs at thestudent level.student level.

Which years are a success for the student and for Which years are a success for the student and for

the school?the school?

* - The “Gap” is the difference between the student score and the cut score.* - The “Gap” is the difference between the student score and the cut score.

GradeGradeCut Cut

ScoreScoreStudent Student ScoreScore Gap*Gap* GrowthGrowth

33 204204 190190 1414 ----

44 211211 206206 55 1616

55 218218 216216 22 1010

55

The “Student Centered” ModelThe “Student Centered” Model

Students below standard in the previous Students below standard in the previous year are given a target score for the year are given a target score for the current year.* current year.*

Students whose score is at or above their Students whose score is at or above their target “meet growth.” target “meet growth.”

Targets are based on the student’s prior Targets are based on the student’s prior year test score.year test score.

– Students are expected to grow toward Students are expected to grow toward meeting standard.meeting standard.

– Lower performing students must exhibit Lower performing students must exhibit higher growth.higher growth.

* - does not apply to 10* - does not apply to 10thth grade. grade.

66

Sample Reading Growth TargetsSample Reading Growth Targets Growth targets will require the “gap” between a Growth targets will require the “gap” between a

student’s score and benchmark to decrease over time, student’s score and benchmark to decrease over time, generally by about 40% to 50% each year.generally by about 40% to 50% each year.

Score Score in 3in 3rdrd

GradeGrade

Target Target for 4for 4thth

Grade Grade

ExpectedExpectedGrowthGrowth

3rd 3rd Grade Grade “Gap”“Gap”

44thth Grade Grade “Gap”“Gap”

Student AStudent A 190190 203203 1313 -14-14 -8-8

Student BStudent B 195195 206206 1111 -9-9 -5-5

Student CStudent C 200200 209209 99 -4-4 -2-2

Student MStudent M 204204 211211 77 00 00

77

Required “Gap” closuresRequired “Gap” closuresThe gap closures below are based on growth in The gap closures below are based on growth in the meets scores over one- and three- year the meets scores over one- and three- year periods.periods.

Required Decrease Required Decrease in the “Gap”in the “Gap”

PreviousPreviousGradeGrade

Current Current GradeGrade ReadingReading MathMath

33 44 39%39% 44%44%

44 55 44%44% 43%43%

55 66 31%31% 25%25%

66 77 36%36% 33%33%

77 88 44%44% 40%40%

88

Growth From Grade 5

-10

-5

0

5

10

15

20

25

200 210 220 230 240 250

Growth to Grade 6 Growth to Grade 7 Growth to Grade 8

Median Reading Growth – Actual Data

99

Sample TrajectoriesSample Trajectories

The next charts show the trajectories of The next charts show the trajectories of students who students who exactly meet their growth exactly meet their growth targets each yeartargets each year..

The charts include the option for “growth The charts include the option for “growth toward a 236” for those meeting or toward a 236” for those meeting or exceeding (these are still draft and are not exceeding (these are still draft and are not used in accountability).used in accountability).

Four examples are given – one student Four examples are given – one student each at the Very Low, Low, Meets, and each at the Very Low, Low, Meets, and Exceeds.Exceeds.

1010

Sample Math Trajectories

185

195

205

215

225

235

3 4 5 6 7 8

Grade

RIT

Sco

re

Very Low Low Meets Exceeds Cut Scores

1111

Sample Math Trajectories - Showing "Gaps"

-20

-10

0

10

20

3 4 5 6 7 8

Grade

Ga

p B

etw

ee

n S

co

re a

nd

Cu

t-S

co

re

Very Low Low Meets Exceeds

1212

07-08 Statewide Results07-08 Statewide Results  

Grade

06-07 to 07-08 Growth 2007-08 Status “Status or Growth” 

  Students in Model

Number Meeting Growth

% Met Growth Students

Number Meeting % Met

Met Status or Growth %

MathMath

3 139 113 81% 39,888 31,576 79% 31,594 79%

4 37,900 23,493 62% 39,974 31,550 79% 33,653 84%

5 37,764 21,523 57% 39,751 31,254 79% 33,120 83%

6 38,046 20,810 55% 40,069 28,991 72% 31,817 79%

7 38,124 26,278 69% 40,027 30,568 76% 33,671 84%

8 38,101 20,389 54% 39,834 28,232 71% 29,950 75%

10 - - - 39,566 21,602 55% 21,602 55%

ReadingReading

3 135 106 79% 39,894 34,436 86% 34,445 86%

4 36,846 20,523 56% 39,976 34,022 85% 35,258 88%

5 37,539 16,494 44% 39,770 30,812 77% 31,930 80%

6 37,855 23,552 62% 40,078 30,230 75% 33,242 83%

7 38,055 24,285 64% 40,046 30,354 76% 32,614 81%

8 38,051 16,137 42% 39,830 26,813 67% 27,994 70%

10 - - - 39,792 26,660 67% 26,660 67%

1313

Math06-07 to 07-08 Growth 2007-08 Status “Status or Growth”

Students in Model

Met Growth

% Met Growth Students Met

% Met

Status or Growth %

All Students 190,074 112,606 59% 279,109 203,773 73% 215,407 77%

EconomicallyDisadvantaged 86,678 47,243 55% 124,742 77,527 62% 85,551 69%

Limited EnglishProficient 24,158 12,874 53% 34,379 17,354 50% 20,636 60%

Students with Disabilities 25,384 11,973 47% 37,521 16,893 45% 20,371 54%

American Indian/Alaskan Native 3,911 2,118 54% 5,722 3,601 63% 3,924 69%

Asian/Pacific Islander 8,996 6,077 68% 13,189 10,928 83% 11,287 86%

African American 5,241 2,904 55% 7,903 4,548 58% 5,115 65%

Hispanic 32,143 17,354 54% 46,183 25,763 56% 29,444 64%

White 131,945 79,543 60% 194,599 150,298 77% 156,547 80%

Multi-Racial/Multi-Ethnic 4,742 2,798 59% 6,874 5,158 75% 5,428 79%

1414

4th Grade Growth

0

20

40

60

80

100

180 190 200 210 220 230 240

3rd Grade Score

Pc

t M

ee

tin

g G

row

th

Math Reading 5 per. Mov. Avg. (Math) 5 per. Mov. Avg. (Reading)

Percent Meeting Growth Based on Prior Year Score

1515

195

207

216 217

204

211

218

222

Target 221

190

200

210

220

230

240

Grade 3Bobs Elementary School

2005-06

Grade 4Bobs Elementary School

2006-07

Grade 5Spruce Elementary School

2007-08

Grade 6Bobs Elementary School

2008-09

Sc

ale

Sc

ore

*

Student Scores Standard Target

Reading Longitudinal Student Growth Report – Mock-Up

1616

““School Centered” ModelSchool Centered” Model

Examines the effect of a student’s Examines the effect of a student’s prior year test score on the odds a prior year test score on the odds a student will meet in the current year.student will meet in the current year.

Suggested by American Institutes of Suggested by American Institutes of Research (AIR), which has worked on Research (AIR), which has worked on growth with two other states.growth with two other states.

1717

Percent Meeting in 4Percent Meeting in 4thth Grade Grade by 3by 3rdrd Grade Score Grade Score

Statewide Results 2006-07 to 2007-08Statewide Results 2006-07 to 2007-08

0

20

40

60

80

100

170 190 210 230

3rd Grade Score

% M

et in 4th Grade

Logistic Regression Actual

1818

Range of School EffectsRange of School Effects

Probability of Meeting in 4th Grade Reading based on 3rd Grade Reading Score

0%

20%

40%

60%

80%

100%

160 180 200 220 240

Third Grade Score in Reading in 2006-07

Pro

babi

lity

of M

eetin

g in

4th

G

rade

in 2

007-

08

Highest Ranked Average School Lowest Ranked

This chart shows the models highest and lowest performing This chart shows the models highest and lowest performing schools at 4schools at 4thth grade in 2007-08. Notice the wide range in grade in 2007-08. Notice the wide range in school effects.school effects.

1919

““Probability Spreadsheet”Probability Spreadsheet”The spreadsheet contains district level data. The spreadsheet contains district level data. ODE plans to release this data before the end of ODE plans to release this data before the end of the school year.the school year.

2020

Scatter Plot of Tipping Points by SES Rank in a Large Oregon District

200

205

210

215

0 200 400 600 800

SES Rank (1=Highest Poverty)

Tip

pin

g P

oin

t

4th to 5th Grade Math Tipping Point

2121

Report Card Report Card School Rating School Rating

FormulaFormula

2222

TimelineTimeline

Sept 24Sept 24 Preview of Detail sheets Preview of Detail sheets

Oct 1Oct 1 Preview of Summary sheets Preview of Summary sheets

Oct 15Oct 15 22ndnd preview preview

Oct 29Oct 29 3rd preview3rd preview

Nov 5Nov 5 Final district previewFinal district preview

Nov 10Nov 10 Public ReleasePublic Release

2323

Changes to the Report CardChanges to the Report Card

Only three overall rating categoriesOnly three overall rating categories– OutstandingOutstanding– SatisfactorySatisfactory– In Need of ImprovementIn Need of Improvement

Graduation Rates used instead of Graduation Rates used instead of dropout rates.dropout rates.

The growth model will be The growth model will be incorporated into the rating formula.incorporated into the rating formula.

2424

New Report Card ValuesNew Report Card Values

We reward schools forWe reward schools for– students meeting standardstudents meeting standard– students exceeding standardstudents exceeding standard– students showing growth students showing growth – closing the achievement gapclosing the achievement gap

We have been developing this model We have been developing this model since Summer 2008 with input from since Summer 2008 with input from key stakeholder advisory groupskey stakeholder advisory groups

2525

The New Achievement IndexThe New Achievement Index

Points awarded according to:Points awarded according to:– 133 points for Exceeds133 points for Exceeds

– 100 points for Meets100 points for Meets

– 100 points for Did Not Meet, but Met Growth100 points for Did Not Meet, but Met Growth

Use the same students as are used for AYP Use the same students as are used for AYP – includes extended assessmentsincludes extended assessments

– includes the 1% rule for extended assessmentsincludes the 1% rule for extended assessments

At high school we will use growth in school At high school we will use growth in school performance as the growth measure, and will performance as the growth measure, and will give partial credit for nearly meets and low.give partial credit for nearly meets and low.

2626

Overall RatingOverall Rating

Subgroups are not rated, only a holistic Subgroups are not rated, only a holistic school rating is given.school rating is given.

To address the Achievement Gap, a To address the Achievement Gap, a school’s Achievement Index is a weighted school’s Achievement Index is a weighted average the achievement of subgroups.average the achievement of subgroups.

Subgroups in the weighting include “All Subgroups in the weighting include “All Students” plus subgroups with an historic Students” plus subgroups with an historic statewide achievement gap.statewide achievement gap.

Low Attendance/Graduation and Low Attendance/Graduation and Participation can lower the school rating.Participation can lower the school rating.

2727

Detail Sheet PreviewDetail Sheet Preview

2828

Likely Impact on School RatingsLikely Impact on School Ratings

About half of the formerly “strong” schools About half of the formerly “strong” schools will be rated as “Outstanding”will be rated as “Outstanding”

““Low” and “Unsatisfactory” schools Low” and “Unsatisfactory” schools generally move to “Needs Improvement”generally move to “Needs Improvement”

The following table shows how schools would The following table shows how schools would have been rated in 07-08 if the new formula have been rated in 07-08 if the new formula had been applied.had been applied.

2929

Potential Changes – Impact on RatingsPotential Changes – Impact on Ratings

3030

Cohort Cohort Graduation Graduation

RateRate

3131

TimelineTimeline June 2009 -- The 0708 Informational Cohort June 2009 -- The 0708 Informational Cohort

Rate provided to districts.Rate provided to districts. Summer 2009 -- ODE will submit for approval Summer 2009 -- ODE will submit for approval

from the US Department of Educationfrom the US Department of Education Summer 2009 -- begin data cleanup for the Summer 2009 -- begin data cleanup for the

0809 cohort0809 cohort Spring 2010 -- 2008-09 Cohort rate will be Spring 2010 -- 2008-09 Cohort rate will be

reported reported Fall 2010 -- 2008-09 Cohort rate will be used Fall 2010 -- 2008-09 Cohort rate will be used

in AYP determinationsin AYP determinations

3232

What is the Cohort Rate?What is the Cohort Rate?

The Cohort Rate is the percent of students The Cohort Rate is the percent of students who receive a regular diploma within four who receive a regular diploma within four years of entering high school. years of entering high school.

The Cohort Rate tracks students over the The Cohort Rate tracks students over the course of 4 years starting with their first course of 4 years starting with their first year in high school.year in high school.

Each member of a cohort will be assigned Each member of a cohort will be assigned a final outcome at the end of the four a final outcome at the end of the four years.years.

3333

Data Sources and ProcessData Sources and Process

The Cohort rate uses Resident Institution The Cohort rate uses Resident Institution only.only.

ODE uses records from ADM, Membership, ODE uses records from ADM, Membership, Early Leavers, and High School Completers Early Leavers, and High School Completers to assign students to a probable cohort.to assign students to a probable cohort.

Districts confirm the cohort of the student. Districts confirm the cohort of the student. Final outcomes are determined from the Final outcomes are determined from the

most recent events in our databases.most recent events in our databases. Students are assigned to their most recent Students are assigned to their most recent

diploma granting institution.diploma granting institution.

3434

Who is included?Who is included? The The unadjusted cohort unadjusted cohort is comprised of any is comprised of any

student, enrolled in an Oregon public school by student, enrolled in an Oregon public school by the end of the 2007-08 school year, who first the end of the 2007-08 school year, who first entered high school in 2004-05 in any school entered high school in 2004-05 in any school inside or outside Oregon.inside or outside Oregon.

The The adjusted cohort adjusted cohort is comprised of the is comprised of the unadjusted cohort minus students who died, unadjusted cohort minus students who died, entered the home schooling system, or entered the home schooling system, or transferred out to a diploma granting institution transferred out to a diploma granting institution outside the Oregon Public Education system as outside the Oregon Public Education system as their final high school educational setting.their final high school educational setting.

3535

Sample Statewide DataSample Statewide DataOutcomeOutcome NumberNumber

Unadjusted CohortUnadjusted Cohort 5730257302

Inferred Transfers outInferred Transfers out 1109711097

Non-dropout leaversNon-dropout leavers 240240

Honorary DiplomaHonorary Diploma 435435

Adjusted CohortAdjusted Cohort 4549745497 PctPct

GraduatesGraduates 3306433064 72.7%72.7%

DropoutsDropouts 43434343 9.6%9.6%

ModifiedModified 833833 1.8%1.8%

GEDGED 15191519 3.3%3.3%

Adult HS DiplomaAdult HS Diploma 3333 0.1%0.1%

Continuing EnrollmentContinuing Enrollment 43744374 9.61%9.61%

Non-continuing, non-graduateNon-continuing, non-graduate 6464 0.1%0.1%

Alternate Credential or No DiplomaAlternate Credential or No Diploma 12671267 2.8%2.8%

3636

Federal RequirementsFederal Requirements

State must set a graduation GoalState must set a graduation Goal– This need not be 100%This need not be 100%

State must set yearly targets that State must set yearly targets that show significant progress toward the show significant progress toward the goalgoal

Minimum group size should not Minimum group size should not exceed the minimum group size for exceed the minimum group size for participation (40).participation (40).

Margin of Error is unlikely to be Margin of Error is unlikely to be approved.approved.

3737

Federal FlexibilityFederal Flexibility

AYP reports may average data over AYP reports may average data over two years, or use the most recent two years, or use the most recent year, whichever is higher.year, whichever is higher.

States are responsible for developing States are responsible for developing subgroup membership rules.subgroup membership rules.

Four-year rate can include summer Four-year rate can include summer graduates.graduates.

3838

Available OptionsAvailable Options

State may develop an extended-year State may develop an extended-year rate.rate.– It may not supplant the four-year rate.It may not supplant the four-year rate.– Targets must be more aggressive (goal Targets must be more aggressive (goal

is the same).is the same). Targets for individual schools may Targets for individual schools may

varyvary– States can implement a “Safe Harbor” States can implement a “Safe Harbor”

provisionprovision

3939

The following slides represent The following slides represent ODE leanings on the required ODE leanings on the required

policy decisions. policy decisions.

These are subject to revision.These are subject to revision.

4040

How to “Meet” on GraduationHow to “Meet” on Graduation

Schools/subgroups can meet on AYP Schools/subgroups can meet on AYP

Graduation through:Graduation through:– Meeting the target for the four-year rate; based Meeting the target for the four-year rate; based

on a two-year average or the most recent year; on a two-year average or the most recent year; oror

– Decreasing the % of students not graduating Decreasing the % of students not graduating by 10% from one year to the next; orby 10% from one year to the next; or

– Meeting the target for the Extended rate; Meeting the target for the Extended rate; based on a two-year average or the most based on a two-year average or the most recent yearrecent year

4141

Graduation Goal and TargetsGraduation Goal and Targets

Graduation Goal of 90%Graduation Goal of 90% Targets with a 2% increase from current Targets with a 2% increase from current

targets to the goal in 2020-21.targets to the goal in 2020-21.

Reporting Reporting YearYear

4-year4-year

CohortCohort5-year5-year

ExtendedExtended

2009-102009-10 68%68% 73.5%73.5%

Yearly Yearly IncreaseIncrease 2%2% 1.5%1.5%

2020-212020-21 90%90% 90%90%

4242

Minimum Cell SizeMinimum Cell Size

ODE recommends the minimum be a total ODE recommends the minimum be a total cohort size of 20 (an average of 10 for each cohort size of 20 (an average of 10 for each year).year).

Distribution of Single Year Cohort Size:Distribution of Single Year Cohort Size:

Less than 10: Less than 10: 29 schools*29 schools*

10 to 19:10 to 19: 30 schools30 schools

20 to 39:20 to 39: 48 schools48 schools

40 or more:40 or more: 208208* -- These schools are typically rated on attendance.* -- These schools are typically rated on attendance.

4343

Subgroup MembershipSubgroup Membership

Race/EthnicityRace/Ethnicity– Based on the last student record.Based on the last student record.

Special Education, Economically Special Education, Economically Disadvantaged and Limited English Disadvantaged and Limited English ProficientProficient– Based on whether the student was Based on whether the student was

classified in these subgroups at any time classified in these subgroups at any time during high schoolduring high school

4444

Contact InformationContact Information

For more information on the Growth Model For more information on the Growth Model or the Report Card, contact:or the Report Card, contact:

Jon Wiens at Jon Wiens at

jon.wiens@state.or.usjon.wiens@state.or.us

Phone: 503 947-5764Phone: 503 947-5764

or visit the Growth Model Web Page at:or visit the Growth Model Web Page at:http://www.ode.state.or.us/search/page/?id=2495http://www.ode.state.or.us/search/page/?id=2495