2012 traditional spf

2012 Traditional SPFBackground & Measures

September 17, 2012

2

Purposes of the SPF

1. Provides a body of evidence related to student and school performance

2. Is the basis of school accreditation ratings required by statute

3. Aligns district goals, state requirements, and federal mandates

4. Provides information for teacher and principal compensation systems

3

Indicator Weights: 2011 vs 2012Traditional Schools

4

New Methodology To DetermineSimilar SchoolsBackground

The SPF provides information on how each school performs relative to similar schools in the district.

Similar school clusters were previously based on only % FRL and % ethnic minority.

2012 Methodology Shift from %FRL and % ethnic minority cluster model to allow inclusion of additional demographic variables. Each school will receive a “School Characteristics Indicator” calculated using the weights below:

FRL (40%) + ELL (20%) + SpEd (20%) + Mobility*(20%)

Schools are then rank-ordered by Ed Level and compared with 10 schools that are closest to them (i.e., clusters are customized for each school).

*Mobility is defined as the total number of students who entered or left the school after 10/1 divided by the number of students in the school as of 10/1.

5

Similar Schools Calculations Example

School Name % FRL % ELL % SpEd % Mobility School Characteristics Indicator

School A 30.35% 16.47% 7.06% 13.00% 19.45School B 37.82% 14.26% 6.34% 20.00% 23.25School C 34.03% 36.39% 5.76% 7.00% 23.44School D 49.19% 10.93% 9.92% 14.00% 26.65School E 51.92% 21.39% 5.12% 4.75% 27.02School F 47.92% 17.25% 12.14% 12.00% 27.45School G 52.22% 17.21% 8.32% 10.83% 28.16School H 50.86% 17.08% 15.55% 12.00% 29.27School I 57.02% 24.47% 13.83% 9.00% 32.27School J 56.98% 20.16% 8.91% 18.99% 32.40School K 62.50% 13.78% 13.78% 14.00% 33.31School L 60.57% 26.49% 7.19% 15.00% 33.97

Scho

ol F

’s Cl

uste

r

FRL(40%) + ELL(20%) + SpEd(20%) + Mobility(20%) = School Characteristics Indicator

6

2012 SPF Indicators & Measures: Growth and Status

6

Indicators Measures

1. Student Progress Over Time - Growth

1.1a-c Median Growth Percentiles (MGPs)

1.2a-c* MGPs Compared to Similar Schools (FRL+ELL+SpEd+Mobility)1.3a-c Catch Up Growth

1.4a-c Keep Up Growth

1.5a-c Continuously Enrolled Growth (ES & MS only)

1.6 COAlt Growth

1.7a-c* CSAP/TCAP Subgroup Growth (MGPs for each subgroup)

1.8a-c* CSAP/TCAP Subgroup Growth Comparison (Compare focus and reference groups MGPs)

1.9* Students w/Disabilities Subgroup Growth Comparison (comparison w/state)

1.10* CELA MGPs

2. Student Achievement Level - Status

2.1a-d CSAP/TCAP % Proficient or Above

2.2a-d* CSAP/TCAP % Proficient or Above Compared to Similar Schools (FRL+ELL+SpEd+Mobility)

2.3a-c* CSAP/TCAP Subgroup Status (comparison w/district standard)

2.4* CSAP/TCAP Students w/Disabilities Subgroup Status Comparison (comparison w/state)

2.5 CSAP/TCAP % Advanced2.6 CELA % at Level 52.7* DRA/EDL % At Grade Level or Above (current year data only)

*measure modified from 2011 SPF

7

Status & Growth Indicators:CSAP % Proficient or Above

• Two separate measures:o Percentage of students “Proficient” or “Advanced”

on Reading, Writing, Math, and Scienceo Percentage of students “Proficient” or “Advanced”

(based on the tests above) compared to similar schools in the district

o Lectura and Escritura are included in this measure.

8

Status & Growth Indicators:Gap STATUS Measures

BackgroundThe SPF previously assessed how well four subgroups (FRL, Minority, SpEd, and ELL) at each school compared to district reference groups (e.g., FRL students at a school vs. Non-FRL students district-wide).

Gap measures were reported by content area (reading, writing, math, and science).

2012 Measures Measure 2.3 a-c. CSAP/TCAP Subgroup Status: What percent of FRL, Minority, and ELL

students are proficient or advanced in relation to district standards (same cut points used for measure 2.1a-d) rather than reference groups (non-FRL, non-Minority, and non-ELL)?

Measure 2.4. Students with Disabilities Subgroup Comparison: What percent of students with disabilities are proficient or advanced in relation to the percent of students with disabilities state-wide (i.e., no longer comparing SpEd vs non-SpEd)?

Reporting: Measures will be reported by subgroup rather than content area.

9

Status & Growth Indicators:% CSAP Advanced

• Measures the percentage of tests scored at “Advanced” on the CSAP in relation to all tests administered at a particular school

10

Status & Growth Indicators: % Above Proficient on CELA

• The Colorado English Language Assessment (CELA) measures the speaking, listening, reading and writing skills of English language learners.

• Measures the percentage of tests scored Above Proficient on the CELA in relation to all tests administered at a particular school

11

Status & Growth Indicators:DRA2/EDL2 Status

BackgroundPrior to the spring of 2012, DPS assessment guidelines directed teachers to score the Oral Reading Proficiency of the DRA2/EDL2 in a way that was incongruent with specific directions in the DRA2/EDL2 Administration Manual. This was corrected for 2012 resulting in scores that may not be comparable with prior years.

2012 Measure Measure 2.7. DRA/EDL % At Grade Level or Above: The status measure will only be based on 2012 scores (i.e., the two-year rubric will not be applied). The two-year rubric will be reinstated for the 2013 SPF.

12

Status & Growth Indicators:Growth Percentiles• In order to receive a growth percentile, students need a valid

English CSAP/ TCAP score over two consecutive years with a typical grade level progression (e.g., third grade to fourth grade)

• Each student receives a growth percentile indicating how much growth they achieved in the current year compared to other students who earned similar scores in prior years. A growth percentile of 50 is considered “typical” growth.

• Every student’s growth percentile is then rank-ordered and the middle score, or median, for the population is identified. This is the median growth percentile (MGP).

• For accountability purposes (i.e., inclusion in the SPF, students need to have been enrolled in the same school since October 1 of the same school year.

13TEST YEAR

2009 2010 2011 2012

CSA

P (M

ATH

) SC

ALE

SC

OR

E

200

400

600

399

440

374

650

Status & Growth Indicators:Growth Percentiles cont.

360308

413

99th Percentile

50th Percentile

25th Percentile

8th Percentile

78th Percentile598

14

Status & Growth Indicators:Growth Percentile cont.

15

Status & Growth Indicators:Catch-Up & Keep-Up Growth• Includes all students who took CSAP for two

consecutive years.• Different from the state’s Catch-Up and Keep-Up.

– State’s is progress needed to be proficient in 3 years or by 10th grade.

– DPS Catch-up: the percentage of students transitioning from a lower to higher performance level from one year to the next.

– DPS Keep-up: the percentage of students staying in the proficient and advanced categories or moved from proficient to advanced.

• This measure is limited to CSAP Reading, Math, and Writing.

16 16

Status & Growth Indicators:Catch-Up & Keep-Up Growth cont.

2012

2011Unsatisfactory

Partially Proficient-

Low


High Proficient Advanced

Unsatisfactory Catch Up Catch Up Catch Up Catch Up


LowCatch Up Catch Up Catch Up


HighCatch Up Catch Up

Proficient Keep Up Keep Up

Advanced Keep Up

17

• Addresses the transient student population • Accounts for growth shown by continuously

enrolled students– 3 years in the same school – Uses median growth percentile as measure– Not applicable to high schools

Status & Growth Indicators:Continuously Enrolled Growth

18

• Similar to the concept of Catch-Up and Keep-Up

• Students moving from CSAP-A to taking CSAP are considered improving.

• 16 students are needed to be rated on this measure

Status & Growth Indicators:CoAlt/CSAP-A Growth

19

Status & Growth Indicators:Gap GROWTH Measures

BackgroundThe SPF previously assessed how well four subgroups (FRL, Minority, SpEd, and ELL) at each school compared to district reference groups (e.g., FRL students at a school vs. Non-FRL students district-wide) and determined whether those gaps were closing from one year to the next.Gap measures were reported by content area (reading, writing, math, and science).

2012 Measures Measure 1.7 a-c. CSAP/TCAP Subgroup Growth: What is the MGP of FRL, Minority, and

ELL students using the same cut points as measure 1.1a-c? Measure 1.8 a-c. CSAP/TCAP Subgroup Growth Comparison: How did the MGPs for FRL,

Minority, and ELL students compare to non-FRL, non-Minority, and non-ELL students in the same school?

Measure 1.9. Students with Disabilities Subgroup Growth Comparison: How did the MGPs for students with disabilities compare to the MGPs for students with disabilities state-wide?

Reporting: Measures will be reported by subgroup rather than content area.

20

Status & Growth Indicators:CELA Growth

BackgroundThe CELA growth measure was previously based on the percent of students moving up one or more performance bands. This favored schools who inherited low performing ELLs because it’s easier to move up from CELA 1 and 2 than it is to move up from CELA 3 and 4.

2012 Measure Measure 1.10. CELA MGPs: The CELA growth measure will now solely be based on median growth percentiles (MGPs).

21

2012 SPF Indicators & Measures: PSR Growth and Status

21

Indicators Measures

3. Post-Secondary Readiness Growth (high schools only)

3.1a-d* 10th Grade CSAP/TCAP to 11th Grade COACT Growth

3.2* CDE “best-of” Graduation Rate Change (“best of” from prior year vs same year for current year)

3.3* DPS 4-year Cohort Graduation Rate Change (first-time 9th graders only)

3.4 On track to Graduation Change

3.5* AP/IB/CE Enrollment Change (test taking or course enrollment)

3.6 AP/IB Test Taking Rate Change

3.7 AP/IB Test Passing Count Change

3.8 Concurrent Enrollment Passing Count Change

3.9a-c** College Remediation

4. Post-Secondary Readiness Status (high schools only)

4.1a-d Colorado ACT

4.2a-d* Colorado ACT Compared to Similar Schools (FRL+ELL+SpEd+Mobility)

4.3 CDE “best-of” Graduation Rate

4.4* CDE Graduation Rate Compared to Similar Schools (FRL+ELL+SpEd+Mobility)

4.5 On Track to Graduation

4.6* AP/IB/CE Enrollment (test taking or course enrollment)

4.7 AP and IB Test Taking Rate

4.8 AP and IB test Passing Rate

4.9 Concurrent Enrollment Passing Rate

4.10a-c** College Remediation

4.11a-c** College Remediation Compared to Similar Schools (FRL+ELL+SpEd+Mobility)

*measure modified from 2011 SPF**new measure for 2012 SPF

22

PSR Status & Growth Indicators:Colorado ACT

• The percentage of students who are “proficient” on Reading, Math, English, and Science.

• “Proficient” are benchmarks defined by ACT as “college ready”.– Reading = 21– English = 18– Math = 22– Science = 24

23

PSR Status & Growth Indicators:10th Grade CSAP to COACT

BackgroundBecause the COACT is only administered in 11th grade, the change (growth) measure was based on comparisons between two completely different student cohorts (e.g., 2011 11th grade vs. 2012 11th grade).

2012 Measure Measure 3.1a-d. 10th Grade CSAP/TCAP to 11th Grade COACT Growth:

What percent of 11th graders achieved typical growth from 10th grade (based on TCAP scores) to 11th grade (based on COACT scores)?

This measure will include reading, writing, math, and science.

24 24

PSR Status & Growth Indicators:CDE ‘best of” grad rate

• Each year, schools receive graduation rates from CDE for several different cohorts, or “class of”. The SPF uses the best rate of those available.

• Note there is a one year lag for this measure (i.e., graduation rates for the most recent school year are not yet available when the SPF is published in the fall)

4 Year 5 Year 6 Year 7 Year

Spring of 2008 49.2 52.1 53.8 53.9

Spring of 2009 52.8 52.9 53.3

Spring of 2010 49.3 51.5

Spring of 2011 51

Current Year

25

PSR Status & Growth Indicators:CDE “best of” graduation rate changeBackground

For the 2011 SPF, this measure compared the “best of” from the prior year to the “best of” for the current year.

2012 Measure Measure 3.2. CDE “best of” Graduation Rate Change: The rate is calculated by comparing the best of 4, 5, 6, and 7 (if available) rates from prior year to the same rate for current year. (e.g., If your “best of” from prior year is your 6-year rate, then the current year 6-year rate will be used to assess change – see example below).


Spring of 2007 51.1 52.2 52.7 53

Spring of 2008 49.2 52.1 53.8

Spring of 2009 52.8 52.9

Spring of 2010 49.3


Spring of 2008 49.2 52.1 53.8 53.9

Spring of 2009 52.8 52.9 53.3

Spring of 2010 49.3 51

Spring of 2011 51

Prior Year Current Year

26

PSR Status & Growth Indicators:DPS four year within school graduation rate change

BackgroundIn prior years, the DPS 4-year cohort graduation rate change measure was based on the percent of ninth graders in a school’s official October Count who graduated from the same school four years later. Ninth graders were defined based on credits earned, not the number of years in high school.

2012 Measure Measure 3.3. DPS 4-year Cohort Graduation Rate Change: What percent of first-time ninth graders graduated from the same school four years later? In addition, this measure will be based on the first school each student attended as a ninth grader in DPS, rather than the school associated with the official October Count.

27

PSR Status & Growth Indicators:On-Track to Graduation

• Are students on-track to where they should be based on the number of years since they first entered 9th grade? For example, having 60 credits (or DPS equivalent) by end of 9th grade is considered to be “on track”.

• This measure only includes grades 9-11 as 12th graders are addressed in the official graduation rates.

28

PSR Status & Growth Indicators: AP, IB, Post-Secondary Course Measures

• Post-Secondary Course Credit/IB Enrollment: percent of students who take at least one of these courses. (For schools that do not offer AP courses, AP tests taken will be counted as AP course enrollment.)

• AP and IB test taking rate: proportion of tests taken compared to courses taken.

• AP and IB test passing rate: proportion of tests with a passing score (3 for AP, 4 for IB) compared to tests taken.

• Post-Secondary Course Passing Rate: proportion of courses with a passing grade (C or above) compared to courses completed.

29

PSR Status & Growth Indicators:College Remediation – Status & GrowthBackground

One of the primary goals of DPS is to prepare students for success in post-secondary institutions. Some schools place more emphasis on lower college remediation rates than higher on-track to graduation rates.

2012 Measures Measure 3.9 a-c. College Remediation Growth: Are your college remediation rates for reading, writing, and math decreasing over time?

Measure 4.10a-c. College Remediation Status: What percent of your students require remediation in college for reading, writing, and math?

Measure 4.11a-c. College Remediation Status Compared to Similar Schools: What percent of your students require remediation in college in reading, writing, and math in relation to similar schools?

All of these measures are based on student-level data provided by the Colorado Department of Higher Education. As a result, only students who attend a Colorado institution (2-year or 4-year) the year immediately after high school are included.

30

2012 SPF Indicators & Measures: Student & Parent Engagement & Enrollment

30

Indicators Measures

5. Student Engagement & Satisfaction

5.1 Attendance Rate

5.2 Student Satisfaction5.3 Center-Based Program Offerings

6. Enrollment

6.1* Re-Enrollment Rate (end of year to October)

6.2** Enrolled Entire Year (elementary & middle school only)

6.3** Dropout Rate (high school only)6.4** Enrollment Change Bonus Points

7. Parent Engagement

7.1 Parent Satisfaction

7.2 Parent Response Rate

*measure modified from 2011 SPF**new measure for 2012 SPF

31

Student Engagement & Satisfaction:Attendance, Satisfaction, Center-Based Programs

• Attendance: Measures the school’s average attendance rate

• Student Satisfaction:– The Student Satisfaction Survey is administered annually to all

students.– Measures the school’s percentage of student satisfaction survey items

that received a positive response– Schools must have a minimum of a 50% student response rate in order

to earn points on this measure

• Center-based programs are worth up to 3 extra credit points (added to the total points earned but not in the total possible points).

32

Enrollment IndicatorRe-enrollment, enrolled entire year, dropout rate

BackgroundThe Re-Enrollment Indicator on the 2011 SPF only included one re-enrollment measure: % of students in the October count who are in the October count the following year

2012 Measures For elementary and middle schools, replace the current measure with:

Measure 6.1. Re-enrollment rate: What percent of students who were at your school at the end of the year returned the following year in relation to similar schools?

Measure 6.2. Enrolled Entire Year: What percent of your students who were in your October Count remained in your school until the end of the year in relation to similar schools?

For high schools, replace the current measure with: Measure 6.1. Re-enrollment rate: What percent of students who were at your school at the end of the year returned the following year in relation to similar schools? Measure 6.3. Dropout rate: What is our annual dropout rate as reported by CDE?

33

Enrollment IndicatorEnrollment Net Change Bonus Points

BackgroundSchools will receive bonus points based on net changes in enrollment as depicted in the graphic below.Numerator = Students in October Count still enrolled at the end of the year + students who enroll between October Count & March 1 and are still enrolled at the end of the yearDenominator = October Count Enrollment

2012 Bonus Points

Elementary Schools: +4% = 2 points +2% = 1 point

Middle Schools: +2% = 2 points positive change = 1 point

High Schools: +2% = 4 points positive change = 2 points

October Count March 1 End of

Year

34

Parent Engagement Indicator:Parent Satisfaction & Response Rate

• The Parent Satisfaction Survey is administered to all parents every April.

• Parent Satisfaction– Measures the school’s percentage of parent satisfaction

survey items that received a positive response.• Response Rates

- Measures the response rate of parents who filled out and returned a parent satisfaction survey

- Schools must achieve a 50% response rate to receive full points on this measure

35

Computation Process

Data Collection & Aggregation

Measure Points & Stoplight

Apply SPF Rubrics

Indicator Total Stoplight

Sum and Apply Cut-Offs

Overall TotalAccreditation Rating

Sum and Apply Cut-Offs

Level 1

Level 2

Level 3

Level 4

Exceeds Standard

Meets Standard

Approaching Standard

Does Not Meet Standard

36

Two-year rubric

Cut-points

Computation Process: Based on 2-Years of Data

1.1 a-c Median Growth Percentile: Was the school's CSAP median growth percentile at or above 50? 0. Does not meet standard

The median growth percentile was less than 35.

2. Approaching standard The median growth percentile was at or above 35 and less than 50. 4. Meets standard The median growth percentile was at or above 50 and less than 65.6. Exceeds standard The median growth percentile was 65 or higher.

Year 2 Year 1 0. Does not meet 2. Approaching 4. Meets 6. Exceeds 0. Does not meet 0. Does not meet 0. Does not meet 2. Approaching 2. Approaching 2. Approaching 0. Does not meet 2. Approaching 2. Approaching 4. Meets 4. Meets 2. Approaching 2. Approaching 4. Meets 4. Meets 6. Exceeds 2. Approaching 4. Meets 4. Meets 6. Exceeds

37

Computation Process: Based on 2-Years of Data

2012: Meets

2011: Approaching

Example: TCAP Median Growth Percentile

2012 Measure Rating: Approaching

38

Have a question???

Contact: [email protected]

2012 traditional spf

Documents