CDE’s Growth Model &Growth within CCSD

Assessment & Evaluation, October 2012

District Accountability Handbook CDE – School and District Accountability guidelineshttp://www.schoolview.org/RegionalTraining.asp SPF Technical Guidehttp://www.schoolview.org/PerformanceFrameworks.asp Video/training materials on SPF http://www.cde.state.co.us/media/training/SPF_Online_Tutorial/player.html

CDE RESOURCES

Creating Meaning:

How will I recognize “notable change” in performance over time?

Factors that Influence the “significance” of a score change

Many factors can influence the interpretation of a difference between two numbers. Leaders must consider all before ascribing noteworthy status to a change in a percent, mean, or median.

How Certain We Need To Be

Statistical conventions in scientific research often call for high levels of certainty (90%-99%) that require very large differences for “significance.” If we are satisfied being "fairly certain," then lesser changes can be regarded as “notable.”

Pre-Existing Differences Between Groups

Statistical comparisons begin with an assumption that groups to be compared are the same in every way except for the intervention. If two groups begin with very different characteristics, then differences in % P/A or growth may be due pre-existing difference.

Size of Comparison Groups

A difference of 5 percentage points can be significant when both groups being compared are large but insignificant when the groups being compared are small.

Multiple Points over Time

A 2% or 3% change may not be statistically significant from one year to the next. However, if percent values change by 2 or 3 points over several successive points in time, then the overall change may be significant.

How will I recognize "notable score change" over time?

General “rules of thumb” for identifying percent P/A changes that may be significant between a moderate to high level of certainty:

Most Classrooms

Most Elementary Grades

Most Middle School Grades

Most High School Grades

District Results

Approx.Number of Students

About 20-30 students

About 75-125 students per grade

About 400 students per grade

About 600 students per grade

About 4000 students per grade

Approx.Percent change

13-20 percentage points

6-12 percentage points

4-8 percentage points

2-5 percentage points

1-3 percentage points

These are approximate guidelines. The larger the difference, the more certain we can be that it is not due to chance.

School Performance Framework Indicators

Achievement

Percent Proficient and

Advanced• Reading

(TCAP, Lectura, and CoAlt)

• Writing (TCAP, Escritura, and CoAlt)

• Math (TCAP and CoAlt)

• Science (TCAP and CoAlt)

Growth

Actual Growth & Target Growth

• TCAP Reading, Writing and Math

• Median Student Growth Percentiles

• Adequate Median Student Growth Percentiles

Gaps

Growth Gaps

Median Student Growth Percentiles for disaggregated

groups:• Poverty (FRL)• Race/Ethnicity• Disability• English

proficiency• Below

proficient

Postsecondary and Workforce Readiness

Colorado ACT

Graduation Rate

Dropout Rate

SPF: PurposeTo define the type of plan your school must create for submission to CDEI. PerformanceII. ImprovementIII. Priority ImprovementIV. Turnaround

Performance Improvement Priority Improvement

Turnaround

All Cherry Creek Schools were assigned to either Performance or Improvement plans (most to Performance).

SPF Sample

SPF Sample• CELA growth is

added to the growth section.

• Distinguish between adequate growth targets and actual median percentile growth.

• The State defines minority students as “non-white.” District KPIs provide a better picture of growth for students of color.

Growth IndicatorKEY CONCEPTS FOR UNDERSTANDING GROWTH

Actual Growth and Adequate GrowthCatching Up and Keeping Up

For StudentsSGP – Student Growth Percentile

AGP – Adequate Growth Percentile target

For SchoolsMGP -Median Growth Percentile

Median Adequate Growth target

Office of Assessment & Evaluation

For each student, the State calculates a Student Growth Percentile (SGP) for the current year

The SGP is based on the comparison of a student’s growth to growth in his or her State “academic peer group”

SGPs are normative

Some students’ SGP calculations use TCAP/CSAP data from several years, while others reflect only 2 data points

STUDENT GROWTH PERCENTILES (SGP) INDIVIDUAL STUDENT GROWTH

School Median Growth Percentiles

Student Pctls Sorted Pctls Median Pctls4578993211915567431077

Median Growth for this school is 55

Search for the middle value…

Student growth percentiles

REVIEW

Basis for "% Meeting/Exceeding Target Gain":

Adequate Growth Percentile (AGP)Adequate Growth Percentiles: Are Students Growing Enough?

• For each student, the State projects growth paths in order to calculate an AGP for the current year.

• The AGP is based on the calculated projection of how much a student needs to grow.

Students Below Proficient need to grow enough to “Catch Up,” or become Proficient in 3 years or by Grade 10

Students Proficient or above need to grow enough to “Keep Up,” or remain Proficient or above for 3 years or until Grade 10

Calculating AGP Growth to Catch Up to Proficient

Each year for each student, the State projects the growth necessary for that student to score at or above Proficient in 3 years or by grade 10

N o t P r o fi c i e n t

P r o fi c i e n t7th grade 8th grade 9th grade 10th grade6th grade

76 is the minimum-this student’s adequate growth TARGET

808595 76

2010 2011 2012 20132009

Calculating AGP Growth for Proficient+ Students to Keep Up

Each year for each student, the State projects the growth necessary for that student to continue to score at or above Proficient for 3 years or until grade 10.

7th grade 8th grade 9th grade 10th grade6th grade

N o t P r o fi c i e n t

P r o fi c i e n t

2010 2011 2012 20132009

38251250

50 is the maximum -this student’s adequate growth TARGET

Pair & Share

What does the “Adequate Growth Percentile” calculation mean for classrooms with a high proportion of very Unsatisfactory or low scoring Partially Proficient students?

What does the AGP calculation mean for classrooms with high proportions of Advanced or high Proficient students?

CSAP/TCAP Growth on BObj Reports

• BObj growth reports offer administrators and instructional leaders a variety of ways to explore and disaggregate CSAP/TCAP growth results from the Colorado Growth Model.

• Most BObj growth reports are designed in a “crosstabular” form that allows the user to view results by row for students in each prior year’s CSAP/TCAP performance level (A, P, PP, US) and for all students (bottom row).

CSAP/TCAP Growth on BObj Reports

Low, Typical, and High Growth The percent of students with Student Growth Percentiles (SGPs) in Low, Typical, and High ranges, and at or above the 50th percentile (the State median or average).

Average Growth The median is the middle value in a rank ordered list of SGPs for this group of students. The larger the group of students, the more reliable the information from this statistic .

% Target Gain The percent of students who demonstrate Adequate Growth defined by CDE, or growth sufficient to Catch Up (reach Proficient in 3 years or by grade 10) or Keep Up (stay at or above Proficient for three years or until grade 10).

Student Growth Percentiles (SGP) from the Colorado Growth Model

tell us how a student grew compared to other students in the

State with the same CSAP performance starting point.

CSAP/TCAP Growth on BObj Reports

KEY PERFORMANCE INDICATORS

Understanding Cherry Creek School’s

KPI’s and New Racial/Ethnic Categories

The new federal racial/ethnic categories prompt a change in district reporting of results for our two racial/ethnic groupings (“White/Asian” and “Black/Hispanic/Native American”):

Students in the Multi-racial category will be grouped:

• White/Asian: Students with any combination of White, Asian, and/or Pacific Islander

• Black/Hispanic/Native American Indian: Any students with a racial/ethnic identification that includes Black and/or Native American

NOTE: Regardless of how many races are chosen, Hispanic students are not considered multi-racial

Achievement Status & Achievement

Growth

CSAP achievement status

on most recent tests

10 points

CSAP achievement growth

over 2 most recent consecutive tests

30 points

Equity Among Student SubGroups

CSAP achievement status differences

5 points

CSAP achievement growth differences

15 points

Post-Secondary Readiness

Colorado ACT performance~

most recent tests

10 points

High School graduation rates~ prior year

10 points45 total points

for growth

Key performance indicators KPI Reports

CCSD Key Performance Indicators (KPI)

•TCAP Status•TCAP GrowthExcellence

•TCAP Status Equity•TCAP Growth EquityEquity•ES/MS: 3rd Reading/ EXPLORE•HS: COACT, Graduation Rate

College Prep

CCSD KPI OverviewTotal points are summed across all elements. The final percent of points earned is the result of total points earned divided by the total points possible. For middle & elementary schools, the total number of points possible is 70. For high schools, the total possible is 80.

KPI Reports, Summary page

Excellence: TCAP GrowthThe State longitudinal growth percentile allows us to compare the average growth of our students with average growth for students in the State with similar starting points.

The objective is for CCSD schools to meet or exceed the State median growth percentile for students with similar performance starting points (i.e., median growth percentiles at or above 50).

Rubric* Rating Pts

>=60th pctl Exceeds 1550th - 59th pctl Meets 1035th - 49th pctl Approaching 5

<35th pctl Does Not Meet 0

Cherry Creek Schools Office of Assessment & Evaluation CSAP_Longitudinal Growth on Data Access System from CDE student data fileResults include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing, or were new to the school after October 1.

School N Median Pts N Median Pts N Median PtsX 1,000 50.0 10 998 53.0 10 1,003 57.0 10 30 66.7% 20.00 XY 1,093 56.0 10 1,114 55.0 10 1,111 55.0 10 30 66.7% 20.00 YZ 725 52.0 10 724 55.0 10 721 55.0 10 30 66.7% 20.00 Z

State 50

School

Reading Writing Math Total Rubric Pts

(45 Possible)

% of Poss Rubric Pts (Total 45)

Element Pts Earned

(Total Poss 30)

RUBRIC POINTS-15 points possible per content area, Total Possible=45. ELEMENT POINTS= % Rubric Pts x 30

KPI Reports

Note:Similarities & Differences

Excellence: CSAP Performance IndicatorHow is this KPI similar to or different from the SPF performance indicator? What areas are “red” or “blue” (weakest or strongest) on each report?

Excellence: CSAP Growth IndicatorHow is this KPI similar to or different from the SPF performance indicator?Note any similarities or differences in the degree to which your school meets or exceeds growth expectation on both systems. What areas are “red” or “blue” (weakest or strongest) on each report?

SPF, KPI Reports, Notes Page

The objective is for all student subgroups in CCSD to meet or exceed the State median growth percentile for students with similar performance starting points (i.e., median growth percentiles at or above 50).

Equity: TCAP Growth

Rubric* Rating Pts

>=60th pctl Exceeds 1550th - 59th pctl Meets 1035th - 49th pctl Approaching 5

<35th pctl Does Not Meet 0

Cherry Creek Schools Office of Assessment & Evaluation CSAP Longitudinal Growth on Data Access System from CDE student data fileResults include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing, or were new to the school after October 1.Calculations are included for all four ethnic groups regardless of numbers of students. Native American Indian student groups are not summarized due to very small N's across all grades & schools.Student program labels are taken from the CSAP test booklets and preprinted labels.

(Reading shown. Rubric Points totaled across Reading, Writing, and Math.)

School N

Median SGP

WHITE Pts N

Median SGP

ASIAN Pts N

Median SGP

HISPAN Pts N

Median SGP

BLACK Pts

Total RD Rubric Points

(60 Poss)X 602 49.0 5 82 55.0 10 138 51.0 10 173 48.0 5 30Y 809 54.0 10 108 67.0 15 90 55.5 10 83 55.0 10 45Z 225 48.0 5 40 60.0 15 196 54.0 10 261 52.0 10 40

State 50

READING RUBRI C POI NTS-15 points possible within each Student Subgroup, Total 90 Rubric Points Possible

Calculations included for all four ethnic groups regardless of numbers of students. Native American Indian student groups arenot summarized due to very small N's across all grades & schools. Student program labels are taken from the CSAP test booklets and preprinted labels.

X XY YZ Z

130 72.22% 10.83130 72.22% 10.83

115 63.89% 9.58

Total Rubric Pts Earned-All

Content & Ethnic Groups

% of Possible

Rubric Pts (Total 180)

Element Pts Earned

(Total Poss 15) SchoolSchool

TOTAL ELEMENT POINTSRUBRI C POINTS-15

points possible per content area & student subgroup,

Total Possible=180

ELEMENT POINTS= % RubricPts x 15

Calculations included for all four ethnic groups regardless of numbers of students. Native American Indian student groups arenot summarized due to very small N's across all grades & schools. Student program labels are taken from the CSAP test booklets and preprinted labels.

KPI Reports

District Performance Plan (DPP) District Performance Plan

Creating Meaning:

What types of information should I look for

on reports?

Multiple perspectives on results yield a complete summary picture

• Remember the fable of the 6 men in the dark trying to describe an elephant?

• What do we need to see in order to get a “complete picture” of summary performance and growth?

Levels of Summary Analysis and Views of Summary Results

Views of Results

Overall More Specific

Details

Levels of Analysis

Overall Conten

tStanda

rdsBenchmark

s

Levels of Summary Analysis:"Whole to Part”

Entire School

Grade Level

Classroom/ Student

Different views of TCAP results for a complete picture

TCAP BObj reports provide several “lenses” through which to examine school results. From different views, you can create a complete picture of summary school performance and growth.

Average/Median or overall %P/A.

Distributions.

Trends.

Disaggregations.

Comparisons

Overall ResultsAll Schools Change Report (pdfs emailed and on COLE)

and5-Year Performance Summary (BObj)

Results for Student Subgroups5-Year Ethnic Summary (BObj) and

Disaggregated Summary (BObj)

Standards & SubContent AreasStandards & SubContent Area Summaries

TCAP GROWTH Results on BObjA&E’s “Pick Three”

Overall medians over time for all schoolsPerformance for Subgroups

Overall medians, crosstabs, % in high, middle,

and low growth levels, some disaggregations

In what ways do the following report “picks” provide:1. Multiple Views Of Summary Results

Overall %P/A, Distributions, Trends, Disaggregations, Comparisons

2. Multiple Levels of Summary AnalysisOverall

Standards

Benchmarks

Levels of Analysis