performance based monitoring analysis system (pbmas… update 092017.pdf · sd (year 1) for 7...

PERFORMANCE BASED MONITORING ANALYSIS SYSTEM

(PBMAS)2017 OVERVIEW

KELLY JOSEPH

SPECIAL EDUCATION

ACCOUNTABILITY SPECIALIST

210-370-5664

[email protected] SEPTEMBER 20, 2017

mailto:[email protected]

of the 2017 PBMAS Manual

DESCRIPTION OF NEW INDICATORS

• These regulations require 98 separate indicators to evaluate districts’ data regarding (a) special education representation [49 indicators]; (b) disciplinary removals [35 indicators]; and © educational placements [14 indicators].

SIGNIFICANT DISPROPORTIONALITY PENALTIES

, any district that exceeds the established thresholds is required to: (a) provide for the review and, if appropriate, revision of the district’s policies procedures and practices; (b) allocate 15% of its PART B funds to be used for comprehensive coordinated early intervening services to serve children in the district, particularly, but not exclusively, children in those groups that were significantly over-identified; and (c) publicly report on the revision to its policies, procedures and practices.

DISCIPLINE INDICATORS IN DATA VALIDATION THIS YEAR

• However, the three discipline indicators from the 2016 PBMAS will need to be replaced with the five integrated

discipline indicators required to implement the new federal regulations. There is insufficient time for those

indicators to be developed and included with the 2017 PBMAS. Instead, the discipline indicators will be

previewed in the 2017 Discipline Data Validation system and then fully integrated into the 2018 PBMAS. The

following table summarizes the impact of the new federal regulations on the SPED PBMAS indicators.

of the 2017 PBMAS Manual

2017 PBMAS INDICATOR PREVIEW: SPED

Program Area and Indicator Number

Indicator Name 2016 PBMAS 2017 PBMAS

SPED #1(i-v)

Additional provisions pertaining to this indicator will be covered in the 2017 PBMAS Other System Components preview.

SPED STAAR 3-8 Passing Rate(M, R, S, SS, W)

PLs were assigned based satisfactory student performance and above.

RI but no SA Two years of data

available for analysis

Add SA. Three years of

data available for analysis

Copyright © Texas Education Agency 2017. All rights reserved.

7




SPED #2(i-v)


SPED YAE STAAR 3-8 Passing Rate(M, R, S, SS, W)

PLs were assigned based on satisfactory student performance and above.

No RI or SA Three years of

data available for analysis (two years for math)

Add SA. Three years of

data available for analysis


8

2017 PBMAS INDICATOR PREVIEW: SPEDProgram Area and Indicator Number


SPED #3(i-iv)


SPED STAAR EOC Passing Rate(M, S, SS, ELA)

PLs were assigned (including ELA) based on satisfactory student performance and above.

Changes to the cut points were implemented.

RI (except ELA) but no SA

Two years of data available for analysis (one year for ELA)

Add RI for ELA. Add SA (except

ELA) Three years of data

available for analysis (two years for ELA)


9




SPED #4 SPED STAAR Alternate 2 Participation Rate

PLs were assigned with RI and SA.

Three years of data available for analysis

No changes


10




SPED #6


SPED Regular Class ≥80% Rate (Ages 6-21)

PLs were assigned based on ages 6-21.

No RI or SA One year of data


SD RO by race/ethnicity was added.

Add RI. Discontinue SD

RO by race/ethnicity.

Two years of data available for analysis


11




SPED #5


SPED Regular Early Childhood Program Rate (Ages 3-5)

PLs were assigned with RI but no SA.

Two years of data available for analysis

Three years of data available for analysis


12



SPED #7


SPED Regular Class <40% Rate (Ages 6-21)

PLs were assigned based on ages 6-21.

No RI or SA One year of data


SD RO by race/ethnicity was added.

Revise the indicator based on final federal regulations under 34 CFR Part 300, issued by USDE on December 19, 2016.

Assign SD Year 1 PLs based on race/ethnicity.

Add RI to overall component, where two years of data are available.


13



SPED #8


SPED Separate Settings Rate (Ages 6-21)

N/A New indicator required by final federal regulations under 34 CFR Part 300, issued by USDE on December 19, 2016.

Assign SD Year 1 PLs based on race/ethnicity.

Overall component is Report Only.


14




SPED #9 SPED Annual Dropout Rate (Grades 7-12)

PLs were assigned with RI and SA.


No changes

SPED #10 SPED Graduation Rate



No changes


15




SPED #11


SPED Representation


One integrated indicator based on final federal regulations under 34 CFR Part 300, issued by USDE on December 19, 2016.

Assign SD Year 1 PLs based on race/ethnicity and disability category.


16




SPED #15 SPED Discretionary DAEP Placements


Second year of Report Only disproportionality rates

MSR was changed to NUM1 ≥30.

Discontinue the indicator.

New SPED discipline indicators based on final federal regulations under 34 CFR Part 300, issued by USDE on December 19, 2016, will be previewed in the 2017 PBM Discipline Data Validation System.


17




SPED #16 SPED Discretionary ISS Placements







18




SPED #17 SPED Discretionary OSS Placements







19

PBMAS Before Final Regulations USDE Final Regulations10 Indicators Evaluating Placement, Discipline, and Representation

98 Indicators Evaluating Placement,Discipline, and Representation

Range of Performance Levels Pass/Fail SD Thresholds


20 PBMAS VS. USDE FINAL REGULATIONS: SUMMARY OF MAJOR DIFFERENCES

PBMAS Before Final Regulations USDE Final RegulationsContinuous Improvement Planning (if PL 3)

Financial penalty if any one of the 98 indicators’ threshold missed; must review, and if appropriate, revise policies, procedures, and practices; must publicly report on the revision of policies, procedures, and practices



PBMAS Before Final Regulations USDE Final RegulationsIf racial/ethnic group or comparison group MSRs not met, then PL = Not Assigned.

If racial/ethnic group MSRs not met, then SD calculation not required. However, if comparison group MSRs not met, then Alternate Risk Ratio must be used to calculate SD.

All Students Comparison Other Students Comparison



PBMAS Before Final Regulations USDE Final Regulations

Disproportionality Rate Risk Ratio/Alternate Risk Ratio

Required Improvement if Improvement from Prior Year to Current Year Sufficient in Relation to a Subsequent Year

Reasonable Progress Only if Improvement in EACH of TwoConsecutive Prior Years in Relation to Current Year


23

PBMAS VS. USDE FINAL REGULATIONS: SUMMARY OF MAJOR DIFFERENCES



PBMAS Before Final Regulations USDE Final Regulations

In representation indicators, exclusions were applied for Hospital Class, State Supported Living Centers, Residential Care and Treatment Facilities, and Regional School for the Deaf.

Only court-ordered students placed in Residential Facilities or Group Homes can be excluded from representation indicators.

By July 1, 2018,states must identify districts that

are subject to the federal sanctions (based on any one of the 98

indicators).Copyright © Texas Education Agency 2017. All rights reserved.

25 TIMELINE

Two Options: A district that has SD for any of the 98 indicators in 2018 is

subject to the federal sanctions.

or

A district that has SD for the same indicator in both* 2017 and 2018 and does not meet Reasonable Progress is subject to the federal sanctions.


26 TIMELINE

*After 2018, this option can include three years.

2017 PBMAS 2018 PBMAS

SD (Year 1) for 7 <40% indicators

SD (Year 1) for 7 Separate Settings indicators

SD (Year 1) for 49 Representation indicators

SD (Year 1) for 35 Discipline indicatorso 2015-2016 Data

SD (Year 2)o 2016-2017 Data

SD (Year 1)o 2016-2017 Data

SD (Year 2) for each of 98 indicators

RP (Year 2) for each of 98 indicators

SD (Year 1) for each of 98 indicators


27DRAFT TIMELINE

2017 PBM Discipline Data Validation

Red font indicates districts are under federal sanctions.

2019 PBMAS 2020 PBMAS

SD (Year 3) = districts that exceeded the threshold for three consecutive years and did not meet RP in 2019

RP (Year 3) = districts that exceeded the threshold for three consecutive years but met RP in 2019

SD (Year 1 and Year 2 )

SD (Year 3) and (Year 4) = districts that exceeded the threshold for three or four consecutive years and did not meet RP in 2020

RP (Year 3 and Year 4) = districts that exceeded the threshold for three or four consecutive years but met RP in 2020

SD (Year 1 and Year 2)

Etc...


28

Red font indicates districts are under federal sanctions.

DRAFT TIMELINE

• Promote equity in IDEA.

• Help states meaningfully identify districts with significant disproportionality.

• Ensure children with disabilities are properly identified for services.

• Address the well-documented and detrimental over-identification of certain students for special education services.

• Source: Federal Register/Vol. 81, No. 243/December 19, 2016, p. 92376.


29 STATED PURPOSE OF USDE’S REGULATORY ACTION

• PBMAS has historically included three discipline indicators that evaluate the placement of students with disabilities compared to the placement of all students in:

• Disciplinary Alternative Education Programs (DAEP)• In-School Suspension (ISS)• Out of School Suspension (OSS)

• They will be discontinued beginning with the 2017 PBMAS, and 35 new discipline indicators will be developed as a result of the final regulations.


30 DISCIPLINE INDICATORS

Indicator Numbers Indicator Name1 – 7 Out-of-School Suspensions and Expulsions ≤10 Days

(for Seven Race/Ethnic Groups)8 – 14 Out-of-School Suspensions and Expulsions >10 Days

(for Seven Race/Ethnic Groups)15 – 21 In-School Suspensions ≤10 Days

(for Seven Race/Ethnic Groups)22 – 28 In-School Suspensions >10 Days

(for Seven Race/Ethnic Groups)29 – 35 Total Disciplinary Removals, Including ISS, OSS, expulsions, removals by

school personnel to an interim alternative education setting, and removals by a hearing officer(for Seven Race/Ethnic Groups)



We are unable to develop the 35 indicators for inclusion in the 2017 PBMAS.

Instead, we plan to develop them for inclusion in 2017 PBM Discipline Data Validation.

As part of that release, we will identify SD Year 1 districts (based on 2015-2016 data), SD Year 2 districts (based on 2016-2017 data), and SD Year 1 districts (based on 2016-2017 data).

All of those results will subsequently be incorporated into, and reported a second time, in the 2018 PBMAS.



After that, all the SD indicators will continue to be incorporated into, and reported through, each year’s PBMAS.

Additional information about the progress of the new discipline indicators will be shared at a future time.

See also DRAFT TIMELINE slides.



The federal regulations are very prescriptive. There are only four components where states have some degree of flexibility:

1. Consecutive Years2. Minimum Size Requirements3. Thresholds4. Reasonable Progress


34 LIMITED OPTIONS AVAILABLE UNDER THE FEDERAL REGULATIONS

States can decide whether the penalties kick in for districts based on one year, twoyears, or three years of SD for a particular indicator.

Using just one year of SD is the most punitive option.

Advantages of using multiple years of SD:◦ Less punitive to districts◦ Requires the SD to be identified beyond a single year◦ Allows certain districts to receive the benefit of Reasonable Progress, thereby avoiding

the SD determination for a particular year

As noted earlier, the maximum number of years available for the first round of SD determinations is two: 2017 and 2018.


35 CONSECUTIVE YEARS

States have the option to apply MSRs on the calculations’:◦ Two numerators (N1 and N2) and ◦ Two denominators (D1 and D2)

Please refer to Slide 72.


36 MINIMUM SIZE REQUIREMENTS

For the first calculation shown (Risk Ratio = 6.3)◦ N1 is 240 and D1 is 950 (i.e. the race/ethnic group)◦ N2 is 100 and D2 is 2,506 (i.e., the other students group)

Per the federal regulations, if the race/ethnic group MSR (N1 or D1) not met, then the district is not evaluated for SD.

However, if N1/D1 MSR are met but N2 or D2 MSR not met, then the STATE N2 and D2 must be used to evaluate the district for SD. This is called the Alternate Risk Ratio.



The federal regulations state that N1/N2 MSR of 10 and D1/D2 MSR of 30 are “presumptively reasonable.”

The federal regulations require states that chose MSRs that are not presumptively reasonable to include a detailed explanation of why the alternative MSRs chosen are reasonable and how they help ensure an appropriate analysis for SD.



The threshold refers to the risk ratio number that states will use to assign SD.

Any district that exceeds that threshold will be identified as SD (either Year 1, Year 2, Year 3, etc.)

If the state decides to implement Reasonable Progress (RP), a district may avoid the federal sanctions that would otherwise kick in (or continue) for a particular year - as long as it meets RP for all indicators where it exceeded the established thresholds.


39 THRESHOLDS

For the Representation Indicator, states may set different threshold for different disability categories but may not set different thresholds for different racial/ethnic groups.

States may also set different thresholds for the Placement Indicators and the Discipline Indicators – but again may not set different thresholds for different racial/ethnic groups.

Based on stakeholder input, TEA proposed setting the same threshold for all Representation and Placement indicators in its 2017 PBMAS Manual rule adoption.


40 THRESHOLDS

In PBMAS, we were transitioning to a disproportionality rate of 100% or higher to identify SD. This is equivalent to a risk ratio of 2.0.

In early 2017, TEA provided stakeholder groups with modeling data for the Representation and <40% indicators based on thresholds of 2.0, 2.5, 3.0, and 3.5.

Based on stakeholder input, TEA proposed a threshold of 2.5 for the Representation, <40% , and Separate Settings indicators in its 2017 PBMAS Manual rule adoption.


41 THRESHOLDS

TEA has not yet proposed any thresholds for the 35 new discipline indicators required by the federal regulations.

Development on those indicators will occur in the fall.

See Slides 76 and 77.


42 THRESHOLDS


43

THRESHOLDSIndicator Category Allowable Thresholds

Representation• (7 indicators – one for each race/ethnic group)

1 threshold

Representation • (42 indicators – 6 disability categories x 7

race/ethnic groups)

Up to 6 (1 for each disability category)

Placement• (7 <40% indicators – one for each race/ethnic

group)

1 threshold

Placement• (7 separate settings indicators – one for each

race/ethnic group)

1 threshold

Discipline• (35 indicators – 5 disciplinary removals x 7

race/ethnic groups)

Up to 5(1 for each type of disciplinary removal)

§300.647(d)(2): A State is not required to identify a district for significant disproportionality until…the district has exceeded the risk ratio threshold and has failed to demonstrate reasonable progress.

Reasonable Progress is lowering the risk ratio in each of two consecutive prior years.

The State does not have the option to postpone a finding of SD if the district has only achieved a decrease over a multiple year period.


44 REASONABLE PROGRESS


45 REASONABLE PROGRESSExample SD Year 1 SD Year 2 SD Year 3 RP?

District A 4.9 4.3 3.6 Yes

District B 4.9 3.6 4.3 No

• In this example, the risk ratio threshold is >3.0.• Both districts started at the same place two years ago (SD Year 1).• Both districts exceed the threshold in the current year (SD Year 3).• Both districts reduced their risk ratio between SD Year 1 and SD Year 3.• However, per federal regulations, District B does not get RP because

its ratio increased from 3.6 to 4.3 between SD Years 2 and 3.

More information, including a proposal for calculating Reasonable Progress, will be shared at a future time.

We believe our PBMAS Required Improvement calculation is a valid and meaningful calculation, and we will continue to explore how/if we can incorporate it into a Reasonable Progress calculation.

Based on the parameters of federal requirements, however, it is unlikely that as many districts will receive RP as typically receive RI in PBMAS.


46 REASONABLE PROGRESS

Separate Handouts:

Sample District Report Guide to Examples

SD Excerpt #1 SD Excerpt #2


4

2017 PBMAS SAMPLE DISTRICT REPORTS

2107 PBM TRAINING SESSIONS (ESC20)

• Performance-Based Monitoring Analysis System (PBMAS)

October 26, 2018 Session being rescheduled

(session # 42810)

• Performance-Based Monitoring Analysis System (PBMAS)

November 9, 2017 8:30am-3:30pm

(session # 43081)

ACCOUNTABILITY MONITORING INTERVENTION GUIDANCE AND RESOURCES

District IR and PBMAS Pacing Guide found on the page linked below

http://tea.texas.gov/Student_Testing_and_Accountability/Monitoring_and_Interventions/Program_Monitoring_and_Interventions/Accountability_Monitoring_Intervention_Guidance_and_Resources/

2017 PBMAS Manual found on the page linked below

http://tea.texas.gov/pbm/PBMASManuals.aspx

http://tea.texas.gov/Student_Testing_and_Accountability/Monitoring_and_Interventions/Program_Monitoring_and_Interventions/Accountability_Monitoring_Intervention_Guidance_and_Resources/

http://tea.texas.gov/pbm/PBMASManuals.aspx

performance based monitoring analysis system (pbmas… update 092017.pdf · sd (year 1) for 7...

Documents