from idea to implementation: getting effective practices into the classroom

From IDEA to Implementation: Getting Effective Practices

into the Classroom

The Illinois PBIS Network

Lucille Eber Ed.D.lewrapil@aol.com

State Director, IL PBIS Network

2008 OSEP Project Director’s Conference

State, Regional and Local Structures

Consistent statewide training and TA

Ensure readiness & commitment before training

Data disseminated and used

newsletters

website

presentations

trainings

reports Continuous Regeneration

Sustainability of PBIS in IL:

Implementation Model

Local Structures School-based teams District planning teams Coaches (internal & external)

Regional Structures Coordinators and Coaches Delivery of Training and Technical Assistance

Statewide Structure Development of Training & Technical Assistance Evaluation & Dissemination National Center Support

State-level Structure

Implementation Staff (25 FTE)• Technical Assistance Directors• Technical Assistance Coordinators

Administrative Support (9 FTE)• Training/Communications• Evaluation and Data Management• IT support

SYST

EMS

PRACTICES

DATASupportingStaff Behavior

SupportingDecisionMaking

SupportingStudent Behavior

OUTCOMES

Social Competence &Academic Achievement

Tier 3/Tertiary Interventions 1-5%•Individual students•Assessment-based•High intensity

1-5% Tier 3/Tertiary Interventions•Individual students•Assessment-based•Intense, durable procedures

Tier 2/Secondary Interventions 5-15%•Some students (at-risk)•High efficiency•Rapid response•Small group interventions• Some individualizing

5-15% Tier 2/Secondary Interventions•Some students (at-risk)•High efficiency•Rapid response•Small group interventions•Some individualizing

Tier 1/Universal Interventions 80-90%•All students•Preventive, proactive

80-90% Tier 1/Universal Interventions•All settings, all students•Preventive, proactive

School-Wide Systems for Student Success:A Response to Intervention (RtI) Model

Academic Systems Behavioral Systems

Illinois PBIS Network, Revised May 15, 2008. Adapted from “What is school-wide PBS?” OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. Accessed at http://pbis.org/schoolwide.htm

Positive Behavior Interventions & Supports:

A Response to Intervention (RtI) Model

Tier 1/Universal School-Wide Assessment

School-Wide Prevention Systems

Tier 2/Secondary

Tier 3/Tertiary

In

terv

entio

nAssessm

en

tIllinois PBIS Network, Revised May 15, 2008Adapted from T. Scott, 2004

SIMEO Tools: HSC-T, RD-T, EI-T

Small Group Interventions (CICO, SSI, etc)

Group Interventions withIndividualized Focus (CnC, etc)

Simple Individual Interventions(Simple FBA/BIP, Schedule/ Curriculum Changes, etc)

Multiple-Domain FBA/BIP

Wraparound

ODRs, Attendance, Tardies, Grades,

DIBELS, etc.

Daily Progress Report (DPR) (Behavior and Academic Goals)

Competing Behavior Pathway, Functional Assessment Interview,

Scatter Plots, etc.

IL PBIS Schools and Districts

PBIS Schools Trained & Implementing

394444

520587

654

800

96 115 134 151 170 200

0

100

200

300

400

500

600

700

800

900

Year 5 6/03

Year 6 6/04

Year 7 6/05

Year 8 6/06

Year 9 6/07

Year 106/08

num

be

r of sc

ho

ols

Illinois PBIS Schools

74%

14%

12%

83%

12%

5%

50%

60%

70%

80%

90%

100%

% o

f stu

de

nts

with

OD

Rs

Partia lly Implementing

(n=58)

Fully Implementing (n=141)

0-1 ODRs 2-5 ODRs 6+ ODRs

Mean Percentage of Students with Major ODRs 2006-07, Statewide

The differences between fully and partially implementing schools were statistically significant in all three levels of ODRs

(0-1 ODR, Mann-Whitney U=3035.0, p=0.004; 2-5 ODR, Mann-Whitney U=3050.0, p=0.005; 6+ODR, Mann-Whitney U=3062.0, p=0.005).

Comparison of Partial & Fully Implementing

Schools

on Suspensions/Expulsions FY07 per 100 Students

11.2

5.7

0

2

4

6

8

10

12

Partial (n=58) Full (n=140)

Implementation

Num

be

r o

f su

spe

nsio

ns

pe

r 10

0

stud

ents

Illinois PBIS Schools

Comparing School Safety Survey

Partial vs. Full Implementation

46 41

7378

0

20

40

60

80

100

Partial (n=20) Full 80/80 (n=62)

Partial vs Full Implementation

Perc

enta

ge o

f Risk

and

p

rote

ctio

n fa

cto

r

Risk factor Protection factor

Illinois PBIS Schools

Illinois PBIS Schools

59.3

68.4

54

56

58

60

62

64

66

68

70

Partial (n = 89

schools)

Full (n = 120

schools)

Level of PBIS Implementation

Pe

rce

nta

ge

of st

ud

ents

The difference between the two types of schools was significant (t=3.72, df=159, p<0.001).

Illinois 2005-06 Proportion of Students who Meet

or Exceed Third Grade ISAT Reading Standard

Illinois PBIS Schools

65.6

86.6

0

10

20

30

40

50

60

70

80

90

100

Partial (n = 160

schools)

Full (n = 157 schools)

Level of PBIS implementation

Perc

en

tag

e o

f sc

ho

ols

tha

t m

et

AYP

Findings suggest that fully implementing PBIS schools met AYP at a significantly higher percentage than partially implementing schools (χ2=19.17, df=1,

p<.001).

Illinois 2005-06 Proportion of Schools

that Met AYP

Small Group & Individual Interventions Rated "High" or

"Very High" in Fully & Partially Implementing PBIS

Schools 2006-07

142

42

0

50

100

150

Fully Implementing

Schools (n= 70 schools)

Partially Implementing

Schools (n= 24 schools)

Level of implementation

Nu

mb

er o

f in

terv

en

tion

Illinois PBIS Schools

Summary of FY 2008 Office Referrals:Mean Number of Office Discipline Referrals

per SIMEO Student

6.3

3.6

0

1

2

3

4

5

6

7

8

Baseline Time 2

Office Disciplinary Referrals

N=44N=44

Summary of FY 2008 Office Referrals:Mean Number of Office Discipline Referrals

per SIMEO Student for Students with Same Data Points

2.27

6.3

3.35

0

1

2

3

4

5

6

7

8

Baseline Time 2 Time 3

Office Disciplinary Referrals

N=19N=19 N=19

Building Local Capacity: Coaches

Coaches are school personnel who have:

Fluency with systems & practicesCapacity to delivery high level technical

assistanceCapacity to sustain teams in efforts to

implement systems & practices

Redefine Roles of District-level Staff to Coach

Sustainability & Accountability

Hands-on technical assistanceGuide problem solvingLocal trainingTeam start-up & sustainabilityPublic relations/communicationsSupport local leadershipLocal coordination of resourcesProvide prompts & reinforcers

PBIS External Coach Expansion from FY05 to FY08

118

71 76

100

124

150

0

20

40

60

80

100

120

140

160

num

ber o

f ext

ern

al c

oache

s

Coach Role/Goals

• Provide information and technical assistance:– best practices– current research– funding sources

• Know and anticipate local needs and resources

• Keep teams focused/functioning

• Volume• Quality• Consistency• Dosage/practice• Layering Content from Tier to Tier• Systems, Data, Practices at all 3 Tiers

Ongoing Staff DevelopmentComponents to Consider:

A Multi-level Training Plan

Awareness/ReadinessUniversal/Tier 1Secondary/Tier 2Tertiary/Tier 3Coaches SeriesAudience Specific (H.S, Alt. schools, etc)

District Leadership Team TrainingsAdministrator Academies

Continuum of Support for Secondary-Tertiary Level

Systems1. Group interventions (BEP, social or academic skills

groups, tutor/homework clubs, etc)2. Group Intervention with a unique feature for an

individual student, (BEP individualized into a Check & Connect; mentoring/tutoring, etc.)

3. Simple Individualized Function Based Behavior Support Plan for a student focused on one specific behavior (simple FBA/BIP-one behavior; curriculum adjustment; schedule or other environmental adjustments, etc)

4. Complex Function-based Behavior Support Plan across settings (i.e.: FBA/BIP home and school and/or community)

5. Wraparound: More complex and comprehensive plan that address multiple life domain issues across home, school and community (i.e. basic needs, MH treatment, as well as behavior/academic interventions) multiple behaviors

3.8.08

Secondary Training Events

A Two Year Comparison

0

200

400

600

800

2005-06 2006-0701020

304050

Secondary Participants

Secondary Trainings

Tertiary Training Events

A Two Year Comparison

0

200

400

600

800

1000

2005-06 2006-070

5

10

15

20

25

30

Tertiary Participants

Tertiary Trainings

Content for integrated coaches training

across related initiatives (state-level)Systems, Data and Practices

LeadershipData based Decision MakingUniversal ScreeningProgress MonitoringStandards Aligned CurriculumEvidenced Based Practice3-Tiered Model of InterventionsDifferentiation for Academics and BehaviorTeam Based ImplementationSustainability

EX200 Objectives• Develop fluency with fidelity tools to

guild district planning• Develop fluency with data-based

decision-making for meeting the educational needs of all students

• Practice strategies to coach district PBIS Leadership Teams

Example from Advanced Coaches Training Course

Objective 2Develop fluency with data-based decision

making to meet the educational needs of all students

• District Data AuditThis data is currently available in

schools; locating, aggregating, and analyzing it for trends is the challenge that proves most rewarding when completed.

Illinois PBIS Individual School Data Audit

Purpose: To guide schools and districts in the collection, analysis, comparison and decision making for school improvement

Compares and Correlates Multiple Data Sets• Demographics • Special Ed Data• Academic Achievement • Discipline Data

Start with one year and add additional years to see trends

Serves as a compilation sheet for information that is available to districts

Demographic and Academic Data

• Enrollment Data: Fall Housing Report, SWIS, IIRC

• Retention, Drop out, Graduation, Suspension/Expulsion, ODR’s – building and district records

• Academic Data: ISAT Reports, Interactive School Report Card (iirc.niu.edu)

Special Education Data:

• What is percentage of total enrollment by ethnicity?• What percentage of each ethnic group has an IEP?• What is their EE continuum? (% of time outside of

General Education)• Does the data indicate an increase or decrease in

students referred/placed in Special Education?• Does the data indicate an increase or decrease in

least restrictive environment?

Beyond ethnicity – replace ethnicity with gender, English Language Learners (ELL), free and reduced lunch

How to Find Special Education Data

• Students referred: Building Records, possibly District Records

• Students with IEP’s and EE data with option of multiple reports: IEPOINT Data Base (District Special Education Director or Coop Director identifies who has access and who uses the final data products )

• Special Ed Profile District Trend Data http://webprod1.isbe.net/LEAProfile

Challenge: Finding and Organizing Data on Placements and EE

Use of Out-of-home-School Tool (OHSC-T):– Different people have the data; Sp. Ed. and

Gen. Ed. data separate in most districts;– Who has data on students in ‘short term’ safe

school placements?– Coding and other ways that ‘hide’ data makes

the task hard; For example, students in certain placements didn’t “show up” in data review

– Students in 100% Sp. Ed but in home school didn’t come up in data

Out of home tool: Data trends

• AA students overrepresented in EBD and in restrictive settings

• AA students underrepresented in Autism • Many kids in restrictive/placement have

moved buildings a lot• Large #s of youth “not qualifying” for IEP

but in ‘safe school’ • Students with other “label” may have

behavior problems (ex. Students with LD in separate setting)

• Placement of students in100% Special Education in home school viewed as “better” EE data (but it is not….)

District “X”: Out of Home School • Most out of home placements are BD eligibility

• Most are for high school and middle with high school being greatest

• All out of home school placements at elementary came from one elementary school with two being from the MI program. 

• The numbers of students going out of district has dropped dramatically.

• The numbers of students successfully transitioning back to their home school (from district off-site BD) has dramatically increased.

• Larger #’s of students 60% > in general ed instruction (see IEPoint Data and FACTS data)

Next Steps

• Continue to analyze data around elementary out of home school placements and build capacity in buildings to succeed with students in general education settings.

• Further review special education placement data:– Review trends of students in general education

60%. Or more to determine effectiveness of students returning from out of district?

Activity B: District Data Audit

Review the Data Audit for the District and the 2 Individual Schools. Process the data with the guide questions and suggest strategies for this district to improve the EE data.

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

Taking it to Scale

District Summit Outcomes

Participants will be able to:

• identify the need for, membership of, function and mission of the district leadership team.

• identify and analyze relevant data to determine academic and behavioral needs of students within the district.

• integrate district initiatives to work more efficiently and avoid duplication of services.

District Summit Outcomes

Participants will be able to:

• self-assess the status of district-wide PBIS support.

• develop a plan for long-term sustainability of the PBIS process in district schools.

Statewide Participation in District Summits, 2007 - 2008

40

277

58

370

0

50

100

150

200

250

300

350

400

# Teams # Participants

2007

2008

45% 34%

SIMEO Database (Systematic Information Management of Education Outcomes)

Technical Features: Database Development

online data collection and graphing database system for individual student receiving intensive level planning and supports

Results of Implementation of Wraparound within SW-PBS in

IL• Three year pilot• Enhance SOC wraparound approach

– data-based decision-making as part of wraparound intervention

• Development of strength-needs data tools• Use of web-based data system

Wraparound

0

5

10

15

20

25

30

FY 05 (N=18) FY 06 (N=26) FY 07 (N=26)

PBIS Network Staff School Social Worker PBIS Coach Other School Personnel

Shift in Responsibility for Individual Student

Data Management at Tertiary Demo Sites

IL PBIS Tertiary Demos

“Mary Ellen”

Home, School, Community Tool

Example of SIMEO-06Individual Student Data System

Using the Data to get to Strengths and Needs at Wrap meetings

1

1.5

2

2.5

3

3.5

4

Baseline 3 months 6 months

Controls Anger Has friends Gets along with children

1

1.5

2

2.5

3

3.5

4

Baseline 3 months 6 months

Controls Anger Has friends Gets along with children

Immediate & Sustainable Change Noted in Placement

Risk

1.3

1.78

1.5

1

1.25

1.5

1.75

2

Baseline Time2 Time 3

High Risk

Low/No Risk

(n = 19)

Wraparound-07

Ensuring Capacity at All 3 Tiers

• Begin assessment and development of secondary and tertiary tiers at start-up of universal– Assess resources and current practices

(specialized services)– Review current outcomes of students with higher

level needs– Position personnel to guide changes in practice– Begin planning and training with select personnel

• All 3 tiers addressed at all district meetings and at every training

District-wide Secondary/Tertiary

Implementation Process• District meeting quarterly

– District outcomes– Capacity/sustainability– Other schools/staff

• Building meeting monthly– Check on all levels– Cross-planning with all levels– Effectiveness of practices (CICO/BIP/Wrap, etc)

• Secondary/Tertiary Coaching Capacity• Wraparound Facilitators

System Data to Consider

• LRE– Building and District Level– By disability group

• Other “places” kids are “parked”– Alternative settings– Rooms w/in the building kids are sent

• Sub-aggregate groups– Sp. Ed.– Ethnicity

Ongoing Self–Assessment of Secondary/Tertiary

ImplementationBuilding Level:• IL Phases of Implementation (PoI) Tool • IL Secondary/Tertiary Intervention Tracking Tool• Sp. Ed Referral Data• Suspensions/Expulsions/Placements (ongoing)• Aggregate Individual Student Data (IL SIMEO data)• LRE Data trends• Subgroup data (academic, discipline, Sp. Ed. Referral, LRE, etc)

District Level:• Referral to Sp.Ed. Data• LRE Data (aggregate and by building)• IL Out-of-Home-School-Tracking Tool (multiple sorts)• Aggregate SIMEO data• Aggregate PoI Data

System Action Planning in Progress: A Tertiary Demo

DistrictStudents with history of retention

– Students who are retained 2 or more times have a 90% chance of dropping out

– Tertiary level plans should address if students should be put in grades with peers who are their chronological age (similar to supports implemented for students with IEP’s.

– Plans should be comprehensive and address skill deficits students may have

– Concerns were also discussed regarding how to plan for these situations when it is in the best interest of the student to be in a higher grade, but credits are missing at a Middle School or High School Level. District team needs to action plan around these issues

Resources at www.pbisillinois.org

• “Commitments for Success” (click on “Getting Started”)

• “IL PBIS Network FY09 Phases of Implementation (PoI) Rubric (click on “Recognition Process,” then “Criteria”)

• Detailed Course Descriptions (click on “training”)