the importance of coaching in implementation of evidence-based practices

The Importance of The Importance of Coaching in Coaching in Implementation of Implementation of Evidence-based PracticesEvidence-based Practices

Rob HornerUniversity of Oregon

www.pbis.org

GoalsGoalsCurrent assumptions/research

about coaching

Define our experience with coaching in PBS implementation

Implications for building district capacity

Coaching DefinedCoaching DefinedCoaching is the active and iterative

delivery of: ◦ (a) prompts that increase successful

behavior, and ◦ (b) corrections that decrease unsuccessful

behavior.

◦ Coaching is done by someone with credibility and experience with the target skill(s)

◦ Coaching is done on-site, in real time ◦ Coaching is done after initial training◦ Coaching is done repeatedly (e.g. monthly)◦ Coaching intensity is adjusted to need

Outcomes of CoachingOutcomes of CoachingFluency with trained skillsAdaptation of trained concepts/skills

to local contexts and challenges And new challenges that arise

Rapid redirection from miss-applications

Increased fidelity of overall implementation

Improved sustainability Most often due to ability to increase coaching

intensity at critical points in time.

Training Outcomes Related to Training Components

Training OutcomesTraining Components

Knowledge of Content

Skill Implementation

ClassroomApplication

Presentation/ LecturePlusDemonstrationPlus PracticePlus Coaching/ Admin SupportData Feedback

10% 5% 0%

30% 20% 0% 60% 60% 5%

95% 95% 95%

Joyce & Showers, 2002

Coaching Coaching within SWPBS within SWPBS ImplementationImplementationContext:

◦9000 schools implementing SWPBS nationally

Defining the Role Internal vs External

Selecting CoachesTraining and support for coachesAssessing Impact

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Local Demonstration Schools

Active Coordination

BehavioralExpertise

Coaching vs. TrainingCoaching vs. TrainingCoaching involves active

collaboration and participation, but not group instruction.

◦Small group◦Build from local competence◦Sustainable

Who should be a coach?Who should be a coach?Internal vs External

Internal coaches are employed in the school where they provide support

External coaches are employed outside the schools where they provide support (e.g. by district, region, state).

Who should be a coach?Who should be a coach?Internal Coach External Coach

Advantages Knowledge of schoolStaff relationshipsRegular access

Independent Outside perspectiveMultiple schools experience

Disadvantages Conflicting rolesNarrow range of experiences

Limited knowledge of schoolLimited relationshipsLess frequent access

Who should be a coachWho should be a coachCoaching Competencies

Necessary PreferredParticipate in team training

Able to attend team meetings at least monthly

Effective working with adults

Knowledgeable about school operating systems

Professional Commitment

Knowledge about SWPBS

Knowledge about behavior support practices (targeted, individual)

Skilled in collection and use of data for decision-making.

What Coaches DoWhat Coaches DoWork with team during initial SW-PBS trainingMeet with new teams monthly on-siteTelephone/email contact as needed“Positive” nag

Self-assessment (EBS Survey, Team Checklist) Action planning Activity implementation On-going evaluation

School self-evaluation efforts State-wide Initiative evaluation efforts (SET)

Guide State-wide initiative Feedback to Taskforce

What Coaches DoWhat Coaches DoDissemination of outcomes and

effectsSWIS Facilitation

◦Implement and support use of data-based decision making.

Commitment of CoachesCommitment of CoachesTeam Support

◦ First Year (1-2 teams) (participate in training and planning)◦ Second Year (Maintain initial teams, start 3-5 teams)◦ Future Years (10-15 teams total)

FTE commitment◦ 20-50%

Roles/Background◦ Behavior Specialists, Special Education

Teachers◦ Consultants, Administrators◦ School Psychologists, Counselors, Social

Workers

Guiding Principles for Effective Guiding Principles for Effective CoachingCoaching

Build local capacity Become unnecessary…but remain available

Maximize current competence Never change things that are working Always make the smallest change that will have the biggest

impactFocus on valued outcomes

Tie all efforts to the benefits for childrenEmphasize Accountability

Measure and report; measure and report; measure and report.Build credibility through:

(a) consistency, (b) competence with behavioral principles/practices, (c) relationships, (d) time investment.

Precorrect for success

Specific ExpectationsSpecific Expectations◦ Attend and participate in team training◦ Meet with your team(s) at least monthly

Provide technical assistance as needed◦ Monitor and report on team efforts

Team Checklist EBS Survey/ SET/ ISSET Annual Profile/Summary Data

◦ Present on School-wide PBS at district, state, national forums.

◦ Assist district to build capacity for sustained implementation (re-define your role over time)

◦ Meetings with Coordinator and Taskforce for purposes of state-wide planning

Assist Teams in Using Data for Assist Teams in Using Data for Decision-makingDecision-making

Using Team-Checklist and EBS Survey data for Team Action Planning

Using SET/ TIC data for evaluationUsing ODR/ Academic (ORF) data

for assessment, planning and reporting.

Keeping faculty involved through regular data reporting.

ExamplesExamplesIllinoisNorth CarolinaMichigan

PBIS in Illinois

July 17, 2008Developing Local Systems of Care for Children and Adolescents with

Mental Health Needs and their FamiliesTraining Institutes

Nashville, TN

Lucille Eber Ed.D.IL PBIS Network

PBIS Schools Over Ten Years: PBIS Schools Over Ten Years: Trained & Partially or Fully ImplementingTrained & Partially or Fully Implementing

23120

184303

394 444520

587654

874

0100200300400500600700800900

1000

Year1

9/98

Year2

9/00

Year3

9/01

Year4

6/02

Year5

6/03

Year6

6/04

Year7

6/05

Year8

6/06

Year9

6/07

Year10

6/08

num

ber o

f sch

ools

# IL PBIS Schools & # External CoachesJune 30, 2008

The Organization of PBIS in The Organization of PBIS in IllinoisIllinois900 schools implementing 900 schools implementing SWPBSSWPBS ISBE

Coordination

ChicagoCoordinators

NorthCoordinators

CentralCoordinators

SouthCoordinators

46 Coaches(10)

33 Schools

495 Coaches(84)

525 Schools

193 Coaches(20)

203 Schools

105 Coaches(29)

127 Schools

Elementary

Middle

CapacityCapacitySchools per Coach per RegionSchools per Coach per Region

Schools per Coach by Region

0

1

2

3

4

5

6

7

8

Chicago North Central South

Scho

ols

per C

oach

Schools/External Schools/ Internal

Illinois Suspension Rates Illinois Suspension Rates per 100per 100

PBS slope = -1.15

Non PBS slope = -.37

Illinois Suspension Rates per Illinois Suspension Rates per 100 for Black and Hispanic 100 for Black and Hispanic StudentsStudents

PBS Slope = -1.85

Non PBS Slope = -,34

North CarolinaNorth CarolinaPositive Behavior Support Positive Behavior Support InitiativeInitiative

Partners’ UpdateFebruary 2009

Heather R. ReynoldsNC Department of Public InstructionBob AlgozzineBehavior and Reading Improvement Center

http://www.dpi.state.nc.us/positivebehavior/

State PBS CoordinatorHeather R Reynolds

North CarolinaNorth CarolinaPositive Behavior Support Positive Behavior Support InitiativeInitiative

North CarolinaNorth CarolinaPositive Behavior Support Positive Behavior Support InitiativeInitiative

Office discipline referral data (majors) from schools implementing PBS in North Carolina [07-08] compare favorably with national averages.

North CarolinaNorth CarolinaPositive Behavior Support Positive Behavior Support InitiativeInitiative

Levels of behavior risk in schools implementing PBS were comparable to widely-accepted expectations and better than those in comparison schools not systematically implementing PBS.

North CarolinaNorth CarolinaPositive Behavior Support Positive Behavior Support InitiativeInitiative

[A]chievement causes [B]ehavior?

[B]ehavior causes [A]chievement?

[Context causes [A]chievement and [B]ehavior?.

Steve Goodmansgoodman@oaisd.orgwww.cenmi.org/miblsi

GoalsGoals

1. Share information about Michigan’s Integrated Behavior and Learning Support Initiative (MiBLSi)

2. Provide examples of improving the quality and quantity of the data collected

3. Provide examples of acting upon project data to improve outcomes

Participating Participating SchoolsSchools

2004 Schools (21)

2005 Schools (31)

2006 Schools (50)

2000 Model Demonstration Schools (5)

2007 Schools (165)

0

50

100

150

200

250

300

2003-2004 2004-2005 2005-2006 2006-2007Existing Schools New Schools

Project Data:Project Data:Outcomes, Process and System Outcomes, Process and System DevelopmentDevelopment

Major Discipline Referrals per 100 Students per Year Major Discipline Referrals per 100 Students per Year (Schools implementing (Schools implementing >> 80% on Team Implementation Checklist) 80% on Team Implementation Checklist)

DIBELS Instructional Recommendations and Major DIBELS Instructional Recommendations and Major Discipline Referral per Cohort per YearDiscipline Referral per Cohort per Year

DIBELS Benchmark

Major Discipline Referrals

0%10%20%30%40%50%60%70%80%90%

100%

2000 2001 2002 2003 2004 2005Year

Perc

ent o

f stu

dent

s

School District

Participating School Example: Participating School Example: Fourth Grade Reading MEAP Results Fourth Grade Reading MEAP Results

Began MiBLSi Implementation

Improving the quality and Improving the quality and quantity of project dataquantity of project data

Percent of Process and System Data Collected by Percent of Process and System Data Collected by CohortCohort

Improving the accuracy and Improving the accuracy and Consistency of Recording Office Consistency of Recording Office Discipline ReferralsDiscipline Referrals

Developing Fluency with Developing Fluency with Discipline Referral Discipline Referral CategoriesCategories

Example Exercise 2: Match the example situation below to the correct problem behavior on the discipline categories answer sheet. Write the letter in the column for Exercise 2.

Example Situation

A. Issac picked up his lunch and notice that another student was sitting in his usual spot at the table. Issac squeezes into his spot forcing the student to abruptly slide off the seat and onto the floor.

B. Ian comes to school wearing a T-shirt with a sexually suggestive picture.

C. During the test, Timothy is looking at Jennifer’s paper and copying her answers.

D. Taneshia walks to school each morning. She plays in the yard across from the school until the bell rings. After she sees all the student run into the building, she walks towards the school, late again.

E. During lunch, Franz collects the empty milk cartons from his classmate. He sets them on the floor in a line and then stamps on the cartons making a loud noise with each stomp.

F. Alexandra has been following Ben around the playground. She sometimes makes various noises (that seems to bother him) when she is near. Sometime she tells him how she and her brother will stop him on the way home from school and beat him up.

G. Susan and Jill are best friends. Jill lives several houses down from the school. During lunch Jill talks Susan into coming over to see her new puppy. The school has a rule about staying on school property at lunch.

H. Billy’s teacher asks the class to take out their math book. Billy groans and says that math is stupid while not taking out his book. His teacher then tells him he needs to get his book from his desk. Billy says he hates math, he hates school and he is not going to do it.

I. Mr. Walter happens to walk into the boy restroom to find Matt and Chris wadding up paper towels and stuffing them into the toilet.

J. Rebecca has been having difficulty with her assignments at school. Her teacher, Ms. Swanson, sent a note home with Rebecca to be signed by her parents and returned the next day. However, when the note was returned the signature was Rebecca’s mother’s name but written in Rebecca’s handwriting.

K. The first grade class writes in a journal. The topic focuses on a particular letter each day. During the letter “F” day, Sarah writes a word she has learned from some tough kids on the playground. She shows her journal to all of the students around her.

Problem Behavior Exercise 1 Exercise 2 Exercise 3

Inappropriate language

Fighting physical injury

Defiance/Disrespect

Lying/Cheating

Harassment/Tease/Taunt

Disruption

Excessive Tardiness

Truancy

Property Damage

Forgery/Theft

Other

Acting on the Data to Improve Acting on the Data to Improve Classroom ManagementClassroom Management

Major Discipline Referrals by LocationMajor Discipline Referrals by Location

Began focusing on Began focusing on classroom management classroom management

support 2005-2006support 2005-2006

MEAP- 4MEAP- 4thth grade Reading Assessment grade Reading Assessment29 Elementary Schools In Michigan29 Elementary Schools In Michigan

Schoolwide: Over 55% of major discipline referrals from classroom

Schoolwide: Under 55% of major discipline referrals from classroom

Probability of scoring below 75% proficiency on 4th grade MEAP (Reading): .78

Probability of scoring above 75% proficiency on 4th grade MEAP (Reading): .75

Improving Targeted Student Improving Targeted Student InterventionIntervention

Interviews with staff and self assessment indicate a need to develop targeted support systems

Checklist for Individual Student Support Systems Checklist for Individual Student Support Systems (CISS) Results from Cohort 4 (n=34 schools) (CISS) Results from Cohort 4 (n=34 schools)

Improving Targeted Student Improving Targeted Student Intervention StrategiesIntervention Strategies

Building Leadership Teams:◦“Quick Sort” process for identifying

students and linking to interventionsFocused training for practitioners:

◦ Using Behavior Education Program (check in- checkout)

Supporting CoachesSupporting Coaches

Conducting Self-Assessment to identify needs

Providing support based on results◦Coach training 2 – 4 time per year◦Coach manual◦Coach website◦Coach conference (March 13-14)

Coaches Self-AssessmentCoaches Self-Assessmentadapted from: Sugai, Todd and Horner, 2006adapted from: Sugai, Todd and Horner, 2006

01234567

Sep Oct Nov Dec Jan Feb Mar Apr May

Avg.

Ref

erra

ls pe

r Day

05-06 06-07

Example of the Impact of Coaching on Student Outcomes:Example of the Impact of Coaching on Student Outcomes:Average Major Discipline Referrals per Day per MonthAverage Major Discipline Referrals per Day per Month

Coach returns Coach returns from leavefrom leave

Working with School Teams to use Working with School Teams to use datadata

Data Review/Action planning with building leadership teams

Pre-training coaches/principals in the data review content

Providing worksheets to guide data review process

Team Evaluation of Outcome, Process and Systems Team Evaluation of Outcome, Process and Systems DataData

EBS Team Implementation Checklist by FeatureEBS Team Implementation Checklist by Feature

Lesson LearnedLesson Learned

Implementation cannot be faster than your school staff capacity to implement

Teams need to be taught how to analyze and use data

Emphasis on directing resources to need and removing competing activities

Building Capacity Building Capacity StatewideStatewideDistricts need capacity to:

◦Deliver regular training on core content

Annual orientation Staff development

◦Incorporate expectations in regular staff evaluations

◦Provide expertise for more intense support need requirements.

Regular meetings with building personnel around “emerging challenges.”

Avoid passing the Avoid passing the planning buckplanning buckState asks districts to build a

planDistricts ask schools to build a

planSchools ask teachers to build a

plan

““Even if you’re on the right track, Even if you’re on the right track, you’ll get run over if you just sit you’ll get run over if you just sit there” there”

- Will Rogers - Will Rogers