Using Data:Which, When, How

Susan Barrett

sbarrett@pbismaryland.org

Sheppard Pratt Health System

Goals

• Define use of data driven decision to reach full implementation of school-wide PBS

• Team Checklist

• EBS Survey

• Safety Survey

• SWIS

Which?

• Process Tools- fidelity– TIC, Survey, Other checklists/ tracking forms

• Progress Monitoring Tools– SWIS, CICO-SWIS, DIBELS, AIMS web

• Outcomes– Decrease problem behavior, increase safety, increase

academic achievement, teacher/staff satisfaction, decrease special ed referral, increase parent and community participation, decrease discrepancy

When

• Frequency ? Data Entry, Report Generation-

• Must see value to use be reliable, valid– Tied to school improvement, curriculum– Priority– Buy in, feedback, voice– Common approach, language

How

Data BaseSWIS

Decision Making•Precision Statements•Build Solutions

Evaluation Questions

School ImprovementGoals and Objectives

Marketing and Dissemination•PresentationsNewsletters

Annual Reports

Assumptions• School teams will be successful if:

– They start with sufficient resources and commitment

– They focus on the smallest changes that will result in the biggest difference

– They have a clear action plan– They use on-going self-assessment to

determine if they are achieving their plan– They have access to an external coach who is

supportive, knowledgeable and persistent.

Data Driven Solutions-Using the Process Measures

• Survey, Safety

• Team Checklist- (TIC)– Self-assessment for Primary Prevention

systems.– Emphasis is on milestones

• Are we doing what we should be doing?

Team Checklist

• Self-assessment tool for monitoring implementation of School-wide PBS.

• Start-Up Elements (17 items)– Establish Commitment– Establish and Maintain Team– Self-assessment– Establish school-wide expectations (Prevention)

» Establish consequences for behavioral errors– Establish information system– Establish capacity for function-based support

• On-going Elements (6 items)

Team Checklist

Use of the Team Checklist

• Who completes the Team Checklist?• The school-team (individually or together)

• When is Team Checklist completed?• At least quarterly, best if done monthly• Check with your local coordinator (www.pbssurveys.org)

• Who looks at the data?• Team• Coach• Trainers/State Evaluation

• Action Planning

Action Planning with the Team Checklist

• Define items (or POINTS) In place or Partially in place.– Points: 2=in place, 1= partial, 0=not in place

• Identify the items that will make the biggest impact

• Define a task analysis of activities to achieve items.

• Allocate tasks to people, time, reporting event.

Implementation by Feature This report shows, for each completed Checklist, the percentage

implemented and partially implemented for each of the following features:

• Establish commitment (questions 1-2) • Establish & maintain team (3-5) • Conduct self-assessment (6-8) • Define expectations (9) • Teach expectations (10-12) • Establish reward system (13) • Establish violations system (14) • Establish information system (15) • Build capacity for function-based support (16-17)

Putting your School in Perspective

• Use % of Total Items/ or % of points

• Compare multiple schools– Messages:

• It is possible• You don’t need to be perfect immediately

Overall Implementation

This report shows, for each completed Checklist, overall scores as (a) the percentage of items fully implemented and partially implemented and (b) the percentage of implementation points. The report displays one row of this data for each Checklist in ascending date order.  The associated column chart shows the percentage of items implemented and partially implemented on each Checklist and, in a separate area, the percentage of implementation points.

Data Driven SolutionsUsing Outcome Measures to Make

Decisions

• School-wide Information System– www.swis.org

Using Data -Using Data -

• Is Your Data give you an accurate picture? Is Your Data give you an accurate picture? Reliable? What is your confidence level? Do Reliable? What is your confidence level? Do you share with all staff?you share with all staff?

• General Data Decision RulesGeneral Data Decision Rules• Are you able to make precision statements ? Are you able to make precision statements ?

Do you present to faculty to get buy in?Do you present to faculty to get buy in?• How do you know when to move “up the How do you know when to move “up the

triangle”?triangle”?

Data System- ChecklistData System- Checklist

• What is the process? How do I refer? How What is the process? How do I refer? How do I complete form? How does it get to do I complete form? How does it get to office? Do you want to know when I refer office? Do you want to know when I refer to school nurse? Or school counselor? to school nurse? Or school counselor? When should I expect to hear back from When should I expect to hear back from office? Do we track minor offenses? Is the office? Do we track minor offenses? Is the form different for minors? What is the form different for minors? What is the process for referring minors? process for referring minors?

Computer ApplicationComputer Application

• Can I enter in referrals easily? 30 Can I enter in referrals easily? 30 seconds/referralseconds/referral

• Can I generate reports easily? 5 second Can I generate reports easily? 5 second rule !!rule !!

• Do I have access to the Big 5? What other Do I have access to the Big 5? What other reports do I need on a regular basis?reports do I need on a regular basis?

Decision MakingDecision Making

• Is there a problem?Is there a problem?• What areas/systems are involved?What areas/systems are involved?• Are there many students or a few involved?Are there many students or a few involved?• What kinds of problem behaviors are occurring?What kinds of problem behaviors are occurring?• When, where, with whom are these behaviors occurring?When, where, with whom are these behaviors occurring?• Do we have agreement on precision statement?Do we have agreement on precision statement?• What is the most effective use of our resources to What is the most effective use of our resources to

address this problem?address this problem?

GuidelinesGuidelines• When does a recurring behavior become a major?When does a recurring behavior become a major?

• Same behavior (3 minors = 1 major)Same behavior (3 minors = 1 major)

• From one particular teacherFrom one particular teacher

• Suggested time frame ( 3minors within 4 weeks)Suggested time frame ( 3minors within 4 weeks)

• Used as a tool to identify patterns of behaviorUsed as a tool to identify patterns of behavior• When are the behaviors occurring? (math, transition)When are the behaviors occurring? (math, transition)

• What are the recurring behaviors? What are the recurring behaviors?

• What are the classroom interventions that have been used? Are these What are the classroom interventions that have been used? Are these interventions working or does something else need to be utilized?interventions working or does something else need to be utilized?

• Why is the behavior occurring? (motivation, example: Johnny rips up his Why is the behavior occurring? (motivation, example: Johnny rips up his math sheet and is given time out and gets out of his work. He always gets math sheet and is given time out and gets out of his work. He always gets to avoid doing his math work)to avoid doing his math work)

From Problem to PreciseFrom Problem to Precise

Precise or Primary ???

Examples

• Alexander Hamilton ES

• Indian Head Elementary

• Cost Benefit