when preparing for this workshop you must either: (1) print registered schools own data or (2) go to...

When preparing for this workshop you must either:(1) Print registered schools own data OR(2) Go to Dummy data in “materials to be printed in office” and print dummy data according to the level of the school AND what they do not have already (e.g. if SET done but no SSS print dummy SSS for them to use).

• Do not print multi-year SWIS graphs.• Bring the TIPS Flip Cart.

NOTE: IN OCTOBER, THE SET AND SELF-ASSESSMENT ORDER IS BACKWARDS (the SET is newer than the self-assessment)

Data-Based Decision Making for Tier 1 School-wide Positive Behavior Support

Presented byPresented by

DevelopmentDevelopment2

www.edprodevelopment.com

Providing staff development and technical assistance to schools with 25 years of experience to Tennessee

schools.

http://www.edprodevelopment.com/

Please ensure that cell phones do not ring Create Your

Name Tent!

Restrooms are located….

Our Quiet Cue

9:30-9:45

11:30-12:30

TASL: 14 credit hours + 1

additional credit every 2 hours

9:30-9:45

11:30-12:30

2:15-2:30

Being PromptArrive at the designated time.Return from a break within designated time allotted.When timer goes off, finish conversation, and be ready for the quiet cue.

Being RespectfulKeep cell phones off or on vibrate. If you must take/receive a call or send a text message, please leave the room before answering the call or texting.Keep on your name tags.When responding to the quiet cue focus on the presenter and stay quiet.

Being InvolvedAttend to workshop presenters quickly after quiet cue.Initiate pairs/group activities when directed. Work as a school or table team when requested and ensure all team members are engaged in the activity.Have fun!

Reward Ticket given by EdPro Staff

Being PROMPT

Being RESPECTFUL Being INVOLVED

EdPro DevelopmentI caught

In your teams, create a poster to introduce your school to the rest of the group. You will have 2 minutes to plan, 2 minutes to create, and 2 minutes to share your team’s poster with the group.

On your poster please include:

1.Your school

2.Number staff and students

3.Where in the SWPBS process are you NOW?

•Get out your Placemat Partners worksheet.

PLACEMAT PARTNERS

_______________

My fork partner

____

_____

____

____

My plat

e partn

er

_______________

My knife partner

_______________

My spoon partner

L O O K S L I K

E T H I S

• Get out your Placemat Partners sheet.• Move around the room and find four different

partners, one for each item in the place setting If someone signs your paper for “fork”, you must do the same for him/her.

• Once you have four different signatures on your placemat partners sheet, find your spoon partner and introduce yourselves and share your burning question about SWPBS data with each other.

• This is the first person with whom you will work in a little while. Keep your Placemat Partners sheet for both days of this workshop.

Workshop OutcomesWorkshop Outcomes

1.Become familiar with and use TIPS with SWIS data for effective decision-making.

2.Understand how precision statements, hypothesis development, and recommendations drive the development of the School-wide Evaluation Tool (SET) Report provided by your coaches.

3.Know how to collect and summarize data from the School Safety Survey.

4.Generate recommendations based on the results of the School Safety Survey and using the School Safety Survey Data Review Manual.

5.Use a process for reviewing end of year data and generating action plans.

• School teams are involved in regular meetings in which they are expected to identify problems and develop academic and behavioral solutions that improve the quality of the school as a learning environment.

• There is increasing access to data about student academic and behavior performance.

• There is a need to help teams identify how to use data productively… and efficiently.

• Workshop participants have been involved in regularly conducted Tier1/Primary Preventions team meetings and are fluent in effective and efficient meeting procedures.

Major Evaluation Major Evaluation Questions Related to Questions Related to

Desired OutcomesDesired Outcomes

1.1.Are our schools safe? Getting Are our schools safe? Getting Safer?Safer?

2.2.Are we getting decreased Are we getting decreased problem behavior?problem behavior?

3.3.Are we implementing SWPBS Are we implementing SWPBS with fidelity?with fidelity?

SWPBS Data Sources

•Office Discipline Referral Data•Suspensions, Expulsions, Remands

•School-Wide Evaluation Tool (SET)

•School Safety Survey•TCAP, Gateway, and benchmark tests

For each data source, we want to For each data source, we want to know…know…

Why do we need this data?Who collects the data?What is measured by the data?When are the data collected?How are the data summarized?Where do we go from here?

• Efficient decisions/solutionsEfficient decisions/solutions• Effective decisions/solutionsEffective decisions/solutions• Informed decisions/solutionsInformed decisions/solutions• Objective decisions/solutionsObjective decisions/solutions• Timely decisions/solutionsTimely decisions/solutions

Where Do We Go From Here?

Are we getting decreased problem

behavior?

• Review current status and identify problems early– Use data on a regular basis (every two weeks) to monitor key

indicators, and identify problems before they become difficult

• Refine a problem statement to a level of precision that will allow functional solutions

• Use data to test possible solutions.

• Use data to assess if solutions are working

– If many students are making the same mistake it typically is the system that needs to change not the students..

ODRs are recorded by ALL faculty and staff on students in the building who demonstrate behavior that violates one or more school rules.

Paper form, often in triplicate, that documents important information concerning events surrounding an occurrence of problem behavior, including name of student, problem behavior, location, time, motivation, referring staff, date, and grade level.

ODRs are universal across schools and, if designed effectively, will give school teams information to make informed decisions at least 80% of the time.

ODRs are turned in to a data recorder once the building administrator has intervened. The ODR information is entered into a discipline data management system (e.g., SWIS). Visual summaries are reviewed by the Tier 1/Primary Preventions team charged with reviewing the data AND with the full faculty and staff on a routine basis.

Each time a student engages in rule violating behavior (any major OR multiple minors as per school procedures for documenting problem behavior.)

An efficient system for gathering discipline data, web-based computer application for data entry and report generation, and practical process for using decision information for decision making.

Summarizes office discipline referral information by (a) how often, (b) where, (c) when, (d) what and (e) who.

Summaries provided in tables and graphs. Confidentiality protected. $250 per school per year annual fee (+$50 for CICO-SWIS). 7432 schools: 4410 Elementary, 1205 Middle, 598 High,

1219 K-8/12

SWISTM OFFICE DISCIPLINE REFERRAL FORM

Student(s) _________________ Referring Staff _____________ Grade Level ____ Date ______ Time ___

LocationClassroom PlaygroundCommons/common areaHallway/ breezeway

Cafeteria Bathroom/restroomGymLibrary

Bus loading zoneParking lotOn busSpecial event/assembly/ field trip

Other __________

Problem Behaviors (check the most intrusive)

MINOR (Staff Managed) Inappropriate lang. Physical contact Defiance/disrespect/ non-compliance Disruption Property misuse Other _______________________

MAJOR (Office)Abusive lang./ inapprop. lang Fighting/ physical aggression Defiance/ disrespect/ insubordination Harassment/ tease/ tauntDisruptionTardy

Skip class/ truancy Forgery/ theftDress code violationLying/cheating TobaccoAlcohol/drugsCombustibles

Vandalism Property damageBomb threatArson Weapons Other __________

TIPSTIPSTeam Initiated Problem Team Initiated Problem

SolvingSolving(Horner, Todd, Newton, (Horner, Todd, Newton,

Algozinne, and Algozinne)Algozinne, and Algozinne)A five step process to define a

problem and develop a workable implementation and

evaluation plan

TIPS

• Four year grant 2008-2012• Goal: Develop a “problem-solving model” for

school teams that results in active use of data to (a) define problems, (b) build solutions, and (c) transform solutions into practical action plans.

• Steve Newton, Anne Todd, Bob Algozzine, Kate Algozzine, Rob Horner

Goals

• Define logic for data use• Define “problem statements” that can lead

to functional solutions• Define standards for using data to refine

problem statements and build solutions• Define core elements of behavioral solutions• Assist teams with using data for problem

solving

Main Idea

• The process a team uses to problem solve is important:– Roles:

– Facilitator; Recorder; Data analyst; Active member

– Organization– Agenda; Old business (did we do what we said we would

do); New business; Action plan for decisions.– What happens BEFORE a meeting– What happens DURING a meeting– What happen AFTER a meeting

Main Ideas

• Decisions are more likely to be effective and efficient when they are based on data.

• The quality of decision-making depends most on the first step (defining the problem to be solved)

• Define problems with precision and clarity

Improving Decision-Making

ProblemFrom: Solution

Common Errors in the “Problem-Solution” Sequence:

• Define solution before the problem • Define broad problem that does not allow people to focus

on functional solutions.“Students are behaving in a disrespectful manner.”

• Failure to use data to confirm/define problem“One teacher states that cafeteria is out of control…”

• Agree on solution without plan for implementation or evaluation

• Agree on solution but never to re-visit to determine if solution was implemented or if effective

• Serial problem solving without decisions – “problem admiration”

Many solutions…limited implementation

Main Ideas

• Data help us ask the right questions…they do not provide the answers: Use data to– Identify problems– Refine problems– Define the questions that lead to solutions

• Data help place the “problem” in the context rather than in the students.

Improving Decision-Making

Information for Problem Solving

ProblemFrom:

To:

Solution

Problem Solution

Main Ideas• Build “decision systems” not “data systems”• Use data in “decision layers”

– Is there a problem? (overall rate of ODR)– Localize the problem

– (location, problem behavior, students, time of day)

– Get specific

• Don’t drown in the data• It’s “OK” to be doing well• Be efficient

TIPS:TIPS:Team Initiated Problem Team Initiated Problem

SolvingSolving

1. Determine if a problem exists.2. Develop and refine hypothesis3. Discuss and select solutions4. Develop and implement an action

plan/set of tasks addressing problem(s)

5. Evaluate and revise

CollectCollect and Useand Use

DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions

Develop andImplementAction Plan

Evaluate andRevise

Action Plan

Team Initiated Problem Solving (TIPS) Model

Identifying problems/issues

• What data to monitor– ODR per day per month– OSS, ISS, Attendance, Teacher report– Team Checklist/ SET (are we doing what we planned to do?)

• What question to answer– Do we have a problem?

• What questions to ask of Level, Trend, Peaks– How do our data compare with last year?– How do our data compare with national/regional norms?– How do our data compare with our preferred/expected status?

• If a problem is identified, then ask– What are the data we need to make a good decision?

Interpreting Office Referral Interpreting Office Referral Data:Data:

Is there a problem?Is there a problem?Absolute level (depending on size of

school)– High Schools (1/147)– Middle Schools (1/200)– Elementary Schools (1/454)– K-(8-12) (1/238)

Trends– Peaks before breaks?– Gradual increasing trend across year?

Compare levels to last year– Improvement?


DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions


Evaluate andRevise

Action Plan

Using the Team Initiated Problem Solving (TIPS) Model With SWIS Data

1.Determine if problem exists

Using TIPS with SWIS Using TIPS with SWIS DataData

(Horner, Todd, and Newton 2008)(Horner, Todd, and Newton 2008)

Organizing Data for Decision-making

• Compare data across time

• Moving from counts to count/month

Tot

al O

ffic

e D

isci

plin

e R

efer

rals Data must be

presented in a useful way for accurate decision-making.

Ave

rage

Ref

erra

ls P

er D

ay P

er M

onth

SWIS summary 2009-10 (Majors Only)4,019 schools; 2,063,408 students; 1,622,229 ODRs

Grade Range Number of Schools

Mean Enrollment per school

Median ODRs per 100 per school day

K-6 2565 452 .22

6-9 713 648 .50

9-12 266 897 .68

K-(8-12) 474 423 .42

Office Discipline Referrals per Day per Month per 100 Students

0

2

4

6

8

10

12

14

16

Sept Oct Nov Dec Jan Feb March April May June

# p

er d

ay p

er 1

00 s

tud

ents

Series1

Middle School of 625 students625/200=3.23

0

0.5

1

1.5

2

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May Jun

School Months

Office Referrals per Day per MonthLast year

Elementary School with 150 Students

150/454=.33

0

5

10

15

20 A

ve R

efe

rrals

per

Day


School Months

Office Referrals per Day per MonthLast year

High School of 1800 students

1800/147=12.2

0

5

10

15

20 A

ve R

efe

rrals

per

Day


School Months

Office Referrals per Day per MonthLast Year and This Year

Middle School of 700 students

700/200=3.5

0

5

10

15

20

Ave R

efe

rrals

per

Day


School Months


Middle School of 700 students

0

5

10

15

20

Ave R

efe

rrals

per

Day

Sept Oct Nov Dec Jan Feb Mar Apr May

School Months

Office Referrals per Day per MonthThis Year

Middle School N= 495 495/200=2.5

Is There A Problem?

Is There a Problem?

0

5

10

15

20 A

ve R

efe

rrals

per

Day


School Months

Office Referrals per Day per MonthThis year (Middle)

N= 495

0

5

10

15

20

Ave R

efe

rrals

per

Day


School Months


Middle School N= 495


DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions


Evaluate andRevise

Action Plan

Problem Solving Foundations


1.Determine if problem exists2.Develop and refine hypothesis

Using TIPS with SWIS DataUsing TIPS with SWIS Data

Define problems with precision

Using Data to Refine Problem Statements

• The statement of a problem is important for team-based problem solving.

• Everyone must be working on the same problem with the same assumptions.

• Problems often are framed in a “Primary” form, that creates concern, but that is not useful for problem-solving.

• Frame primary problems based on initial review of data• Use more detailed review of data to build “Solvable Problem

Statements.”

General versus Precision StatementsGeneral

Statements

– Too many referrals

Precision Statements– There were an average of 8

ODRs/day/month for month of September

– Gang behavior is increasing

– September has more suspensions than last year

– The cafeteria is out of control

– Gang behavior has steadily increased at least 3 incidences for the last three months – There were 6 suspensions in September which was 3 more than last September

– Problem behavior occurs most frequently in the cafeteria and accounts for 50 % of all office referrals

WHICH STATEMENT IS MORE PRECISE????

Precise or Primary Statement?

• Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school

• James D. is hitting others in the cafeteria during lunch, and his hitting is maintained by peer attention.


• ODRs during December are higher than in any other month.

• Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by escape from work (but may also be maintained by peer attention… we are not sure).


• Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention.

• Boys are engaging in sexual harassment

What are the data you are most likely to need to move from a Primary to a Precise

statement?• What problem behaviors are most common?

– ODR per Problem Behavior• Where are problem behaviors most likely?

– ODR per Location• When are problem behaviors most likely?

– ODR per time of day• Who is engaged in problem behavior?

– ODR per student

Example Precision Statement There are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students.



– ODR per Problem Behavior

What behaviors are problematic?

0

10

20

30

40

50

Num

ber

of R

efe

rrals

Lang Achol ArsonBombCombsDefianDisruptDressAgg/fgtTheftHarassProp D Skip Tardy Tobac Vand Weap

Types of Problem Behavior

Referrals per Prob Behavior

What behaviors are problematic?

0

5

10

15

Num

ber

of R

efe

rrals







– ODR per Location

0

10

20

30

40

50

Num

ber

of O

ffic

e R

efe

rrals

Bath RBus A Bus Caf ClassComm Gym Hall Libr Play G Spec Other

School Locations

Referrals by Location





– ODR per time of day

0

5

10

15

20

25

30 N

um

ber

of R

efe

rrals

7:00 7:30 8:00 8:30 9:00 9:30 10:0010:3011:00 11:3012:0012:30 1:00 1:30 2:00 2:30 3:00 3:30

Time of Day

Referrals by Time of Day






– ODR per student

Who is contributing to the problem?Referrals per Student

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students

Who is contributing to the problem?

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students

Students per Number of Referrals

1. Get out your Precise Statements Worksheet.

Precise Statements WorksheetProblem Statement General? Precise?

1. Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school.

2. James D. is hitting others in the cafeteria during lunch.

3 ODRs have leveled off; however, they remain above the national average. Girls are receiving more ODRs than boys, and these are occurring most frequently in the hallways.

4. ODRs during December are higher than in any other month.

5. Minor disrespect and disruption are increasing over time. This pattern is most common in 7th and 8th grades and involves many students.

6. Ninth graders are the worst behaved in the school.

7. Three 5th grade boys are name calling and touching girls inappropriately during recess.

8. Fighting occurred 12 times during October. These occurred around 9:30, 11:45, 12:30, and 2:15 which are our transition times..

9. Average ODRs per month have increased to 11. They occur throughout the day in all locations. The problem behavior occurring most frequently is disrespect; however, several other problem behaviors are occurring.

10. Boys are engaging in sexual harassment.

1. Get out your Precise Statements Worksheet. With your spoon partner, identify each statement as a general or precise problem statement. Decide if the statement answers each of the following: Who? What? Where? and When? If it does not what information do you need and where will you get that information.

2. Next, change each general statement to a precise statement, noting what was missing.

Precise or General Statement?

1. Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school.

2. James D. is hitting others in the cafeteria during lunch.

3. ODRs have leveled off; however, they remain above the national average. Girls are receiving more ODRs than boys, and these are occurring most frequently in the hallways.

4. ODRs during December are higher than in any other month .

5. Minor disrespect and disruption are increasing over time. This pattern is most common in 7th and 8th grades and involves many students.

6. Ninth graders are the worst behaved in the school.

7. Three 5th grade boys are name calling and touching girls inappropriately during recess.


8. Fighting occurred 12 times during October. These occurred around 9:30, 11:45, 12:30, and 2:15 which are our transition times.

9. Average ODRs per month have increased to 11. They occur throughout the day in all locations. The problem behavior occurring most frequently is disrespect; however, several other problem behaviors are occurring.

10.Boys are engaging in sexual harassment


1.Determine if problem exists.

2.Develop and refine hypothesis.



Use accurate and current data to identify and refine problem statements

Use Accurate and Current Data

• The data must be “believable” by staff.• The data should be current (no more than

48 hours old) • The data should be in a format that

provides information useful for decision-making

2.Develop and refine hypothesis


Use accurate and current data to identify and refine problem statements

Develop hypothesis statements to explain the precision problem statements. Why is the problem sustaining? Why does it keep happening?







– ODR per student• Why are problem behaviors sustaining?

– Custom graph

Why Are Problem Behaviors sustaining?

Hypotheses = WHYIN WORDS

SUCH AS…

~ We believe

~ We think

~ Because, etc.

Example Hypothesis Statement There are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students.and the aggression is related to getting access to the new playground equipment.

•High school freshman have more office discipline referrals (ODRs) than sophomores, juniors, and seniors collectively. ODRs occur most frequently due to tardiness upon arrival to school and during class transitions. We believe this is because there is limited staff visible during transitions, staff that are visible are not reminding/prompting students to move on to next class, nor are they acknowledging students for getting to class on time at a rate that students need/are responding to.

Example Hypothesis Statements

ODRs occur most frequently on the buses, both in the morning and afternoon, and are mostly for inappropriate language. ODRs are occurring in all buses. Twenty five different students have received at least one ODR while on the bus during August and September. We believe this is because there is little involvement of the bus drivers in the schools SWPBS efforts.


•For the third year in a row office referrals are highest in December and above the national average. Over 150 students (45% of the school) have at least one ODR. ODRs are occurring in both classroom and non-classroom settings and across the school day, and disruption is the most common problem behavior. We believe this is due to students AND staff becoming both anxious and excited about Christmas break. In addition, it could be that staff are less focused on implementing their primary preventions practices.


Do you need additional

information before you generate the

WHY?


DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions


Evaluate andRevise

Action Plan



1.Determine if a problem exists2.Develop and refine hypothesis3.Discuss and select solutions

• Brainstorm options• Prioritize - Focusing Four - Delphi weighting

procedure

What is the most effective use of our resources to address this problem?


1. Determine if a problem exists2. Develop and refine hypothesis3. Discuss and select solutions

• Brainstorm options

What is the most effective use of our resources to address this

problem?


Brainstorm

• Record brainstormed ideas on chart paper.

• Elicit ideas only.• No criticism or questions• Push for between 8-18

ideas.• Go for the Gusto!

Brainstorming

Get in groups of four. Each take a card that describes your role (e.g., Negative Nellie, Polly Positive, Outrageous Oscar, & Level-Headed Lou). Select a facilitator for your group. Brainstorm what where you would want to vacation for two weeks in the US if you had all the money you needed to go and have fun.

1.Determine if a problem exists2.Develop and refine hypothesis3.Discuss and select solutions

•Brainstorm options•Prioritize - Focusing Four

What is the most effective use of our resources to address this problem?


The Focusing FourThe Focusing FourBrainstormBrainstorm Record brainstormed ideas on chart paper. Elicit ideas only. No criticism or questions. Push for between 8-18 ideas.

ClarifyClarifyAsk author for clarification if/where needed.Stop clarification when questioner indicates understanding.Condense and separate ideas per request. Team members need to defend request/need.

The Focusing FourThe Focusing Four(Cont)(Cont)

AdvocateAdvocate Team members may advocate for as many items and as

many times as they wish. Advocacy must be brief and phrased in the positive.

CanvassCanvass Ask team members to identify which 3 ideas they feel are

most important. They do not need to be placed in rank order.

Take a hand count to determine which items are of most importance to the group (majority vote).

Focusing FourWith your list generated during your “Brainstorming Activity”, use the Focusing Four to pick your top three ideas. A different group member should be identified as the facilitator for this activity.

1. Determine if a problem exists2. Develop and refine hypothesis3. Discuss and select solutions

• Brainstorm options• Prioritize - Focusing Four - Delphi weighting procedure

What is the most effective use of our

resources to address this problem?


Example: Example: Where should we go to lunch?Where should we go to lunch?

Your group brainstorms and comes up Your group brainstorms and comes up with 4 options of where we could go to with 4 options of where we could go to lunch. lunch. •Burger KingBurger King•Arby’sArby’s•McAlister’sMcAlister’s•Pizza HutPizza Hut

Now each team member will individually Now each team member will individually assign each option a number of 1 thru 4, assign each option a number of 1 thru 4, with 4 being his/her top option.with 4 being his/her top option.

Example: Example: Where should we go to lunch?Where should we go to lunch?

One member of the team collects all the numbers given to each place. The option with the highest total score is the place we will all go to eat.

Options Team Member 1

Team Member 2

Team Member 3

Total

A. Burger King 3 4 3 10

B. Arby’s 2 3 4 9

C. McAlister’s 4 1 2 7

D. Pizza Hut 1 2 1 4

FirstFirst: Assign each potential : Assign each potential solution a letter of the solution a letter of the alphabet.alphabet.

For example, one team For example, one team generated four solutions. generated four solutions. They assigned each solution They assigned each solution a letter of the alphabet as a letter of the alphabet as follows:follows: A. Increase staff A. Increase staff supervisionsupervision B. Re-teach school rules in B. Re-teach school rules in hallwayhallway C. Get students to create C. Get students to create rules postersrules posters D. Create directional signs D. Create directional signs for traffic flowfor traffic flow

The solution believed to be The solution believed to be MOST important to address, MOST important to address, right now, given the right now, given the considerations, is assigned considerations, is assigned the greatest number.the greatest number.

The solution you believe is The solution you believe is LEAST important to address LEAST important to address right now is assigned the right now is assigned the value of 1.value of 1.

SecondSecond, Each team member assigns a , Each team member assigns a priority value to each solution, rank priority value to each solution, rank ordering the solutions from most to least ordering the solutions from most to least important. The rank ordering, important. The rank ordering, HOWEVER, occurs in a specific manner HOWEVER, occurs in a specific manner and is described as follows:and is described as follows:

11stst

22ndnd

The solution believed to be NEXT The solution believed to be NEXT MOST important to address is MOST important to address is assigned the value of the total assigned the value of the total number of identified behavior number of identified behavior problems minus 1.problems minus 1.

The solution you believe is NEXT The solution you believe is NEXT LEAST important to address right LEAST important to address right now is assigned the value of 2.now is assigned the value of 2.

SecondSecond, Each team member assigns a , Each team member assigns a priority value to each solution, rank priority value to each solution, rank ordering the solutions from most to least ordering the solutions from most to least important. The rank ordering, important. The rank ordering, HOWEVER, occurs in a specific manner HOWEVER, occurs in a specific manner and is described as follows:and is described as follows:

33rdrd

44thth

The behavior believed to be the third The behavior believed to be the third most important to address (if there most important to address (if there are five or more) is assigned the are five or more) is assigned the value of the total number of value of the total number of identified behavior problems minus identified behavior problems minus 2.2.

Continue on this pattern until all Continue on this pattern until all options are assigned a value.options are assigned a value.

SecondSecond, Each team member assigns a , Each team member assigns a priority value to each solution, rank priority value to each solution, rank ordering the solutions from most to least ordering the solutions from most to least important. The rank ordering, important. The rank ordering, HOWEVER, occurs in a specific manner HOWEVER, occurs in a specific manner and is described as follows:and is described as follows:55thth

Etc.Etc.

ThirdThird, EACH teacher’s value is added to , EACH teacher’s value is added to obtain a total sum value for each solution obtain a total sum value for each solution identified, thereby creating a rank order identified, thereby creating a rank order prioritization list. In the example prioritization list. In the example provided:provided:

Behavior Teacher 1

Assigned value

Teacher 2 assigned

value

Teacher 3 assigned

value

Teacher 4 assigned

value

Total assigned

value

A. Increase supervision

2 4 4 3 13

B. Re-teach

4 2 3 2 11

C. Student Posters

3 3 2 4 12

D. Signs 1 1 1 1 4

Thus, team members rank Thus, team members rank ordered the solutions as ordered the solutions as follows:follows:

Most important: increase

supervision (13)

Next most important: students

create rules posters (12)

Next most important: re-teach (11)

Least important: directional signs

(4)

FourthFourth, team members are asked if , team members are asked if there is a consensus of this there is a consensus of this

weighted outcome. If not, discuss, weighted outcome. If not, discuss, and weight again until consensus and weight again until consensus

is obtained. is obtained. 543210

I am all for this idea. I can be a leader

I’m for the idea. I can provide support.

I’m not sure but I’m willing to accept the group’s opinion.

I’m not sure. I need more discussion.

I can’t support it at this time. I need more information.

No. I need an alternative I can support.

Delphi WeightingIn your same groups, with your list generated during your “Brainstorming Activity”, use the Delphi Weighting procedure to pick your top three ideas.

Using Data to Build Solutions• Prevention: How can we avoid the problem context?

– Who, When, Where– Schedule change, curriculum change, etc

• Teaching: How can we define, teach, and monitor what we want?– Teach appropriate behavior– Use problem behavior as negative example

• Recognition: How can we build in systematic reward for desired behavior?

• Extinction: How can we prevent problem behavior from being rewarded?

• Consequences: What are efficient, consistent consequences for problem behavior?

• How will we collect and use data to evaluate (a) implementation fidelity, and (b) impact on student outcomes?

Define Problem

By answering…• Where• When• Who• What• Why• What other

info needed?

Define SolutionBy answering• Prevention• Teaching• Reward• Extinction• Corrective

Consequences• Monitoring

Precision Statement/Hypothesis

• What• Where• When• Who• Why• What other info needed?

• Possible Solutions?

Solution Development

Prevention

Teaching

Reward

Extinction

Corrective Consequence

Data Collection

Problem Solving Action PlanPrecise Problem Statement

Solution Actions

Who? When? Goal, Timeline, Rule & Updates

Examples

• Trevor Test

• Phoenix Elementary

• Langley

• Sandhill

Trevor Test Middle School

565 students

Grades 6,7,8

0

5

10

15

20

Ave R

efe

rrals

per

Day


School Months


0

10

20

30

40

50

Num

ber

of R

efe

rrals




0

20

40

60

80

Num

ber

of O

ffic

e R

efe

rrals


School Locations


0

10

20

30

40

50

Num

ber

of R

efe

rrals

7:00 7:30 8:00 8:30 9:00 9:30 10:0010:3011:00 11:3012:0012:30 1:00 1:30 2:00 2:30 3:00 3:30

Time of Day


Cafeteria Class Commons Hall

12:00

Lang.

Defiance

Disruption

Harrass Skip

0

20

40

60

80

100

Num

ber

of S

tudents

1 2 3 4 5 6 7 8 9 10111213141516171819

Number of Referrals


Precise Problem Statement &Hypothesis Development

• Many students from all grade levels are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attention

• A smaller number of students engage in skipping and noncompliance/defiance in classes, (mostly in rooms 13, 14 and 18), and these behaviors appear to be maintained by escape.

Solution Development:For disruption in hall and cafeteria

Prevention *Teach behavioral expectations in cafeteria

*Maintain current lunch schedule, but shift classes to balance numbers.

Teaching

Reward Establish “Friday Five”: Extra 5 min of lunch on Friday for five good days.

Extinction Encourage all students to work for “Friday Five”… make reward for problem behavior less likely

Corrective Consequence Active supervision, and continued early consequence (ODR)

Data Collection Maintain ODR record and supervisor weekly report

Problem Solving Action Plan

Precise Problem Statement

Solution Actions Who? When? Goal, Timeline, Rule & Updates

Many students from all grade levels are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attention

Prevention: Teach behavioral expectations in cafeteria

Maintain current lunch schedule, but shift classes to balance numbers

Teachers will take class to cafeteria; Cafeteria staff will teach the expectations

Principal to adjust schedule and send to staff

Rotating schedule on November 15

Changes begin on Monday

Goal: Reduce cafeteria ODR’s by 50% per month (Currently 24 per month average)

Timeline: Review Data & Update Monthly

A smaller number of students engage in skipping and noncompliance/defiance in classes, (mostly in rooms 13, 14 and 18), and these behaviors appear to be maintained by escape.

Recognition: Establish “Friday Five”: Extra 5 min of lunch on Friday for five good days

Extinction: Encourage all students to work for “Friday Five”… make reward for problem behavior less likely

School Counselor and Principal will create chart & staff extra recess

Principal to give announcement on intercom on Monday

Corrective Consequence- Active supervision and continued early consequence (minor/major ODR’s)

Hall and Cafeteria Supervisors Ongoing

Data Collection – Maintain ODR record & supervisor weekly report

SWIS data entry person & Principal shares report with supervisors

Weekly

Phoenix Elementary

Using Data For Decision-Making

You are the PBS team for Phoenix Elementary. 265 students k-5

• Do you have a problem?

• Where?

• With Whom?

• What other information might you want?

• Given what you know, what considerations would you have for possible action?

Phoenix Elementary265/100 = 2.65 2.65 x .34 = .901

Phoenix Elementary Problem Behaviors

Phoenix ElementaryLocations

Year OneYear Two

Phoenix Elementary - Time

Phoenix ElementaryReferrals Per Student (2 + Referrals)

Problem Statement

• Do we have a problem?

• Build a precise problem statement


Prevention

Teaching

Reward

Extinction


Data Collection


Solution Actions


Langley Elementary School

478 Students

K-5


Prevention

Teaching

Reward

Extinction


Data Collection


Solution Actions


Sandhill High school354 students

Sandhill High School: 354 students

Sandhill - Problem Behavior

Sandhill - Location

Sandhill –Referrals by Student

Sandhill - Time


Prevention

Teaching

Reward

Extinction


Data Collection


Solution Actions


Team Training & Follow Up

• Swift at SWIS Training

• Team Meetings:

–First time: Simulated Data

–Second time: Use First Month of Data

–Third time: Problem Solving Statements


565 students: Grades 6,7,8


0

5

10

15

20

Ave R

efe

rrals

per D

ay


School Months


0

10

20

30

40

50

Num

ber

of R

efe

rrals




0

20

40

60

80 N

um

ber

of O

ffic

e R

efe

rrals


School Locations


0

10

20

30

40

50

Num

ber

of R

efe

rrals

7:00 7:30 8:00 8:30 9:00 9:30 10:0010:3011:00 11:3012:0012:30 1:00 1:30 2:00 2:30 3:00 3:30

Time of Day


Cafeteria Class Commons Hall

12:00

Lang.

Defiance

Disrespect

Harass Skip

Students by Major Referrals

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

Students

Num

ber

of R

efer

rals

Precision Statement/Hypothesis• Define Problem• What• Where• When• Who• Why• What other info

needed?

• Define Solution• Prevention• Teaching• Reward• Extinction• Corrective

Consequences• Monitoring

Trevor Test Middle School Problem Statement

• WHAT: Increasing trend in frequency of problem behavior

• WHAT: Two problems: – disrespect, insubordination, inappropriate

language, and harassment– Skipping class

• WHERE: Cafeteria• WHEN: Most problem behaviors occur

during the noon lunch time • WHO: About 24 students with 2-5 ODRs

Problem StatementProblem Statement• Preliminary:

– There is an increase in office discipline referrals from Sept to Jan, and the increase is resulting in more referrals than we consider acceptable, and more than is typical of national norms.

• Precise:– Inappropriate language, harassment, disrespect and

defiance toward staff is occurring in the cafeteria, before, during and after lunch. WHY: Many students and we are unclear what is maintaining these behaviors.

– We believe over-crowding may be a factor, and we did not teach school-wide expectations in cafeteria.

Solution DevelopmentSolution Development

Prevention *Teach behavioral expectations in cafeteria

*Maintain current lunch schedule, but shift classes to balance numbers.

Teaching


Extinction Unclear

Corrective Consequence Active supervision, and continued early consequence

Data Collection Maintain ODR record and report weekly

Information needed to move from a General to a Precise

statement• Where are problem behaviors most

likely?– ODR per Location

• When are problem behaviors most likely?– ODR per time of day

• Who is engaged in problem behavior?– ODR per student

If more than 35% of referrals come

from non-classroom settings

If there are more than 2.5

office discipline

referrals per student

If more than 50% of

referrals come from less than

10% of the classroom

If a small number of students destabilizes the overall functioning

of the school

If less than 10 students continue the same rate of

referrals after receiving

targeted group support

TargetedGroup

InterventionsIndividual system

with Action Team

Structure

Non-classroom

system

If less than 10 students receive more than 10 office discipline

referrals

If more than 40%

of students received

one or more office discipline

referrals

School-wide

System

If more than 60 % of office discipline

referrals come from the

classroom.

ClassroomSystem

If there are more than 15% of

students receiving referrals from non-

classroom settings

If more than 10-15 students receive

more than 5 office discipline

referrals

SWIS TMSWIS TMGeneral Data Decision RulesGeneral Data Decision Rules

Refe

rrals

per

Stu

den

tR

efe

rrals

per

Stu

den

t

0

10

20 N

um

ber

of R

efe

rrals

per

Stu

dent

Students

Who is contributing to the Who is contributing to the problem?problem?

Referrals per StudentReferrals per Student

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students


Trevor Test Middle SchoolStudents by Major Referrals

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

Students

Nu

mb

er o

f R

efer

rals


statement• Where are problem behaviors most

likely?– ODR per Location



• What problem behaviors are most common?– ODR per Problem Behavior

0

10

20

30

40

50 N

um

ber

of R

efe

rrals




0

5

10

15 N

um

ber

of R

efe

rrals





Combining the graphs you have Combining the graphs you have just looked at, what is the just looked at, what is the problem at this school?problem at this school?

Trevor Test Middle School: Problem Statement

• Increasing trend in frequency of problem behavior

• About 24 students with 2-5 ODRs

• Two problems– Most problem behaviors occur during the

noon lunch time and include disrespect, insubordination, inappropriate language, and harassment

– Skipping class is also a problem

Problem Statement• General:



statement• What problem behaviors are most

common?– ODR per Problem Behavior

• Where are problem behaviors most likely?– ODR per Location




Using the available data, change Using the available data, change your general statement to a your general statement to a precise problem statement.precise problem statement.

Problem Statement• General:– There is an increase in office discipline

referrals from Sept to Jan, and the increase is resulting in more referrals than we consider acceptable, and more than is typical of national norms.

•Precise:– Inappropriate language, harassment, disrespect and defiance toward staff is occurring in the cafeteria, before, during and after lunch. Many students are engaging in behaviors and we are unclear what is maintaining these behaviors.


Generate a hypothesis based the Generate a hypothesis based the your precision problem your precision problem

statement.statement.

Problem StatementProblem Statement• General:


• Precise:– Inappropriate language, harassment, disrespect and

defiance toward staff is occurring in the cafeteria, before, during and after lunch. Many students are engaging in behaviors and we are unclear what is maintaining these behaviors.

…Moving into hypothesis– We believe over-crowding may be a

factor, and we did not teach school-wide expectations in cafeteria.


Complete the Solution Complete the Solution Development Table for Trevor Development Table for Trevor

Test Middle SchoolTest Middle School

Solution DevelopmentPrevention *Teach behavioral

expectations in cafeteria*Maintain current lunch schedule, but shift classes to balance numbers.

Teaching


Extinction UnclearCorrective Consequence

Active supervision, and continued early consequence

Data Collection

Maintain ODR record and report weekly


DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions


Evaluate andRevise

Action Plan



TIPS: TIPS: Team Initiated Team Initiated Problem SolvingProblem Solving

(Adapted from: Horner, Todd, and Newton 2008)(Adapted from: Horner, Todd, and Newton 2008)

1.Determine if problem exists2.Develop and refine hypothesis3.Discuss and select solutions4.Develop and implement an

action plan/set of tasks addressing problem(s)

4. Develop and implement an action plan/set of tasks addressing problem(s) Be accountable. Use data to

evaluate implementation and impact•For every decision, define actions & timelines

•Record who, what, & by when for each decision

•Continually share information with staff

•The people who implement a solution should be involved in (a) defining the problem and (b) selecting the solution

•Conduct regular fidelity and social validity checks– intervention implemented as

planned– staff and student satisfaction– update tasks/actions

Agenda


DataData

Review Status and

Identify Problems

Develop andRefine

Hypotheses

Discuss andSelect

Solutions


Evaluate andRevise

Action Plan



Step 5: Evaluate and Revise Action Plan

• Revisit Action Plan items–Review status and identify problem…. The TIPS cycle continues

–Document as you go–Date everything

TIPS: TIPS: Team Team Initiated Problem Initiated Problem

SolvingSolving(Adapted from: Horner, Todd, and (Adapted from: Horner, Todd, and

Newton 2008)Newton 2008)

Get out your Solution Development Sheet from your handouts. Using TIPS Process,

with your school team colleagues, generate an action plan based on your

own SWIS data.• Enrollment• Office discipline referrals• Draw the “comparator” line• Do we have a problem?• What systems are problematic?• Who is involved?• Build a hypothesis• Refine the hypothesis• Identify possible solutions: (Prevention,

Teaching, Rewarding, Extinction, Corrective Action)

• Test solutions against available data• What additional data would you like to have?

SWPBS Primary/Tier 1 Team Meeting Agenda PHASE I (ONE)_________________School _____________, 2009

Ground Rules: Need to establish some (20 minutes) ~ Review from workshop and add to…..1. 2. 3.

1.

Celebrations (3 minutes) 2.

Agenda Review (2 minutes)3.

Overview of SWPBS (10 minutes)

A. Task review

B. Someone demonstrate how to explain SWPBS to an uninformed person OR Quiz.

C. Tasks negotiated today 4.

Faculty and Staff Support is 80% (10 minutes)

Insert graphs from Self Assessment here5.

Team Composition (10 minutes)

A. Do we have all of appropriate people at the table

B. Team has established a clear mission/purpose that does not overlap with other team’s mission/purpose

C. Team has shared mission/purpose with all faculty and staff

•Review of SWIS Data (30 minutes – typically)

A. Task Review

B. Celebrations

C. Precision Problem Statements

D. Hypothesis Statements

E. Solutions

F. Tasks Negotiated Today

DATES Facilitator

Creates agenda and guides

meeting

ScribeTakes notes and copies

for all

Timekeeper

Monitors time on agenda items

Taskmaster

Writes tasks and copies for all

Rudder Ensures team stays on task

(agenda)

Encourager

Checks task status, assists if help is

needed

EqualizerEnsures

everyone is able to participate in

discussion

Jargon Buster

Asks for clarification on

unknown/confusing terms.

SWPBSerExplains SWPBS for this meeting during Overview

of SWPBS

WHO?

’

’

??????

Data source for evaluating “Are discipline problems decreasing?”

Reflect fidelity of implementing SWPBS practices

Should be reviewed at least quarterly

• To be completed throughout the year and reviewed at the last team meeting of the year

• Reviews SET, Self-Assessment, SSS, and SWIS data

• Highlights celebrations and labels SWPBS features for improvement for the next year

End of the Year Data End of the Year Data ReviewReview

and Planning Organizerand Planning Organizer

• End of year SWIS data includes:– Major problem behaviors– Location of problem behaviors– When problem behaviors are

occurring– Suspensions/expulsions– SWIS triangle data




and Planning Organizerand Planning Organizer• SWIS End of Year Triangle Data– Green = Percent of students with 0-1

ODR– Yellow = Percent of students with 2-5

ODRs– Red = Percent of students with 6 or

more ODRs

Directions for completing at the end of the year:• Have team members pair up or get into groups of 3-4• Each pair/group:

– reviews a different tool/piece of data (e.g. SET, SSS, Self-assessment, End of Year SWIS data),

– identifies status of specific systems/features, and – records that information either on this page or the

attached page. • Record “celebrations” and systems/features which are “in place” on the first page; • Record systems/features and things that are “partially in place”/”not in place” on the second page.



Complete the “End of Year Planning Organizer”

with your school’s SWIS data.

The remaining portions of the Planning Organizer will be completed during the

remainder of this workshop.

ActivityActivity

Note on your agenda...Note on your agenda...

The SET is conducted by an outside coach or consultant. The head administrator, at least 15 staff members and 15 students are randomly chosen to be interviewed.

The SET assesses how The SET assesses how well SWPBS Tier well SWPBS Tier 1/primary preventions are 1/primary preventions are being implemented in a being implemented in a building and allows teams building and allows teams to measure progress from to measure progress from year to year. Teams use year to year. Teams use SET results to revise SET results to revise annual action plans.annual action plans.

Schools must continually assess how effective Tier 1/primary preventions are being implemented. Tier 1/Primary preventions are considered in place when schools achieve an 80/80 criteria on the SET. Attaining 80/80 criteria is also associated with a decrease in problem behavior.

At the beginning of implementation for baseline data, and thereafter on an annual basis in October.

The SET is a combination of direct observations and interviews conducted by the district coach or outside consultant. As a result, a school receives a score, chart, and SET report containing the outcomes of the SET with subsequent recommendations.

Commendations, hypotheses, and recommendations should be shared with all faculty and staff. Recommendations are addressed throughout the year.

Example Elementary School SET Results 2 Year Comparison

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

ExpectationsDefined

BehavioralExpectations

Taught

OngoingSystem forRewardingBehavioral

Expectations

System forResponding to

BehavioralViolations

Monitoring &DecisionMaking

Management District LevelSupport

Mean

Per

cen

tage

in P

lace Year 1

Year 2

Year 3

Year 4

Year 5

80/80 Criteria80/80 Criteria

Oakland High School SET Results 2 Year Comparison

0% 0% 0%

38%

25%

50%

0%

16%

100%

30%

67%

88%

75%

69%

50%

68%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

ExpectationsDefined


Taught

On-goingSystem forRewardingBehavioral

Expectations




Management District-LevelSupport

Mean

SWPBS Feature

Pre

cen

t in

Pla

ce

2006-2007

2007-2008

Example High School SET Results 2 Year Comparison

Locate your SET Features Chart.

Locate your SET Features Chart and features thumbnails.

When we tell you to move, please take this and a writing utensil with you.

Find your PLATEPLATE partner.

Mark the statements on each feature slide in your thumbnails with “+” if you think it is present and measureable in your school and “-” if it is not present or measureable.

On the SET Features Chart write down what you need to do in your school to score well on the SET.

Be prepared to share.

Return to your school team and discuss what evidence you need and compare what each of

you generated.

Expectations Defined• All faculty and staff have had input and agreed

upon 3-5 school-wide rules• All faculty and staff have had input and agreed

upon the behavioral expectations that define those rules

• The rules AND expectations are posted all over the school

• The rules AND expectations are posted in formats that ALL students can “read” e.g., pictures, symbols, words, etc.

•All students can recite the rules and expectations.•All adults can recite the rules and expectations. •There is a SYSTEM/plan for teaching ALL students the behavioral expectations.

•The system/plan is codified in teacher handbooks or other source.

•All faculty and staff are involved in some way in teaching the behavioral expectations.

•The primary team reviews pertinent information with the full faculty and staff at the beginning of the year.

•All new faculty and staff are taught the discipline system.

Behavioral Expectations Taught

System for Rewarding/Acknowledging Behavioral Expectations

• There is a SYSTEM/plan for rewarding student BEHAVIOR.

• The system/plan is codified in teacher and student handbooks or other source.

• Students receive rewards.• ADULTS in the building give out rewards.

System for Responding to Behavioral Violations

• There is a SYSTEM/plan for responding to rule violations.

• The system/plan is codified in teacher and student handbooks or other source.

• Adults agree with administration on what is office managed and what is staff managed.

• There is a documented crisis plan.• Adults agree with administration on how to

respond to extreme emergencies.

Monitoring and Decision Making

• School uses a discipline referral form that includes: student, grade, date, time, referring staff, problem

behavior, location, persons involved, probable motivation, &administrative decision

• School uses a clearly defined system for collecting and summarizing discipline referrals.

• Administration explains how that system works.• Team shares data from discipline system (above) with

all faculty and staff at least 3 times per year.• Team uses discipline data to make decisions about

school-wide issues.

Management• School improvement plan includes school-wide discipline as one of

top three goals.• All adults are aware of the primary preventions team and it’s

functions.• The primary preventions team includes representation of the entire

school.• The primary preventions team has at least one team leader that is

accepted by all team members.• The administrator is an ACTIVE member of the primary preventions

team.• The team meets at least monthly.• The team reports progress of primary system to entire staff at least

three times per year.• The primary team has an action plan that guides their work that is

less than one year old.

District Level Support

• School budget contains allocation to support and sustain SWPBS, at all three tiers.

• The administration identifies an out of school liaison who supports the team and school in implementing SWPBS.

Making Effective, Data-Making Effective, Data-Based Decisions from SET Based Decisions from SET

ResultsResults1. “Coach” reviews status and defines

problems with precision2. “Coach” uses accurate and current data

to identify and refine problem statements

3.3. The team should understand how scores The team should understand how scores and recommendations were determined and recommendations were determined and have choices of recommendations.and have choices of recommendations.

4. Use annual SET data as a source of accountability for evaluating the implementation and impact of the SET recommendations selected.

Precision Problem Statements(What data do we need to make effective

decisions based on SET results?)– What feature(s) is/are less than 80% in place?– What practice(s) was/were least evident

within each feature?– Who is responsible for putting the practice(s)

into place? For ensuring the practice(s) are accurately implemented?

– Where were the practices most likely and least likely to be evident.

– When were the practices most likely and least likely to occur.

Example Recommendations:Although 100% of staff stated that they had given “Hawk Wings" to students for following the school-wide rules this year, only 33% (5/15) of students reported they had actually received one. Consider one or more of the following actions:1.All faculty and staff increase the rates and

frequencies with which “Hawk wings” are given out to students.

2.Ensure that students who routinely follow rules receive “Hawk wings", not just students who frequently break the rules.

3.Codify your incentive system4. Ensure all faculty and staff have access to Hawk

Wings and know how to secure more when needed.

1. Get out your SET report.2. Review your recommendations.3. Prioritize recommendations using Delphi

Weighting, checkmark method or Focusing Four.

4. Generate Tasks. Include (a) WHO will complete, (b) WHEN action/task will be completed, and (c) DATA documenting evidence of task completion.

5. Record these on your team task list.6. Complete the SET portion of the

Planning Organizer Share

Data Analyst Function/Role Data Analyst Function/Role

1.Understand how the report was scored. Ask questions for the team in preparation for team member questions.2.Understand the origins and content of the recommendations. Ask the coach for clarification in preparation for team member questions.3.Facilitate prioritizing recommendations for the year.

??????

The SSS is completed by ALL faculty and staff and 4 students from each homeroom/class in grades 4 and above.

The School Safety Survey is a web-based instrument developed to obtain an efficient index of perceived school safety and to generate actions creating safe/safer schools.

The SSS is a web-based survey completed by individuals. Students in upper grades can complete the survey by filling in responses like the adults. Students in lower grades will take a different version of the survey that provides more extensive explanation of questions.

Annually in March/April

This survey provides a summary of "risk factors" and “response plans" that can be useful in determining training and support needs related to school safety and violence prevention.

The primary preventions team should review the risk factors and corresponding responses and generate tasks accordingly.

Get out y

our

School S

afety Survey From Survey Monkey.ComSpend three minutes reviewing the

survey and generating questions/potential questions from your faculty, staff, or students.

???????

School Safety Data Participants

Roles of Participants for School Safety Survey

0.7%

19.7%

3.3%

4.6%

1.3%

0.0%

1.3%

67.8%

0.0%

0.0%

1.3%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Administrator

Techer

Special Education Teacher

Educational Assistant

Off ice Staff

Custodial Staff

Related Service Provider

Student

Parent

Community Member

Other

% of Roles

School Safety Results Risk Factors by Adult and Student Participants

Response Plans by Adult and Student Participants

Section 2: Response Plans for School Safety and ViolenceExample Elementary January 2008

1.4%

24.4%

46.9%

24.3%

3.0%7.1%

20.4%

30.1%24.8%

17.7%

0%

20%

40%

60%

80%

Not at all Minimally Moderately Extensively Don't know

% o

f R

esp

on

ses

Adult Student

Section 1: Individual Risk Factors for Safety & Violence

83% 83%

65%

71%

56%

66%

80%76%

83%

75%

31%34%

39%41%

55%

61%

81%

68%

83%

41%

74% 75%

27%

44%

71%

77%

49%

63%

47%

53%

38%42%

22%

48%

6% 5%

27%

17%

38%

8%10%

13%

6% 6%

42%

15%

53%

46%

16%

2%0%

4%

25%

8% 9% 8%

53%

15%19%

10%

27%

14%

43%

33%

47%

34%37%

14%

2% 2%

8%5% 4% 4%

10%8%

2%0%

23%

11%

6%

11%

2%0% 0% 1% 2% 1% 0% 1%

16%

7%

2% 3%

10%

4%6%

13%15% 17%

41%

7%

0%

20%

40%

60%

80%

100%

Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student

(1) IllegalWeapons

(2)Vandalism

(3) HighStudentMobility

(4) Graffiti (5) GangActivity

(6) Truancy (7)Suspensions

and/orExpulsions

(8) StudentsAdjudicated

by Court

(9) ParentSchoolSafety

Concerns

(10) ChildAbuse

(11)Trespassing

(12) Poverty (13) Crimes (14) IllegalDrugs &Alcohol

(15) Fights,Conflict, &

Assault

(16) Bullying (17) PoorSchool

Condition

% o

f Rat

ing

Not at all to Minimally Moderately Extensively

School Safety Results

Section 2: Individual Response Plans for School Safety & Violence

0%2%

0%

5%

0%

6% 6%3% 2%

7%

2%

12%

2% 3%0%

6% 6%

25%

0%

5%

0%

5%

0%

8%

2% 1% 2%

12%

0%4%

0%

12%

6%10%

22%

17% 17% 17%

29%

19% 20% 20%

59%

31%

27%23% 22%

25% 25%

17% 18%21%

16% 18%

22%20%

31% 31%

47%

17%20%

17%

8%

26%

94%

85%

78%

44%

83%

61%65%

58%

78%

64%

39% 40%

67%

62%

78%

58%

40%

25%

82%

68%

84%

67%

76%

53%

67%

52%

41%

23%

80%

65%

90%

52%

0%

20%

40%

60%

80%

100%

Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student Adult Student

(18)

Extracurricular

Activities

(19) Staff

Training

(20) Emergency

Response Plans

(21) Discipline

Plans

(22) Student

Support

Services

(23) Parent

Involvement

(24) Student

Preparation for

Crises

(25) Supervision

of Students

(26) Suicide

Prevention

(27) Student

Participation in

Academic

Activities

(28) Positive

School Climate

(29)

Acceptance of

Diversity

(30) Response to

Conflict

(31)

Collaboration

with Community

(32) High

Student

Expectations

(33) Eff ective

Student-

Teacher

Relationships

% o

f Rat

ing

Not at all Minimally Moderately to Extensively


Section 3: "Don't Know" Respnses to Risk Factors and Response Plans

9%

0%2%

0%

9%

4%2%

27%

19%

29%

17%

4%

8%

14%

4%

0% 0% 0% 0% 0% 0% 0% 0%

4%

0%

29%

0% 0%2%

0%

10%

0%2%

11%8%

22%

4%

19%

41%

3%

37%

27%

50%

16%

34%

10%

20%

1%

7%

32%

3%

35%

17%

21%

9%

17%

12% 11%

34%

7%10%

20%

16%

49%

15%

10%

0%

10%

20%

30%

40%

50%

60%

% o

f Rat

ing

Adult Responses Student Responses


SSS Data Review ManualSSS Data Review Manual

• A step-by-step guide for making sense of the survey results

• Uses graphs from the School Safety Excel document, sent to your coaches, to identify strengths and weakness in school safety and violence prevention

• Bridges team thinking from recognizing weaknesses to developing an action plan to address the weakness

1. With the “Sample School Safety Survey Data Review Manual” find your Knife partner.

2. With your Knife partner, complete the “Sample SSS Data Interpretation Manual“ as directed by the presenter. For this activity, please stay with the group as best you can.

School Safety Survey School Safety Survey Interpretation:Interpretation:RespondentsRespondentsRole Table:

1.What percentage of the adults completed the safety survey? The goal is for all adults in the school to complete and return the survey. ______% 89

In this example, there are 55 adults

working in the school.The first thing to look at

when reviewing the safety survey results is the number of adults and

students who gave feedback.

School Safety Survey Interpretation:School Safety Survey Interpretation:RespondentsRespondentsRole Table:

2.How many students per certified teacher/homeroom teacher completed the survey? The goal is that 4 students per homeroom teacher will complete and return the survey. ________ students and ________% of students suggested (# of students/ (# of homeroom teachers x 4))

103 86

In this example, there are 30 homeroom teachers.The first thing to look at

when reviewing the safety survey results is

the number of adults and students who gave

feedback.

School Safety Survey Interpretation:School Safety Survey Interpretation:Risk FactorsRisk Factors

Section 1: Risk Factors for School Safety and Violence

13.8%

44.2%

26.8%

8.8% 8.7%

27.4%32.5%

14.6%5.5%

20.0%

0%

20%

40%

60%

80%

100%


% o

f R

esp

on

ses

Adult

Student

1.Summarize Combined Data and Look for Differences between Adults & Students

• What percent of adults and students thought risk factors were moderately to extensively present? ______% of adults and ______% of students

• What percent of adults and students thought risk factors were not at all to minimally present? ______% of adults and ______% of students

• What percent of adults and students did not know if risk factors were prevalent in their school? ______% of adults and ______% of students

35.6 20.1

58.0 59.9

8.7 20.0

School Safety Survey Interpretation:School Safety Survey Interpretation:Risk FactorsRisk Factors

Section 1: Risk Factors for School Safety and Violence

13.8%

44.2%

26.8%

8.8% 8.7%

27.4%32.5%

14.6%5.5%

20.0%

0%

20%

40%

60%

80%

100%


% o

f R

esp

on

ses

Adult

Student

2.Determine the General TrendFor risk factors, recognize that you are looking for both adult and student bars to be higher on the “not at all” and “minimally” end and lower on the “moderately” and “extensively” end. Ignore the “don’t know” responses for this trend.

• What is the general trend for your school’s responses?

The general trend is in the correct direction, with there being fewest “extensively” responses. However, our goal should be to increase “not at all” responses.

School Safety Survey Interpretation:School Safety Survey Interpretation:Review of Individual Risk FactorsReview of Individual Risk Factors

1. Always start by looking at the positives (what is going well)

If the green lines or “not at all to minimally” data are above the 80% goal line, indicated in blue, the risk factor appears to be under control to the adults or students.

• Which risk factors are not prevalent? According to whom?(1) Illegal Weapons – both students and adults (2) Child Abuse – adults only(3) Gang Activity – adults (student response at 73%)


2. Next, pay close attention to the red lines or “extensively” data. These are the areas that need to be addressed by the primary team making recommendations for action school-wide. Pay particular attention to item #16 “Bullying”

• Which risk factors were prevalent? According to whom?(1) Poor School Condition – adults (2) Truancy – adults (3) Bullying – both


2. (continued) • What are some possible actions?(1)Poor School Condition – repaint, use detentions for clean

up(2) Truancy – increase incentives for being on time, involve

parents(3) Bullying – inservice on responding to bullying,

incorporate anti-bullying into expectations matrix

School Safety Survey Interpretation:School Safety Survey Interpretation:Review of IndividualReview of Individual Risk FactorsRisk Factors

3. Are differences seen between adult and student perceptions?

This may be a sign that adults in the school are not present in situations where the risk factor is seen the most.

• Which risk factors are perceived by students to be more prevalent than by adults?Fights, conflicts, and assaults – small difference, is this difference

significant to my school?

1. 89% of the adults and 86% of the students responded to the SSS. 2. 35.6% of the adults and 20.1% of students thought the risk factors were moderately to extensively present. 58% of the adults and 59.9% of the students thought the risk factors were not at all to minimally present. 8.7% of the adults and 20% of the students did not know if risk factors were prevalent in their schools. 3. In general the trend for risk factors is going in the expected direction, however, there is a need for an increased percentage of students and adults reporting that risk factors are not at all or minimally present. 4. Risk factors least prevalent reported by both students and adults were illegal weapons. Gang activity was next least prevent reported by the adults and students. Adults reported child abuse to be least prevalent. 5. Risk factors most prevalent were poor school conditions and truancy reported by adults and bullying reported by both adults and students; however, fights and assaults were reported slightly more prevalent by the students than by the adults. Potential solutions:

SSS Precision Statements – Risk Factors

School Responses to SSSSchool Responses to SSSPositive Behavior SupportStudent Feedback Survey

Answer each question honestly and in detailed responses. Please write in print.

1. Define gang.2. Are there gangs at S.C.H.S.?3. What gang activities are present? Explain.4. Have you been personally affected by gang activity? Explain.

School Safety Survey Interpretation:School Safety Survey Interpretation:Response PlansResponse Plans

1.Summarize Combined Data and Look for Differences between Adults & Students

• What percent of adults and students thought response plans were moderately to extensively present? ______% of adults and ______% of students

• What percent of adults and students thought response plans were not at all to minimally present? ______% of adults and ______% of students

• What percent of adults and students did not know if response plans were prevalent in their school? ______% of adults and ______% of students

71.2 54.9

25.8 27.5

3.0 17.7

Section 2: Response Plans for School Safety and Violence

1.4%

24.4%

46.9%

24.3%

3.0%7.1%

20.4%

30.1%24.8%

17.7%

0%

20%

40%

60%

80%

100%


% o

f R

esp

on

ses

Adult

Student

School Safety Survey Interpretation:School Safety Survey Interpretation:Response PlansResponse Plans

2.Determine the General TrendFor response plans, recognize that you are looking for both adult and student bars to be higher on the “moderately” and “extensively” end and lower on the “not at all” and “minimally” end. Ignore the “don’t know” responses for this trend.

• What is the general trend for your school’s responses? The general trend is in the correct direction, with there being fewest “not at all” responses. However, our goal should be to increase “extensively” responses.

Section 2: Response Plans for School Safety and Violence

1.4%

24.4%

46.9%

24.3%

3.0%7.1%

20.4%

30.1%24.8%

17.7%

0%

20%

40%

60%

80%

100%


% o

f R

esp

on

ses

Adult

Student

School Safety Survey Interpretation:School Safety Survey Interpretation:Review of Individual Response Review of Individual Response

PlansPlans

1. Always start by looking at the positives (what is going well)

If the green lines or “moderately to extensively” data are above the 80% goal line, indicated in blue, the response plans appears to be well recognized by the adults or students.

• Which response plans are prevalent? According to whom?(1)Extracurricular Activities – both students and adults(2)Effective student-teacher relationships – adults (3)

Positive School Climate – adults


PlansPlans

2. Next, pay close attention to the red lines or “not at all” data. These are the areas that need to be addressed by the primary team making recommendations for action school-wide.

• Which response plans were not recognized? According to whom?(1)Suicide Prevention – students (2) Involved Parents – students

(3) Effective Student-Teacher Relationships – students


PlansPlans

2. (continued) • What are some possible actions?(1)Suicide Prevention – post crisis hotline phone numbers in

student areas(2) Involved Parents – create role of parent liaison to consult

on ways to get parents involved, host an event involving food and child care to get parents input or advice

(3) Effective Student-Teacher Relationships – increase positive to negative ratio, implement 2x10 for teachers and students without a good relationship

School Safety Survey Interpretation:School Safety Survey Interpretation:Review of IndividualReview of Individual Response Response

PlansPlans

3. Are differences seen between adult and student perceptions?

This may be a sign that adults in the school are not aware of student perceptions.

• Which response plans are less recognized by students than by adults?Effective Student-Teacher Relationships – why does that faculty think they have good relationships but the students think the

relationships are not as effective?

School Responses to SSSSchool Responses to SSSCrisis Numbers

Adolescent Crisis Intervention & Counseling Nineline1-800-999-9999Domestic Violence Hotline/Child Abuse1-800-4-A-CHILD (1 800 422 4453)National Adolescent Suicide Hotline1-800-621-4000National Suicide Hotline1-800-SUICIDE (1 800 784 2433)National Child Abuse Hotline1-800-422-4453National Drug Abuse Hotline1-800-662-HELP (1 800 662 4357)Pregnancy Hotline1-800-4-OPTIONS (1 800 467 8466)Rape, Abuse, Incest, National Network (RAINN)1-800-656-HOPE (1 800 656 4673)Self-Injury HotlineSAFE (Self Abuse Finally Ends) Alternatives Programwww.selfinjury.com1-800-DONT CUT (1 800 366 8288)Youth Crisis Hotline1-800-448-46631-800-422-0009

1. Which risk factors or response plans are not recognized by the adults or students?The “don’t know” responses have been ignored up to this point.

• Which items? According to whom?

School Safety Survey Interpretation:School Safety Survey Interpretation:“Don’t Know” Responses“Don’t Know” Responses

(1)Child Abuse in the Home – both (2) Truancy – students

(3) Collaboration with Community – students (4) Suicide Prevention - both

Complete the School Safety Data

Review Manual1. Get with your school team colleagues.

2. Get out blank “SSS Data Interpretation Manual” from your notebook and SSS data from your school packet.

3. Using the SSS data interpretation manual with SSS charts, generate actions for your school.

4. Complete the SSS portion of the Planning Organizer.

Share• What information did you get from the data?

• Does the adult and student information match?

• Which risk factors and response plans should be the highest priority for this school?

• What are some possible actions to address the needs of this school?

• Do the “don’t know” responses show an area that would benefit from a marketing plan?

Completing the Web-Completing the Web-Based School Safety Based School Safety

SurveySurvey• Link to survey will be emailed to coaches in late March.

• Coaches will send link to team members as soon as it is received.

• Survey will need to be completed within 2 weeks – ALL adults and a representative sample of 4 students per homeroom teacher/home-base teacher.

• Excel with collected information from your school will be emailed to coach by the end of April.

School Safety Survey OnlineSchool Safety Survey Online

Excludes child abuse and suicide items

School Safety Survey Online: School Safety Survey Online: Adult & Students in 6Adult & Students in 6thth Grade and Grade and

AboveAbove

School Safety Survey Online: School Safety Survey Online: 44thth – 5 – 5thth Grade Students Grade Students

School Safety Survey Online: School Safety Survey Online: Open Response QuestionsOpen Response Questions

Data Analyst Function/Role Data Analyst Function/Role

1.Be intimately familiar with the content of the SSS.2.Review the comments, looking for striking comments to share with team.3.Generate precision statements based on the SSS data using the SSS Data Review Manual as a guide.4.Facilitate brainstorming to generate possible solutions.5.Facilitate team prioritizing actions for EOY action plan.

??????

The Self-Assessment is completed by ALL faculty and staff with in your building.The Self-Assessment survey will help determine what the staff perceive to be already in place and what the staff believe to be a priority.

At the beginning of implementation for baseline data, and thereafter on an annual basis in April.

Use the Self-Assessment as one tool for evidence of buy-in by faculty and staff.

Eighty percent of faculty and staff must see the goals as measured by the Self Assessment as a high or medium priority to pursue SWPBS efforts, efficiently.

If staff buy in is NOT 80%, then pursue the buy in activities listed on the “Getting Started Flow Chart.” (Available at edprodevelopment.com)

Self-Assessment: When starting the SWPBS

journey...Do we have buy-in??

School-wide Non-classroom Classroom

IndividualWith the combined percentages of “high” and “medium” improvement priority near or above 80% for all 4 systems, this school does have staff buy-in.

82% 82% 79%

86%

In year 2 of implementation and In year 2 of implementation and beyond, the Self-Assessment helps beyond, the Self-Assessment helps to prioritize goals in the 7 features to prioritize goals in the 7 features across the 4 systems:across the 4 systems:

• School-wideSchool-wide• ClassroomClassroom• Non-classroomNon-classroom• IndividualIndividualBoth staff perception & attainability Both staff perception & attainability

areare factored into prioritization.factored into prioritization.

The Self-Assessment is an online survey. Staff members enter their answers into a school account on PBSsurveys.org, and the data is summarized for you!

The data are used to The data are used to create the team’s action create the team’s action plan for the school year. plan for the school year. The priorities found in The priorities found in your Self-Assessment your Self-Assessment drive tasks to help drive tasks to help teams reach the goals teams reach the goals deemed high priority by deemed high priority by the entire faculty and the entire faculty and staff.staff.

School-wide

Nonclassroom

Example School Example School

Classroom

Individual

Example School Example School

Example

Example

Red indicates ≤49% complete.

Yellow indicates 50-79%

complete.

Items are categorized by whitewhite, yellowyellow, and red red for “in for “in

place” place” percentagespercentages.White indicates White indicates ≥≥80% complete.80% complete.

Self-AssessmentSelf-Assessment Individual Item

Scores

Self-Assessment:Self-Assessment:

Individual Item Scores

Self-AssessmentSelf-Assessment::

Individual Item Scores

Self-Assessment Data Review Manual

• A step-by-step guide for making sense of the survey results for schools who have been addressing the implementation of SWPBS for at least 1 year

• Uses graphs from the Self-Assessment on PBSsurveys.org to identify strengths and areas for improvement in Tier 1: Primary Preventions implementation

• Bridges team thinking from recognizing weaknesses to developing an action plan to address the weakness

1. Get out your “Sample Self Assessment Data Review Manual” from your packet.

2. Find your Fork partner and sit together for this activity.

3. With your fork partner, complete the “Sample Self Assessment Data Review Manual” as directed by the presenter. You will have approximately 5 minutes for this.

School-wide

Self-Assessment Data Review Manual:

Comparative SummariesFirst, data are compared from different survey dates to look for

improvement.(1) What systems are showing improvement in the implementation of SWPBS? (In place and partially in place combined)Fall: 96%

Spring: 84%

12% Decrease No improvement

Non-Classroom


Comparative SummariesFirst, data are compared from different survey dates to look for improvement.(1) What systems

are showing improvement in the implementation of SWPBS? (In place and partially in place combined)

Fall: 99%

Spring: 91%


Classroom


Comparative SummariesFirst, data are compared from different survey dates to look for improvement.(1) What

systems are showing improvement in the implementation of SWPBS? (In place and partially in place combined)

Fall: 99%

Spring: 87%


Individual


Comparative SummariesFirst, data are compared from different survey dates to look for improvement.(1) What systems

are showing improvement in the implementation of SWPBS? (In place and partially in place combined)

Fall: 92%

Spring: 69%



Comparative Summaries

(2) If your school has more than 2 years of Self-Assessment data, is there a trend in the change in “Current Status” graphs for each system?

A trend could not be identified in the “Current Status” graphs.

First, data are compared from different survey dates to look for improvement.



(3) In which systems is SWPBS Tier 1: Primary Preventions in place? (i.e. combined percentage of “in place” and “partially” ≥80%) Celebrate these successes!!

83%!!!School-wide





91% in place!!

Non-Classroom



Comparative SummariesFirst, data are compared from different survey dates to look for improvement.


87% !!!

Classroom




69% Individual




(4) For those systems in which SWPBS Tier 1: Primary Preventions is not in place, which have been labeled as a high priority for improvement by the faculty and staff?

Not identified as high priority School-wide


Self-Assessment Interpretation Guide:

Comparative SummariesFirst, data are compared from different survey dates to look for improvement.(4) For those

systems in which SWPBS Tier 1: Primary Preventions is not in place, which have been labeled as a high priority for improvement by the faculty and staff?

Non-Classroom

Not identified as high priority





Classroom



Comparative SummariesFirst, data are compared from different survey dates to look for improvement.



Individual


SET vs. Self-Assessment

(1) Celebrations: What features have shown improvement since the SET was conducted in the fall?(1)Management, (2) District

Support, &

(3) Monitoring/Decision Making

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

ExpectationsDefined


Taught


Expectations





Mean

Perc

en

tag

e in

Pla

ce

06-'07

07-'08

'08-'09

'09-'10

'10-'11


Self-Assessment vs. SET

(1) Improvement Needs: Which features still need to be focused on for improvement?Expectations Taught, Reward

System, & Violations System

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

ExpectationsDefined


Taught


Expectations





Mean

Perc

en

tag

e in

Pla

ce

06-'07

07-'08

'08-'09

'09-'10

'10-'11


Features

(1) Features to Celebrate: Of the 7 features from the Self-Assessment, which features are over 80% in place?(1) Expectations Defined & (2) District Support


Features

(2) Features for Improvement: Label which features your team needs to focus on improving.

(1)Reward System, (2) Violation System, & (3) Management


Item Statements(1) Identify the item statements associated with the features labeled for improvement.The item statements below are associated

with the feature “Reward System”.

Item #4 in Non-classroom and Classroom will be our focus.


Item Statements(2) Brainstorm actions/tasks to improve upon the items identified.

(3) Complete the table by entering item statements and actions/tasks

First,First, Assign each topic a letter of the Assign each topic a letter of the alphabet.alphabet.Second,Second, EACH team member assigns a priority EACH team member assigns a priority value to each item, rank ordering the items value to each item, rank ordering the items from most to least important. The rank from most to least important. The rank ordering, HOWEVER, occurs in a specific ordering, HOWEVER, occurs in a specific manner.manner.Third,Third, EACH team member’s value is added to EACH team member’s value is added to obtain a total sum value for each behavior obtain a total sum value for each behavior identified, thereby creating a rank order identified, thereby creating a rank order prioritization list.prioritization list.Fourth,Fourth, team members are asked if there is a team members are asked if there is a consensus of this weighted outcome. If not, consensus of this weighted outcome. If not, discuss, and weight again until consensus is discuss, and weight again until consensus is obtained.obtained.

• Is there measurable improvement in implementation?• How did the feature graph from the Self-Assessment

compare to that of the SET?• On which features and items should this team focus their

efforts?• What are some possible actions the team could take to

better their implementation of SWPBS?

Complete the Self-Assessment Interpretation Guide and

Planning Organizerwith your team using the data and graphs from your

notebook

Making your life easier, one survey at a time!

Read over the directions for Read over the directions for using PBSsurveys.org…. using PBSsurveys.org….

Any Questions??Any Questions??

??????

1. Get out your EOY Planning Organizer from your notebooks.

2. With your school team colleagues, finish any section of the EOY Planning Organizer you were unable to complete over the last 2 days.

2. Review your “celebrations” and “priority for improvement” pages.

3. Take back to your teams and SHARE!

1. Post test

2. EdPro Development Evaluation

3. TASL Evaluation, TASL Cards, … (TASL seekers)

4. Certificate of attendance

when preparing for this workshop you must either: (1) print registered schools own data or (2) go to...

Documents

dummy data

swis data

quiet cue slide

selfassessment slide

school teams

end of year data

problems early use data

databased decision making