the effects of school-wide positive behavior support on school climate: a middle school longitudinal...
TRANSCRIPT
The Effects of School-Wide Positive The Effects of School-Wide Positive Behavior Support on School Climate: Behavior Support on School Climate: A Middle School Longitudinal StudyA Middle School Longitudinal Study
K. Richard Young, BYUK. Richard Young, BYURyan H. Shatzer, BYURyan H. Shatzer, BYURichard P. West, USURichard P. West, USUEllie L. Young, BYUEllie L. Young, BYU
Paul Caldarella, BYUPaul Caldarella, BYU
FundingFunding
U.S. Department of Education U.S. Department of Education Field Initiated Research Projects Field Initiated Research Projects
OSEP Grant # H324CO30124B OSEP Grant # H324CO30124B
Written by Brigham Young University in Written by Brigham Young University in collaboration with participating school districtcollaboration with participating school district
2003 – 2008 2003 – 2008
ContentsContents
• IntroductionIntroduction• MethodsMethods• Results and DiscussionResults and Discussion• QuestionsQuestions
QuestionsQuestions
• What strategies and resources are needed What strategies and resources are needed to implement a successful SWPBS to implement a successful SWPBS program in middle schools?program in middle schools?
• How can you measure the effects of How can you measure the effects of SWPBS in middle schools?SWPBS in middle schools?
INTRODUCTIONINTRODUCTION
Positive Behavior Support (PBS)Positive Behavior Support (PBS)
Primary Prevention:
School-Wide
Secondary Prevention:Students with
At-Risk Behavior
Tertiary Prevention:Students with
High-Risk Behavior
~80% of Students~80% of Students
~15% ~15%
~5%~5%
School ClimateSchool Climate
• ““A set of internal characteristics that A set of internal characteristics that distinguishes one school from another distinguishes one school from another and influences the behavior of its and influences the behavior of its members” members”
(Hoy and Hannum, 1997)(Hoy and Hannum, 1997)
• Encompasses the dimensions of Encompasses the dimensions of (1) Values (1) Values
(2) Educational practices (2) Educational practices (3) Personal interactions (3) Personal interactions
(National Research Council, 2004)(National Research Council, 2004)
School ClimateSchool ClimateOur definition includes Our definition includes
perceptions of ...perceptions of ...
Outcomes of Healthy School ClimateOutcomes of Healthy School Climate• Improved Student AchievementImproved Student Achievement
– Standardized test scoresStandardized test scores– Reading levelsReading levels– GPAGPA– Academic and school adjustment Academic and school adjustment
(Brand et al., 2008; Esposito, 1999; Hoy & Hannum, 1997)(Brand et al., 2008; Esposito, 1999; Hoy & Hannum, 1997)
• Decreased Student Misbehavior Decreased Student Misbehavior – Student drug useStudent drug use– Aggression and school violenceAggression and school violence– Anti-social behavior Anti-social behavior – Absences and suspensionsAbsences and suspensions– Student delinquency Student delinquency
(Astor et al., 2002; Aveyard et al., 2004; Gottfredson et al., 2005; Sprott, 2004)(Astor et al., 2002; Aveyard et al., 2004; Gottfredson et al., 2005; Sprott, 2004)
METHODSMETHODS
PBS FrameworkPBS Framework• Tertiary-Level InterventionTertiary-Level Intervention
– One-on-one interventionsOne-on-one interventions
• Secondary-Level Intervention (Achievement Plus Class)Secondary-Level Intervention (Achievement Plus Class)– Social Skills Instruction: Prevention Plus and peer Social Skills Instruction: Prevention Plus and peer praise notespraise notes– Organizational Skills: Planners and goal settingOrganizational Skills: Planners and goal setting– Learning Strategies: Study skills and accessing supportsLearning Strategies: Study skills and accessing supports– Emotional Resiliency: “Strong Kids” and journalingEmotional Resiliency: “Strong Kids” and journaling– Behavioral Self-Management: Conflict resolution techniquesBehavioral Self-Management: Conflict resolution techniques– Experiential Learning: Ropes course and service-learning Experiential Learning: Ropes course and service-learning
• School-Wide Intervention (SWPBS)School-Wide Intervention (SWPBS)– Faculty training by PBS support staffFaculty training by PBS support staff– Proactive screening for students at-risk for EDProactive screening for students at-risk for ED– School-wide assemblies, activities, and celebrationsSchool-wide assemblies, activities, and celebrations– Posting school rules and common classroom rulesPosting school rules and common classroom rules– School-wide instruction of social skillsSchool-wide instruction of social skills– Teacher praise notes to studentsTeacher praise notes to students– Administrative Intervention Administrative Intervention – Data-based decision making: Indicators of School Quality (ISQ)Data-based decision making: Indicators of School Quality (ISQ)
This PBS framework was implemented over 4 years at the treatment middle school (2004-2007)
SampleSample
Treatment School Control School
n % n %
Teachers 50 56
Students 1063 1331
GenderMale 548 51.6% 693 52.1%
Female 515 48.4% 638 47.9%
EthnicityCaucasian 934 87.9% 1210 90.9%
Other 129 12.1% 121 9.1%
Special Education 141 13.3% 148 11.1%
Reduced Lunch 402 37.8% 366 27.5%
* The numbers reported are school averages over the four years of the intervention
MeasuresMeasures• School Records School Records
– Grade Point Average (GPA)Grade Point Average (GPA)– AbsencesAbsences– Office Discipline Referrals (ODRs)Office Discipline Referrals (ODRs)
• Indicators of School Quality (ISQ)Indicators of School Quality (ISQ)– Teacher ratings of school climate (7 categories)Teacher ratings of school climate (7 categories)– Student ratings of behavior support (5 categories)Student ratings of behavior support (5 categories)
• Supplemental QuestionnaireSupplemental Questionnaire– Teacher ratings of school climate (3 factors)Teacher ratings of school climate (3 factors)
Indicators of School Quality (ISQ)Indicators of School Quality (ISQ)
• Conducted over 9 YearsConducted over 9 Years• 1,300 Schools1,300 Schools• 33 Utah School Districts33 Utah School Districts• 14 States + 2 International14 States + 2 International• 70,000 Teachers70,000 Teachers• 500,000 Families500,000 Families• 700,000 Students700,000 Students
Indicators of School Quality (ISQ)Indicators of School Quality (ISQ)
The Surveys …The Surveys …
• are shortare short
• include include all all parents, parents, students, faculty, and students, faculty, and other school staffother school staff
• employ age and culturally employ age and culturally appropriate languageappropriate language
• evaluates several evaluates several environment domainsenvironment domains
The Reports …The Reports …
• are easy to readare easy to read
• provide data to compare provide data to compare each school to a normative each school to a normative group of schoolsgroup of schools
• provide a progress provide a progress summarysummary
• summarize the statistical summarize the statistical and practical adequacy of and practical adequacy of the response to the the response to the surveysurvey
Parents volunteer often at the school
Parents know what happens at school
Enough parents participate at parent/teacher conferences
Parents support extracurricular activities
Teachers are knowledgeable about the subjects they teach
Teachers care about students as individuals
Teachers maintain discipline in their classrooms
Teachers are well organized
Teachers enjoy teaching
Students are well behaved
Enough students participate in extracurricular activities
Students are excited about learning
Students have pride in their school
Administration is accessible to parents, students, and staff
Administration is supportive of teachers
Administration is well organized
Administration applies appropriate discipline at the school
Administration has high expectations for all students
This school prepares students for future employment
This school provides a quality education
Instruction at this school is innovative
Instruction at this school challenges students
Staff has access to enough ongoing training
Counselors are accessible to students
Students have adequate computer access
The school has enough quality textbooks
Students have enough extracurricular opportunities
Students and staff feel safe at school
Students feel safe going to and from school
The school is clean and in good repair
The school grounds and hallways are well supervised
Parent SupportParent Support
Teacher Excellence
Student Commitment
Administration
Instructional Quality
Resource Accessibility
Safety
Parent Teacher Student StaffIndicators of School QualityStatus Report
3rd Grade 5th Grade 8th Grade 11th GradeParent Teacher Student Parent Teacher Student Parent Teacher Student Parent Teacher Student
Parent Support
Teacher Excellence
Student Commitment
School Leadership
Instructional Quality
Resource Management
School Safety
ISQ Predicts Academic Achievement
Elements of Behavior SupportElements of Behavior SupportElements Items
Positive Relationships Is there an adult at school whom you trust to ask for help?
Clear Expectations Are you frequently confused about what is expected of you at school?
Building SkillsSocial Do you make friends easily?
Academic Do you do well on tests and assignments?
Rewards & Recognition Are you recognized enough for your efforts to succeed?
RESULTS & DISCUSSIONRESULTS & DISCUSSION
Data AnalysisData Analysis
• Analysis of Variance (ANOVA)Analysis of Variance (ANOVA)– Linear TrendLinear Trend
• Were there significant improvements over the Were there significant improvements over the four years in the treatment middle school?four years in the treatment middle school?
– Interaction EffectsInteraction Effects• Were there significant improvements in the Were there significant improvements in the
treatment school, relative to the control school?treatment school, relative to the control school?
School Record DataSchool Record Data
2004-20072004-2007
School Records SummarySchool Records Summary
• Academic Achievement Academic Achievement – GPAGPA
• AbsencesAbsences– Tardies*Tardies*– Unexcused Absences*Unexcused Absences*
• ODRsODRs– Disorderly Conduct ODRs*Disorderly Conduct ODRs*
*Significant interactions
GPAGPA
3.00
3.05
3.10
3.15
3.20
3.25
3.30
3.35
3.40
2004 2005 2006 2007
Treatment
Control
PBS School: Significant increase in GPA, but no interaction effectLinear trend: F (1, 4457) = 11.81, p < .001, d = .14
TardiesTardies
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
2004 2005 2006 2007
Treatment
Control
Periods per student per year
PBS School:Significant interaction: F (3, 10758) = 77.51, p < .001Decrease in tardies: Linear trend: F (1, 4822) = 70.08, p < .001, d = .32
Unexcused AbsencesUnexcused Absences
2
4
6
8
10
12
14
16
18
2004 2005 2006 2007
Treatment
Control
Periods per student per year
PBS School: Significant interaction: F (3, 10758) = 12.04, p < .001Decrease in absences: Linear trend: F (1, 4822) = 5.85, p < .05, d = .11
Disorderly Conduct ODRsDisorderly Conduct ODRs
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
2004 2005 2006 2007
Treatment
Control
Per student per year
PBS School: Significant interaction: F (3, 10758) = 14.01, p < .001Decrease in ODRs: Linear trend: F (1, 4822) = 11.27, p < .001, d = .14
Practical SignificancePractical Significance
• It was estimated that during the last year It was estimated that during the last year of PBS the treatment school of PBS the treatment school savedsaved::– 167 student and administrative hours due to 167 student and administrative hours due to
the reduced number of ODRsthe reduced number of ODRs– 643 days in the classroom due to the 643 days in the classroom due to the
reduced number of absencesreduced number of absences– 213 hours of class time due to reduced 213 hours of class time due to reduced
tardinesstardiness
*Based on 15 minutes spent per ODR and 5 minutes late per tardy
Indicators of School Quality (ISQ)Indicators of School Quality (ISQ)
Teacher Version: 2004-2007Teacher Version: 2004-2007
ISQ Results SummaryISQ Results Summary
• Teacher VersionTeacher Version– Teacher Excellence*Teacher Excellence*– Instructional Quality*Instructional Quality*– School Leadership*School Leadership*– Student CommitmentStudent Commitment– Parent SupportParent Support– Resource Management *Resource Management *– School SafetySchool Safety
*Significant interactions
Teacher ExcellenceTeacher Excellence
4.2
4.3
4.4
4.5
4.6
4.7
2004 2005 2006 2007
Treatment
Control
PBS School: Significant Interaction: F (3, 306) = 2.75, p < .05 Significant Linear Trend: F (1, 145) = 4.71, p < .05, d = .48
Instructional QualityInstructional Quality
3.6
3.8
4.0
4.2
4.4
2004 2005 2006 2007
Treatment
Control
PBS School:Significant Interaction: F (3, 306) = 3.13, p < .05Significant Linear Trend: F (1, 145) = 8.81, p < .01, d = .64
School LeadershipSchool Leadership
3.8
4.0
4.2
4.4
4.6
4.8
2004 2005 2006 2007
Treatment
Control
PBS School:Significant Interaction: F (3, 306) = 6.17, p < .001 Significant Linear Trend: F (1, 145) = 19.69, p < .001, d = 1.15
Student CommitmentStudent Commitment
3.0
3.2
3.4
3.6
3.8
4.0
4.2
2004 2005 2006 2007
Treatment
Control
PBS School:Non-significant Interaction: F (3, 307) = 1.29, p = .28 Significant Linear Trend: F (1, 145) = 11.56, p < .001, d = .74
Parent SupportParent Support
3.0
3.2
3.4
3.6
3.8
4.0
2004 2005 2006 2007
Treatment
Control
PBS School:Non-significant Interaction: F (3, 307) = 0.37, p =.77Significant Linear Trend: F (1, 145) = 4.07, p < .05, d = .50
Resource ManagementResource Management
3.2
3.4
3.6
3.8
4.0
4.2
2004 2005 2006 2007
Treatment
Control
PBS School:Significant Interaction: F (3, 306) = 3.09, p < 0.05Significant Linear Trend: F (1, 144) = 11.67, p < .001, d = .77
School SafetySchool Safety
3.6
3.8
4.0
4.2
4.4
4.6
4.8
2004 2005 2006 2007
Treatment
Control
PBS School:Non-significant Interaction: F (3, 306) = 2.11, p = .10Non-significant Linear Trend: F (1, 144) = 0.44, p =.51
Behavior Support ItemsBehavior Support Items
Student Survey: 2005-2007Student Survey: 2005-2007
Positive RelationshipsPositive Relationships
60%
65%
70%
75%
80%
85%
90%
95%
100%
2005 2006 2007
Treatment
Control
Positive Relationships: Is there an adult at school whom you trust to ask for help?Percentages are those students that marked “Yes”
Clear ExpectationsClear Expectations
60%
65%
70%
75%
80%
85%
90%
95%
100%
2005 2006 2007
Treatment
Control
Clear Expectations: Are you frequently confused about what is expected of you at school?Percentages are those students that marked “No”
Social SkillsSocial Skills
40%
45%
50%
55%
60%
65%
70%
75%
80%
2006 2007
Treatment
Control
Building Social Skills: Do you make friends easily?Percentages are those students that marked “Yes”
Academic SkillsAcademic Skills
60%
65%
70%
75%
80%
85%
90%
95%
100%
2005 2006 2007
Treatment
Control
Building Academic Skills: Do you do well on tests and assignments?Percentages are those students that marked “Yes”
Rewards and RecognitionRewards and Recognition
40%
45%
50%
55%
60%
65%
70%
75%
80%
2005 2006 2007
Treatment
Control
Rewards and Recognition: Are you recognized enough for your efforts to succeed?Percentages are those students that marked “Yes”
Supplemental QuestionnaireSupplemental Questionnaire
Teacher Survey: 2004-2007Teacher Survey: 2004-2007
Factor AnalysisFactor AnalysisItem Factor 1 Factor 2 Factor 3
Students show respect for each other .82
Students use appropriate social skills .81
Students share responsibility for making school safe .75
Students are motivated to use appropriate social skills .66
Students learn important social skills .64
Everyone at school treats others with positive regard .62
Behavior problems dealt with appropriately .57
School develops links to community .83
School involves families .79
Students have opportunities to express feelings .51
Adults invite discussions of safety .51
School uses positive means for student cooperation .51
Student with special needs receive special help .80
School identifies needs of high-risk students .76
School strives for academic success .65
Educational Assistance
School Communication/
Collaboration
Student Pro-social Behavior
α = .90 .82 .76Only factor loadings over .50 are shownAll eigenvalues > 1; Explains 59.38% of the variance
Supplemental Questionnaire SummarySupplemental Questionnaire Summary
• Teacher SurveyTeacher Survey– Student Pro-social Behavior*Student Pro-social Behavior*– School Communication/Collaboration*School Communication/Collaboration*– Educational Assistance*Educational Assistance*
*Significant interactions
Student Pro-Social BehaviorStudent Pro-Social Behavior
3.2
3.4
3.6
3.8
4.0
4.2
2004 2005 2006 2007
Treatment
Control
PBS School: Significant Interaction: F (3, 399) = 13.84, p < .001Significant Linear Trend: F (1, 185) = 38.63, p < .001, d = 1.20
School Communication/CollaborationSchool Communication/Collaboration
3.4
3.6
3.8
4.0
4.2
4.4
2004 2005 2006 2007
Treatment
Control
PBS School:Significant Interaction: F (3, 400) = 5.55, p < .001Significant Linear Trend: F (1, 185) = 15.14, p < .001, d = .78
Educational AssistanceEducational Assistance
3.8
4.0
4.2
4.4
4.6
4.8
2004 2005 2006 2007
Treatment
Control
PBS School:Significant Interaction: F (3, 398) = 5.20, p < .05Significant Linear Trend: F (1, 184) = 10.64, p < .001, d = .65
OverviewOverview
• School-wide PBS showed moderate School-wide PBS showed moderate improvements in school climateimprovements in school climate– School records (GPA, attendance, ODRs)School records (GPA, attendance, ODRs)– Indicators of School Quality (ISQ)Indicators of School Quality (ISQ)– Supplemental questionnaireSupplemental questionnaire
• Gradual, continuous improvementGradual, continuous improvement• LimitationsLimitations
– Measures of treatment fidelityMeasures of treatment fidelity
QUESTIONSQUESTIONS
QuestionsQuestions
• What strategies and resources are needed What strategies and resources are needed to implement a successful SWPBS to implement a successful SWPBS program in middle schools?program in middle schools?
• How can you measure the effects of How can you measure the effects of SWPBS in middle schools?SWPBS in middle schools?
ReferencesReferencesAstor, R. A., Benbenishty, R., Zeira, A., & Vinokur, A. (2002). School climate, observed risky behaviors, and victimization Astor, R. A., Benbenishty, R., Zeira, A., & Vinokur, A. (2002). School climate, observed risky behaviors, and victimization
as predictors of high school students' fear and judgments of school violence as a problem.as predictors of high school students' fear and judgments of school violence as a problem. Health Education & Health Education & Behavior, 29Behavior, 29(6), 716-736. (6), 716-736.
Aveyard, P., Markham, W. A., Lancashire, E., Bullock, A., Macarthur, C., Cheng, K. K., et al. (2004). The influence of Aveyard, P., Markham, W. A., Lancashire, E., Bullock, A., Macarthur, C., Cheng, K. K., et al. (2004). The influence of school culture on smoking among pupils.school culture on smoking among pupils. Social Science & Medicine, 58 Social Science & Medicine, 58(9), 1767-1780. (9), 1767-1780.
Brand, S., Felner, R. D., Seitsinger, A., Burns, A., & Bolton, N. (2008). A large scale study of the assessment of the social Brand, S., Felner, R. D., Seitsinger, A., Burns, A., & Bolton, N. (2008). A large scale study of the assessment of the social environment of middle and secondary schools. environment of middle and secondary schools. Journal of School Psychology, 46Journal of School Psychology, 46(5), 507-535. (5), 507-535.
Esposito, C. (1999). Learning in urban blight: School climate and its effect on the school performance of urban, minority, Esposito, C. (1999). Learning in urban blight: School climate and its effect on the school performance of urban, minority, low-income children.low-income children. School Psychology Review, 28 School Psychology Review, 28(3), 365-377. (3), 365-377.
Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., & Gottfredson, N. C. (2005). School climate predictors of school Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., & Gottfredson, N. C. (2005). School climate predictors of school disorder.disorder. Journal of Research in Crime and Delinquency, 42 Journal of Research in Crime and Delinquency, 42(4), 412-444. (4), 412-444.
Horner, R. H., & Sugai, G. (2004). School-wide Positive Behavior Support: Building Sustainable Support. www.pbis.org.Horner, R. H., & Sugai, G. (2004). School-wide Positive Behavior Support: Building Sustainable Support. www.pbis.org.
Hoy, W. K., & Hannum, J. W. (1997). Middle school climate: An empirical assessment of organizational health and student Hoy, W. K., & Hannum, J. W. (1997). Middle school climate: An empirical assessment of organizational health and student achievement. achievement. Educational Administration Quarterly, 33Educational Administration Quarterly, 33(3), 290-311.(3), 290-311.
Merrill, K (2003). Strong Teens: A curriculum for promoting social and emotional resiliency in students from grades 9-12. Merrill, K (2003). Strong Teens: A curriculum for promoting social and emotional resiliency in students from grades 9-12. The Oregon Resiliency Project. University of Oregon.The Oregon Resiliency Project. University of Oregon.
National Research Council (2004). Engaging Schools: Fostering High School Students' Motivation to Learn.National Research Council (2004). Engaging Schools: Fostering High School Students' Motivation to Learn. Division of Division of Behavioral and Social Sciences and Education. Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.Washington, DC: The National Academies Press.
Sprott, J. B. (2004). The development of early delinquency: Can classroom and school climates make a difference?Sprott, J. B. (2004). The development of early delinquency: Can classroom and school climates make a difference? Canadian Journal of Criminology and Criminal Justice, 46Canadian Journal of Criminology and Criminal Justice, 46(5), 553-572. (5), 553-572.
Taylor, M., West, R., & Smith, T. (2006). Taylor, M., West, R., & Smith, T. (2006). The Indicators of School Quality (ISQ) Survey ManualThe Indicators of School Quality (ISQ) Survey Manual . Logan, UT: Center for the . Logan, UT: Center for the School of the Future.School of the Future.
Young, K., Marchant, M., & Wilder, L. K. (2003). School-based interventions for students with emotional/behavioral Young, K., Marchant, M., & Wilder, L. K. (2003). School-based interventions for students with emotional/behavioral disorders. In P. Allen-Mears and M. W. Fraser (Eds.) Interventions with children and adolescents: An disorders. In P. Allen-Mears and M. W. Fraser (Eds.) Interventions with children and adolescents: An interdisciplinary perspective. (pp. 175-204). Boston: Allyn & Bacon.interdisciplinary perspective. (pp. 175-204). Boston: Allyn & Bacon.
Young, K., & Shatzer, R. (2008, July).Young, K., & Shatzer, R. (2008, July). School-wide Positive Behavior Support: Effects on Middle School Teachers’ School-wide Positive Behavior Support: Effects on Middle School Teachers’ Perception of School Quality. Perception of School Quality. Poster presented at the Office of Special Education Programs (OSEP) Conference, Poster presented at the Office of Special Education Programs (OSEP) Conference, Washington D.C.Washington D.C.
[email protected][email protected]@[email protected][email protected]
http://education.byu.edu/pbsi
301 MCKB Brigham Young UniversityProvo, UT 84602(801) [email protected]