draft final technical report of the safe schools/healthy ... · he safe schools/healthy students...

Draft Final Technical Report of the Safe

Schools/Healthy Students Initiative

National Evaluation

DRAFT

Submitted November 4, 2010

2005 to 2009 Grantees

Draft Final Technical Report

of the

Safe Schools/Healthy Students

Initiative

National Evaluation

2005 to 2009 Grantees

DRAFT

Submitted December 29, 2010

U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services

Administration, Center for Mental Health Services

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 i

ACKNOWLEDGMENTS

This report was prepared for the Center for Mental Health Services (CMHS), Substance Abuse

and Mental Health Services Administration (SAMHSA), U.S. Department of Health and Human

Services (HHS). The report was developed by MANILA Consulting Group under contract

number 280-05-0122, with staff support from MANILA, RMC Research Group, and Battelle

Centers for Public Health Research and Evaluation.

DISCLAIMER

This report was prepared under contract by MANILA Consulting Group for SAMHSA. The

content of this publication does not necessarily reflect the views or policies of SAMHSA or

HHS.

PUBLIC DOMAIN NOTICE

All material appearing in this report is in the public domain and may be reproduced or copied

without permission from SAMHSA. Citation of the source is appreciated. However, this

publication may not be reproduced or distributed for a fee without the specific, written

authorization of the Office of Communications, SAMHSA, HHS.

RECOMMENDED CITATION

Center for Mental Health Services. (2010). Final Technical Report of the Safe Schools/Healthy

Students Initiative National Evaluation: 2005 to 2009 Grantees. Rockville, MD: Substance

Abuse and Mental Health Services Administration.

ORIGINATING OFFICE

Mental Health Promotion Branch

Division of Prevention, Traumatic Stress, and Special Programs

Center for Mental Health Services

Substance Abuse and Mental Health Services Administration

One Choke Cherry Road

Rockville, MD 20857

Printed 2010

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 ii

TABLE OF CONTENTS

Program Highlights ......................................................................................................................... v

Safer Students, Schools, and Communities .............................................................................. v

Healthier Students .................................................................................................................... vi

Introduction ..................................................................................................................................... 1

The Need for Safe School Environments.................................................................................. 1

Origin of the Safe Schools/Healthy Students Initiative ............................................................ 2

Grant Administration and Requirements .................................................................................. 5

National Cross-Site Evaluation ................................................................................................. 7

Program Theory ........................................................................................................................ 8

Overview of This Report ........................................................................................................ 10

Impact of the Initiative on Students and Schools.......................................................................... 11

Safer Students, Schools, and Communities ............................................................................ 11

Healthier Students ................................................................................................................... 11

Government Performance and Results Act Data .................................................................... 12

Sustainability........................................................................................................................... 12

School Climate ........................................................................................................................ 16

Characteristics of Safe Schools/Healthy Students Grantees and Projects .................................... 17

Characteristics of the Partnerships .......................................................................................... 17

Partnership History in the Community ............................................................................. 17

Partnership Structure and Activities ................................................................................. 19

Characteristics of the Operating Environment ........................................................................ 23

School-Level Involvement and Priorities ......................................................................... 23

Importance of Grant Resources in Schools ....................................................................... 27

Implementing Comprehensive Programs and Activities .................................................. 28

Programs and Activities to Address Safety and Violence Prevention .............................. 30

Programs and Activities to Address Substance Use Prevention ....................................... 34

Activities to Address Access to Mental Health Services .................................................. 36

Programs and Activities to Address Early Childhood Development................................ 38

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 iii

Implementing Coordinated and Integrated Services ......................................................... 40

Implementing Enhanced Services ..................................................................................... 43

Implementing Culturally and Locally Responsive Practices ............................................ 44

Conclusion .............................................................................................................................. 46

Factors that Influence the Effectiveness of the Safe Schools/Healthy Students Grant................. 47

Impact of the Pre-Grant Environment ..................................................................................... 48

Impact of Comprehensive, Coordinated, and Integrated Services .......................................... 48

Impact of Grant Operations .................................................................................................... 48

Conclusion .................................................................................................................................... 52

Acronyms ...................................................................................................................................... 54

Appendix A: Safe Schools/Healthy Students Grantees (2005–2009 Cohorts) ........................... A-1

Appendix B: References ............................................................................................................. B-1

Appendix C: Data Sources .......................................................................................................... C-1

Appendix D: Summary of Data Collection Instrument and Protocol Changes .......................... D-1

Appendix E: Data Collection/Survey Summary .......................................................................... E-1

Appendix F: School Climate Survey Findings ............................................................................ F-1

Appendix G: 2005 to 2006 Cohort Meta-Analysis Results: Year 1 to Year 3 ........................... G-1

Appendix H: Meta-Regression Analysis .................................................................................... H-1

Appendix I: Hierarchical Linear Modeling Analysis Report ........................................................ I-1

Appendix J: Methodology............................................................................................................ J-1

Appendix K: SAMHSA’s Eight Strategic Initiative Areas ........................................................ K-1

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 iv

L IST OF EXHIBITS

Exhibit 1. Geographical Distribution of SS/HS Grantees (2005–2009 Cohorts) .......................... vi

Exhibit 2. SS/HS National Evaluation Program Theory Model ..................................................... 9

Exhibit 3. GPRA Results for Fiscal Year 2009 ............................................................................ 12

Exhibit 4. Grantee Characteristics ................................................................................................ 18

Exhibit 5. Partnership History Prior to Grant ............................................................................... 19

Exhibit 6. Project Director Ratings of Partner Contributions ....................................................... 21

Exhibit 7. Partnership Organization by Grant Year ...................................................................... 22

Exhibit 8. Factors Associated With Partnership Ratings .............................................................. 23

Exhibit 9. School-Level High-Priority Activities ......................................................................... 26

Exhibit 10. School Staff Ratings of the Importance of SS/HS Resources in Relation to Other

Sources of Support in Schools .............................................................................................. 28

Exhibit 11. Number of Activities Implemented by Grant Year .................................................... 29

Exhibit 12. Types and Frequency of Activities Implemented ...................................................... 29

Exhibit 13. Most Frequently Implemented EBPs (2005 and 2006 Cohorts) ................................ 44

Exhibit 14. Factors Associated With Higher and Lower Coordination and Service Integration

Ratings .................................................................................................................................. 49

Exhibit 15. Factors Associated With More Staff-Reported Improvement in Grant Areas ........... 50

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 v

PROGRAM H IGHLIGHTS

he Safe Schools/Healthy Students (SS/HS) Initiative is a unique multiagency response to

the goal of using research-based programs and policies to promote safe and effective

learning environments for children and youth. Established by Congress in 1999 as a joint

program of the U.S. Departments of Education (ED), Health and Human Services (HHS), and

Justice (DOJ), SS/HS is a grant program that has helped more than 350 school districts to

develop and implement comprehensive plans in collaboration with local mental health, law

enforcement, and juvenile justice agencies.

The national cross-site evaluation of the SS/HS Initiative confirms expectations that

communities can make effective use of limited funds through high-functioning partnerships that

bring together key local agencies to serve children and youth. The results offer compelling

evidence of the Initiative’s success, including reduced violence and improved school safety,

improved access to mental health services, and reduced alcohol and other drug use.1 These

improvements stand in contrast to national trends. Data for the same period (2005 to 2009) from

sources such as the Youth Risk Behavior Survey (CDC, 2010) showed no significant

improvements in violence, school safety, or current substance use.

Findings also demonstrate the effectiveness of the grant with regard to the collaboration among

SS/HS partners, improved services and systems, and increased use of data to guide policies and

procedures.

SAFER STUDENTS, SCHOOLS, AND COMMUNITIES

Violent incidents decreased 11 percent.

Fewer students reported they had experienced violence (7 percent decrease) or witnessed

violence (4 percent decrease).

Ninety-six percent of school staff surveyed said SS/HS had improved school safety.

More than 90 percent of school staff surveyed said SS/HS had reduced violence on

campus.

Almost 80 percent of school staff surveyed said SS/HS had reduced violence in the

community.

1 All findings on long-term outcomes are from the national evaluation of SS/HS grantees that received awards in

2005 and 2006.

T

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 vi

HEALTHIER STUDENTS

The number of students receiving school-based mental health services increased 263

percent.

The number of students receiving community-based mental health services increased 519

percent.

Almost 90 percent of school staff surveyed reported improved detection of mental health

problems.

More than 80 percent of school staff surveyed said they saw reductions in alcohol and

other drug use.

Almost 70 percent of school staff surveyed said early childhood development had

improved.

Exhibit 1. Geographical Distribution of SS/HS Grantees (2005–2009 Cohorts)

Draft Final Technical Report of the SS/HS Initiative National Evaluation: December 2010 1

INTRODUCTION

he Safe Schools/Healthy Students (SS/HS) Initiative, developed as a collaboration of the

U.S. Departments of Education (ED), Health and Human Services (HHS), and Justice

(DOJ), strengthens the role of community schools as healthy environments supporting the

academic, social, and emotional growth of students. Since 1999, the SS/HS Initiative has

awarded over $2 billion in grants to more than 350 school districts in partnership with their local

mental health, law enforcement, and juvenile justice agencies. These collaborations have in turn

led to the implementation of locally designed, comprehensive plans that contribute to safe,

respectful, and drug-free school environments while promoting vital social skills and healthy

childhood development.

This report describes the SS/HS Initiative and presents findings from the SS/HS national

evaluation, which seeks to quantify the results of the Initiative and explore the factors that

contributed to the grantees’ success. These findings, focusing primarily on 59 sites that received

grant awards in 2005 and 2006 (the first cohorts under the current evaluation for which data

collection has been completed), indicate that the SS/HS Initiative is making a meaningful

difference in many communities. The results also show the Initiative is meeting congressional

expectations for an innovative, community-centered approach that breaks traditional bureaucratic

barriers to achieving lasting improvement in our Nation’s schools.

THE NEED FOR SAFE SCHOOL ENVIRONMENTS

America’s schools should be secure environments where young people can develop their full

potential. But schools are not always secure and safe places for children and adolescents.

Although recent data indicate that the incidence of violent crimes in schools decreased from

1992 to 2007, students are now more likely to experience nonfatal crimes (including theft, simple

and aggravated assault, sexual assault, and rape) in school than outside of school. During the

2007–2008 school year, 85 percent of public schools in the United States recorded that at least

one crime occurred at their school (Dinkes, Kemp, Baum, & Snyder, 2009). In 2007 for the first

time in 15 years, rates of violent crime victimization were higher at school than away from

school. Reported bullying in schools is also on the rise. Whereas in 2001 only 14 percent of

students aged 12 through 18 reported they had been bullied in school (DeVoe, Kaffenberger, &

Chandler, 2005), in 2007 that figure rose to 34 percent, and 4 percent reported they had been

cyber-bullied (Dinkes et al., 2009).

Violence and disruptive, aggressive behaviors such as bullying create a hostile school climate

that interferes with the academic performance and mental health of students. Students who are

exposed to high levels of violence and aggressive behaviors at school, as either victims or

witnesses, are more likely to disengage from school and to experience clinical levels of mental

and emotional disorders than students who experience either no or low levels of violence at

T


school (Bowen & Bowen, 1999; Flannery, Wester, & Singer, 2004; Furlong & Morrison, 2000;

Janosz, Archambault, Pagani, Pascal, Morin, & Bowen, 2008; Morrison, Furlong, & Morrison,

1994). In the classroom, disruptive and aggressive behaviors rob teachers and students of critical

instruction and learning time.

Public and private efforts to address these types of issues often take the form of grants to a

specific type of agency to counteract a specific problem. An effort to address bullying, for

example, might provide grant funds to schools for bullying prevention activities; a program to

reduce youth substance abuse might offer grant funds to law enforcement agencies for training to

prevent drug use. While some programs have made significant contributions, some have had

little relevance to local needs or have encouraged competing, uncoordinated efforts by multiple

grant recipients in the same jurisdiction. A concerted effort to improve school environments in a

wide variety of communities requires the flexibility to focus on community needs and the

incentive to encourage communitywide coordination.

ORIGIN OF THE SAFE SCHOOLS/HEALTHY STUDENTS INITIATIVE

Congress enacted the SS/HS Initiative in 1999 in response to a series of tragic school incidents.

During the 1997–1998 school year, students killed 12 people and wounded 47 others in shooting

rampages in Paducah, KY, Jonesboro, AR, Pearl, MS, and Springfield, OR. The widespread

locations—in rural, suburban, and urban areas—and the absence of either gang membership or

previous criminality among the shooters changed public perceptions of school violence.

America’s young people appeared to be at risk. Members of Congress, senior officials in Federal

agencies, and community leaders were united in seeking an innovative approach to address the

issue.

In September 1998, leaders from the four communities where the shootings had occurred met at

the White House with officials from ED, HHS, and DOJ. The delegations offered suggestions for

how the Federal Government could help prevent similar incidents in the future. The following

month, Congress appropriated funds for ED and the Center for Mental Health Services (CMHS)

of the Substance Abuse and Mental Health Services Administration (SAMHSA) within HHS to

work with DOJ on the creation of a new violence prevention initiative.

The design of the resulting SS/HS Initiative was based on research showing safe school

environments are essential to promoting healthy development and academic success, and

ensuring students and their families feel connected to their school and community. Three key

features set SS/HS apart from other programs authorized by Congress to meet the needs of

children and youth:

1. First, the grant requires schools to take an empirically driven public health approach.

Grantees begin by reviewing data and talking with community stakeholders to identify

the most urgent local needs. Grantees then select and implement best practices and

evidence-based programs (EBPs) that match those needs. The programs are backed by


research showing they actually reduce violence, substance use, or mental health issues or

enhance child development.

2. Second, SS/HS emphasizes long-term systems change. Participating schools and local

agencies coordinate and integrate their services, enabling them to respond quickly and to

remain engaged. Sharing information and resources potentially lowers local costs and

helps the community accomplish more with existing funding.

3. Third, SS/HS stresses data-driven decisionmaking. Grantees are required to continually

monitor progress in meeting their goals and to use data to make modifications to improve

their SS/HS project. They are also encouraged to share those data with their partners and

the community to keep them informed and involved in the project.

A cornerstone of the SS/HS Initiative is the requirement that the grant must be implemented by a

school-community partnership including representatives of the local education agency (LEA)—

usually a public school district or consortium of districts—mental health agency, law

enforcement agency, and juvenile justice agency. The partnerships often include additional

community-based organizations, and each partnership is responsible for planning, implementing,

and monitoring a comprehensive intervention to fulfill the vision of the SS/HS Initiative: ―To

promote the mental health of students, to enhance academic achievement, to prevent violence

and substance use, and to create safe and respectful climates through sustainable school-family-

community partnerships and the use of research-based prevention and early intervention

programs, policies, and procedures.‖

To ensure a comprehensive approach building on the strengths of community partners, SS/HS

grantees are expected to integrate core elements into their projects, including:

Creating safe and violence-free schools. The level of disruptive and aggressive behaviors

of students and how schools respond to such behaviors are directly related to the potential

for violence in a school. Because students’ experiences of violence and their perceptions

of a school’s safety are strongly associated with their academic achievement and

socioemotional and behavioral adjustment (Brand, Felner, Shim, Seitsinger, & Dumas,

2003), it is imperative that schools implement effective, comprehensive violence

prevention programs to improve the safety of the school and reduce aggressive and

violent behaviors in children and adolescents.

Preventing and reducing alcohol, tobacco, and other drug use (ATOD). Research has

shown a strong link between alcohol and drug use and disruptive behaviors, aggression,

and school violence. The use of alcohol and drugs puts children and adolescents at risk

for school failure and involvement in delinquent and violent behaviors, such as fighting,

carrying weapons, and stealing or damaging property (Komro, Williams, Forster, Perry,

Farbakhsh, & Stigler, 2000). Conversely, children and youth who witness violence are

more likely than others to use or abuse substances (Sullivan, Kung, & Farrell, 2004;

Taylor & Kliewer, 2006).


Enhancing early childhood social and emotional learning and development. The

foundations for aggressive and disruptive behaviors and risk for being bullied develop

early. Research has shown that children who enter kindergarten without the adequate

capacity to develop social relationships, to focus their attention on tasks, to effectively

communicate their own emotions or empathize with peers, or to solve social conflicts or

problems are likely to experience academic difficulties and peer rejection during their

elementary schools years (Hemmeter, Ostrosky, & Fox, 2006). Inadequate

socioemotional skills put young children at significant risk for becoming victims of

bullying; becoming depressed, anxious, and disengaged in school; and displaying

behavioral problems, aggression, delinquency, substance abuse, and a host of conduct

problems during adolescence (McClelland & Morrison, 2003; Dodge & Petit, 2003;

Kochenderfer & Ladd, 1996; Laird, Jordan, Dodge, Pettit, & Bates, 2001).

Enhancing mental, emotional, and behavioral health. Many students come to school with

mental, emotional, or behavioral difficulties that put them at risk for engaging in

disruptive, aggressive, and sometimes violent behaviors (Tolan & Gorman-Smith, 2002).

At the same time, students who feel unsafe in school because of the aggressive,

disruptive, or bullying behaviors of other students are at risk for experiencing a range of

mental, emotional, and behavioral disorders including depression, anxiety, aggression,

and truancy (Flannery et al., 2004). Growing evidence shows that school mental health

programs improve educational outcomes by decreasing absences, reducing discipline

referrals, and improving test scores (Paternite, 2005; Rones & Hoagwood, 2000).

Connecting family, schools, and communities. The factors that contribute to students’

disruptive and aggressive behaviors have roots not only in the structure and operations of

the school, but also in the community in which the school is embedded and the

characteristics of students’ families (Laub & Lauritsen, 1998). Consequently, creating a

safe school environment requires more than the efforts of school personnel. Research has

suggested that prevention efforts are most effective when families, schools, community

organizations, and health care and service systems work together to implement programs

and activities to help students (Epstein, 1995; Weissberg, Kumpfer, & Seligman, 2003).

It is critical to select programs that have proven to be effective in creating positive change. The

Federal No Child Left Behind Act of 2001 and other grant programs require educational

practitioners to use interventions based on scientifically verified evidence. Strategies, activities,

curricula, programs, and services included in each SS/HS grantee’s comprehensive plan are

expected to meet the criteria of a well-defined theory or model; provide evidence based on sound

research; and demonstrate cultural, gender, and age appropriateness for the target populations.

Research suggests that schools that put in place comprehensive violence prevention and

intervention services and response plans improve school climate and the morale of school

personnel and students. These improvements in turn reduce dangerous and disruptive behaviors


among students, including physical attacks, suicide, and use of drugs and alcohol (Dwyer &

Osher, 2000).

Because of the myriad intersecting factors that potentially contribute to antisocial behaviors

among young people, LEAs alone do not have the capacity to plan and implement the wide range

of interventions required by the SS/HS Initiative. Efforts to optimize student well-being and

coping strategies by improving access to services for students with mental, behavioral, or

developmental disorders, for example, require LEAs to link with mental health professionals.

Establishing processes and strategies to prevent, prepare for, or respond to threats, incidents of

violence, or crisis and emergency situations requires schools systems to incorporate the insights

and expertise of local law enforcement agencies. Providing an academic environment tailored to

the individual needs of adjudicated students while coordinating their intervention services calls

for LEAs to access the special knowledge and skills of juvenile justice agencies. Working with

families and communities to model appropriate behaviors, engage in problem-solving, and break

the cycle of violence necessitates partnering with parents and community organizations.

School-community partnerships bring together the varying capacities, approaches, and missions

of diverse organizations and individuals to identify some of the issues that contribute to

antisocial behavior, provide a platform for achieving consensus on shared goals and approaches,

and establish frameworks for action (Lasker & Weiss, 2003a). However, the effectiveness of

cross-agency partnerships in addressing broad health and social problems has been unclear

(Mitchell & Shortell, 2000). Partnerships have the potential to falter because they are

relationship-based and resource-intensive, and they often differ from the ways in which people

are accustomed to working (Kreuter, Lezin, & Young, 2000; Mitchell & Shortell, 2000;

Wandersman, Goodman, & Butterfoss, 1997).

Nonetheless, there is widespread belief that attaining common goals and sustaining collaboration

expand the capacity of the partnering organizations to address multidimensional issues (Lasker &

Weiss, 2003b). Federal, State, and private foundations have increasingly required agencies to

collaborate in order to receive funding (Butterfoss, Goodman, & Wandersman, 1993). Research

and theory suggest there is great potential for partnerships to maximize power through joint

action and to minimize duplication of services (Lasker, Weiss, & Miller, 2001; Butterfoss, 2007).

Harnessing the capacities of these partners to create what Putman (2000) calls social capital may

be essential to maintaining school environments that are safe and foster the well-being of

students.

GRANT ADMINISTRATION AND REQUIREMENTS

The SS/HS Initiative is a collaboration of three Federal agencies: ED, HHS, and DOJ. Together,

these partners ensure that Federal funding is channeled to schools and communities to provide

comprehensive services promoting health and safety for children and youth.


To be eligible for funding in fiscal year (FY) 2005 and FY 2006, grant applicants were required

to be LEAs (usually a school district or group of districts) but could not be a current SS/HS

grantee. Applicants proposed a plan that addressed the community’s needs and gaps in each of

the grant elements described below within funding levels based on the district’s urbanicity

designation (a measure of population density).2 The LEAs were required to submit memoranda

of agreement from local partners in law enforcement, juvenile justice, and mental health services

to demonstrate their commitment to the SS/HS Initiative. Applicants were eligible to receive

funding for up to 3 consecutive years, with continuation funding subject to the availability of

Federal funds and progress achieved by the grantee.3

Once awarded funding, grantees were required to address six grant elements:

1. Safe school environment

2. Alcohol and other drug prevention, violence prevention, and early intervention programs

3. School and community mental health preventive and treatment intervention services

4. Early childhood psychosocial and emotional development programs

5. Supporting and connecting schools and communities

6. Safe school policies

Grantees were also required to comply with the Government Performance and Results Act

(GPRA) of 19934 by reporting data annually on a set of measures related to the grant goals. The

measures for the 2005 and 2006 cohorts were as follows:

Decrease in the number of violent incidents at schools

Decrease in substance abuse

Improvement in school attendance

Increase in mental health services to students and families

In FY 2007, based on preliminary findings from the national evaluation and lessons learned from

grantees, Federal Project Officers, and other stakeholders, some grant requirements changed.

Several key changes are highlighted below:

The program elements were refined. The current program elements are as follows: safe

school environments and violence prevention activities; alcohol, tobacco, and other drug

2 Maximum funding amounts for subsequent cohorts have been based on district enrollment rather than urbanicity.

3 The 2007 and subsequent cohorts are eligible for funding for up to 4 consecutive years.

4 GPRA is intended to help improve accountability for the expenditures of public funds; enhance congressional

decisionmaking through more objective information on the effectiveness of Federal programs; and promote a new

Government focus on results, service delivery, and customer satisfaction.


prevention activities; student behavioral, social, and emotional supports; mental health

services; and early childhood social and emotional learning programs.

Because of variations in reporting, the GPRA measures were defined to ensure

standardized data collection and reporting. The current GPRA measures are: decrease

in students who did not go to school on 1 or more days during the past 30 days because

they felt unsafe at school or on their way to and from school; decrease in students who

have been in a physical fight on school property in the 12 months prior to the survey;

decrease in students who report current (30-day) marijuana use; decrease in students who

report current (30-day) alcohol use; increase in the number of students receiving school-

based mental health services; and increase in the percentage of mental health referrals for

students that result in mental health services being provided in the community.

Grantee awards increased from 3 to 4 years, providing local project staff and partners

more time to implement programs and services and achieve their stated goals and

objectives.

For all grantees under the current evaluation (FY 2005 to 2009), there is a requirement to

allocate at least 7 percent of their annual grant budget to a local evaluation. Grantees are also

required to participate in the cross-site national evaluation. The local evaluation measures the

performance of individual grantees, while the national evaluation measures the performance of

the overall SS/HS Initiative.

NATIONAL CROSS-SITE EVALUATION

CMHS funded the current 5-year, cross-site national evaluation of the SS/HS Initiative,5 which

encompasses 175 grantees (see Appendix A) in five successive cohorts that received grants

beginning in 2005. The evaluation design integrated quantitative and qualitative data drawn from

the following sources:

1. Reviews of the grant applications and performance reports

2. Public information such as census data

3. Site visits in Year 1 of the grant

4. Online annual surveys of project directors and school staff

5. Telephone interviews with project directors (annually) and partners (Years 2–3)

6. Collection of annual outcome data as required by GPRA

7. Focus groups with project directors and local partners

5 The members of the National Evaluation Team (NET) are MANILA Consulting Group, RMC Research

Corporation, and Battelle Centers for Public Health Research and Evaluation.


Appendix C describes the data sources in more detail, and Appendix J describes the evaluation’s

methodology.

PROGRAM THEORY

The SS/HS national evaluation seeks in part to test assumptions about the relationships among

systems change, collaboration, and individual outcomes. The assumptions are that the grant can

foster change in community systems that serve children and youth through improved

collaboration among schools and local agencies, and thereby improve outcomes related to

violence and school safety, drug and alcohol use, and other areas of school climate and student

well-being.

The SS/HS Initiative is unique in the nature of this intended change process. Namely, the grant

requirements are meant to counter the effects of funding and organizational ―silos,‖ the separate

and parallel systems that divide service providers and hinder collaboration. Like many school-

based initiatives and grant programs, the SS/HS Initiative focuses on changing the service

delivery system for school-aged youth. Like many public sector and nonprofit initiatives, it aims

to encourage use of best practices and EBPs. But by requiring multiple providers at each site to

link services and collaborate to improve the systems that serve school-aged youth, Congress and

the Federal partners anticipate that SS/HS grantees are more likely to improve school climate and

a variety of individual-level outcomes among their targeted populations.

In keeping with this logic, the National Evaluation of the SS/HS Initiative seeks to address four

overarching questions:

1. How do existing conditions and resources in the pre-grant environment affect the

relationships among grant operations and outcomes?

2. Are SS/HS grant operation characteristics associated with near-term and long-term

outcomes?

3. What near-term outcomes are associated with improvements in long-term outcomes?

4. Overall, does the SS/HS Initiative meet the Federal Government’s expectations of

achieving improvements in long-term outcomes (reduction in substance use and violence,

increased access to mental health services, and improvement in attendance and school

climate) and near-term outcomes (comprehensive programs and activities and improved

coordination and service integration)?

The NET has developed a program theory model (shown in Exhibit 2) reflecting this intended

change process and building on the overarching questions described above.


Exhibit 2. SS/HS National Evaluation Program Theory Model


The SS/HS National Evaluation program theory model identifies the environmental conditions

and resources at the time of grant award and shows how the activities associated with the

Initiative transform these existing conditions into desired near-term and long-term outcomes. The

model also looks at factors that may affect both grantees’ ability to break out of any existing

organizational silos to better serve youth, and the ability of the resulting collaborative efforts to

generate measurable improvements.

OVERVIEW OF THIS REPORT

This report presents findings from 175 SS/HS grantees that received awards in 2005 (40

grantees), 2006 (19 grantees), 2007 (27 grantees), 2008 (60 grantees), and 2009 (29 grantees)

based on data collected from the time of grant award through July 2010. As noted above, the

2005 and 2006 cohorts are the first under the current national evaluation for which data

collection has been completed, and therefore findings on near- and long-term outcomes are

limited to these cohorts.6

The report describes the grant’s impact on students, schools, and communities; characteristics of

the grantees and their local communities and partnerships; and the activities they implemented as

part of the grant (described as near-term outcomes). The report reveals what the grantees

achieved, who they were, and what they did with the grant funding. Taken together, these

findings contribute to our understanding of the overall impact of the SS/HS Initiative and provide

information for future decisionmaking.

6 There was no SS/HS Initiative announcement in 2006. Based on the availability of additional funds from the FY

2005 competition, the Federal SS/HS partners made 19 awards in FY 2006 from the rank-ordered list of more than

400 unfunded applications received during FY 2005.


IMPACT OF THE INITIATIVE ON

STUDENTS AND SCHOOLS

he SS/HS Initiative is designed to produce long-term improvements in the health and

safety of young Americans by changing local systems that serve children and youth.

Findings from the 2005 and 2006 grantees suggest this approach works. On the whole, in

schools that received SS/HS funds, violence and unhealthy behaviors such as underage alcohol

use decreased, early detection of mental health problems increased, and access to mental health

services increased. The grantees exceeded all SAMHSA GPRA targets for 2009. Many have also

taken steps toward ensuring the sustainability of the SS/HS partnership, activities, and

infrastructure through other funding sources.

SAFER STUDENTS, SCHOOLS, AND COMMUNITIES

Violent incidents decreased 11 percent.

Fewer students reported they had experienced violence (7 percent decrease) or witnessed

violence (4 percent decrease).

Ninety-six percent of school staff surveyed said SS/HS had improved school safety.

More than 90 percent of school staff surveyed said SS/HS had reduced violence on

campus.

Almost 80 percent of school staff surveyed said SS/HS had reduced violence in the

community.

HEALTHIER STUDENTS

The number of students receiving school-based mental health services increased 263

percent.

The number of students receiving community-based mental health services increased 519

percent.

Almost 90 percent of school staff surveyed reported improved detection of mental health

problems.

More than 80 percent of school staff surveyed said they saw reductions in alcohol and

other drug use.

Almost 70 percent of school staff surveyed said early childhood development had

improved.

T


GOVERNMENT PERFORMANCE AND RESULTS ACT DATA

Grantees were required to report annually on four GPRA indicators. The evaluation

recommended 12 specific measures to report on these indicators (3 violence, 6 substance use, 1

school attendance, and 2 mental health services measures) to facilitate comparison across sites.

Grantees used a range of data sources to meet their GPRA reporting obligations— including

surveys, incident reports, and service delivery logs—and reported their data in highly variable

ways (e.g., by school, school type, grade level, district). Meta-analytic techniques permitted the

data to be summarized in a common index. Appendix G presents a summary of GPRA data for

the 2005 and 2006 cohorts combined from Year 1 to Year 3.

The SS/HS Initiative exceeded all SAMHSA GPRA targets in FY 2009, the most recent year for

which data were available (see Exhibit 3).

Exhibit 3. GPRA Results for Fiscal Year 2009

MEASURE FY 2009 TARGET FY 2009 RESULT

Ou

tco

me

Increase the number of children served 2,328,500 3,154,305

Target Exceeded

Decrease the percentage of middle school students

who have been in a physical fight on school property 30%

23.8%

Target Exceeded

Decrease the percentage of high school students

who have been in a physical fight on school property 24%

16.1%

Target Exceeded

Decrease the percentage of middle school students

who report current substance use 16%

13.3%

Target Exceeded

Decrease the percentage of high school students

who report current substance use 35%

31.1%

Target Exceeded

Increase the percentage of students attending

school 93%

94.5%

Target Exceeded

Increase the percentage of students who receive

mental health services 66%

74.4%

Target Exceeded

Ou

tpu

t

Percentage of grantees that provided screening

and/or assessments coordinated among two or more

agencies or shared across agencies.

69% 73.9%

Target Exceeded

Percentage of grantees providing training of school

personnel on mental health topics 69%

73.9%

Target Exceeded

SUSTAINABILITY

A key goal of the SS/HS grant is to sustain the programs, activities, partnerships, and

infrastructure developed or expanded with the grant. Research supports the idea that

collaboration by community coalitions may lead to increased support and integration of services


in schools (Domitrovich & Greenberg, 2000). The support of influential community and school

leaders; competent program leadership, including knowledge of the community; and evidence of

program effectiveness are potential predictors of sustainability (Johnson, Hays, Center, & Daley,

2004; Marek, Mancini, Earthman, & Brock, 2002; Grube, 2005). Research also stresses the

importance of identifying long-range sources of financing, particularly from local funding

sources (Shediac-Rizkallah & Bone, 1998). In the school environment, both financial

sustainability planning and alignment of interventions with goals of the school is critical (Tibbits,

Bumbarger, Kyler, & Perkins, 2010). By the end of the third year of the SS/HS grant, all the

grantees developed a plan for sustainability, and more than half secured additional funding for

programs and services.

Grantees from the 2005 and 2006 cohorts, the only cohorts with complete data under the current

national evaluation, sustained a wide range of programs, activities, services, infrastructure, and

system changes across the different grant topical areas. However a comprehensive review of

activities planned, implemented, and sustained suggests that the types of programs and activities

sustained differed across program elements. Grantees were most likely to sustain programs and

activities targeting safety and violence and least likely to sustain activities targeting early

childhood and substance use prevention.

Types of sustained programs, activities, and staff included: EBPs; after-school activities;

procedures and systems for collecting and sharing data and referring students for services;

security policies and procedures such as annual school security assessments and maintenance of

security cameras at schools; and positions such as learning support specialists, school resource

officers, early childhood behavioral assistants and liaisons, and school-based mental health

counselors. Programs closely aligned with school goals and integrated into school operations

were most likely to be sustained, as were those building on existing programs or not requiring

new staff positions. In general, programs that existed prior to SS/HS grant funds enhanced with

SS/HS funds were easier to sustain than new programs initiated with the grant funds. However,

programs requiring new or continued staffing were less easy to sustain.

Grantees across cohorts reported a number of strategies for planning for sustainability, including

early and consistent sustainability planning, strong leadership, effective use of data and

communications, integration of the SS/HS Initiative into the schools and community, and pursuit

of alternate funding.

Project directors from the 2005 cohort who participated in a focus group to discuss sustainability

reported that it was vital to think about and plan for sustainability from the very beginning of the

grant. This finding is consistent with research literature indicating that planning for sustainability

is a predictor of achieving sustainability (Shediac-Rizkallah, et al., 1998; Sridharan, Sodam,

Zinzo, Gray, & Barrett, 2007; Tibbits et al., 2010). Some sites used formal or written

sustainability plans to prepare for sustainability; having a formal document with target dates and

goals helped keep sustainability planning on track.


Many grantees created subcommittees to help address sustainability. Ideas formulated in

subcommittees were brought back to the main partnership where further coordination and

decisionmaking took place. As one project director noted,

The SS/HS grant helped to establish a core group of people working together to make

decisions. Any time a new program or activity is considered, the team discusses whether

they can move forward with it on the systems level. Any time there was a major grant

application or collaborative funding opportunity, it became the norm in this community

that it would start with this partner team.

Grantees also benefited from strong communication, including communication among partners,

as well as strategic communications such as newsletters and presentations to potential funders,

schools, and parents. Grantees reached out to the community through local media such as

newspapers and radio stations. One site reported that changing the mindset that SS/HS was not

just a grant, but an initiative, helped to avoid the connotation that partner efforts and project

programming were temporary. Communication among partners, and with the broader

community, facilitated buy-in. Grantees reported that obtaining buy-in from school

administrators, especially school superintendents, was of particular importance for sustainability.

The importance of obtaining the support of influential community and school leaders is also

supported by the research literature (Johnson et al., 2004; Shediac-Rizkallah & Bone, 1998;

Tibbits et al., 2010).

Demonstrating evidence of program success is important to achieving sustainability (Marek et

al., 2002; Grube, 2005) and grantees reported that using data to support programming was a

common practice. Grantees used data on youth outcomes and cost savings associated with

effective SS/HS programs to inform social marketing efforts directed toward stakeholders and

potential funders. Focus group participants noted additional tactics, including incremental

reductions in funding allocations to prepare partners for diminished funds and holding grant

writing workshops to prepare partners to seek funding.

Grantees that made the least progress toward sustainability tended to rely on funding from the

school district. Alternatively, grantees that showed the most progress toward sustainability by the

end of the grant reported using diverse, creative strategies, such as Web-based training and

applying for nonprofit status to become eligible for specific types of funding. Successful grantees

devised a patchwork of funding streams to sustain their SS/HS projects and infrastructure beyond

the grant period. Sources of funding included school districts; State, county, and city

governments; Medicaid; and grants from Federal agencies and nonprofit organizations. At one

site a partnership became a nonprofit organization in order to qualify for certain funding.

Another site successfully advocated for a tax to fund the project after the grant end.

One of the plans for continuation of mental health services to the students at the schools

is through Medicaid funding. At this time each of the schools are applying for licenses as


satellite centers of the local Human Service District in order to qualify for Medicaid

reimbursement.

Many grantees encountered barriers to sustainability. Several reported submitting unsuccessful

grant applications, or expressed concerns about obtaining additional funding due to current local

or national economic conditions. Grantees across cohorts reported common barriers to

sustainability, including lack of participation from partners, turnover in leadership positions, and

uncertainty about how to continue funding programs.

The school district has had some difficulties because of changes with the superintendent,

assistant superintendent, and some new building principals; there have also been some

internal changes within the district and work has to be done to get to know each other

again.

Potentially endangered, they have written for grants to fund social workers in the

elementary and middle schools but are not sure the project director has identified high

school social work services as being how to fund social workers in the high school.

The economic recession and budget cuts were the most common and the most significant barriers

to sustainability. Some grantees reported that mental health programs were affected significantly

by the economic downturn because such programs require substantial funding and staff. Budget

cuts dismantled some mental health programs and social work positions were eliminated, even in

communities that valued those services. Typically, academic programming received funding

priority above mental health programming despite research indicating that mental health is

central to a students’ academic performance (Fleming, Haggerty, Catalano, Harachi, Mazza, &

Gruman, 2005). Even when mental health programs were sustained, there were sometimes gaps

in services for underinsured or uninsured students, or the programs were provided only at some

of the schools.

The most significant barrier to collaboration experienced by the SS/HS partnership was

the state budget crisis that significantly impacted the operating budgets of all public

agencies that are part of the partnership. Partners had to reduce staff in their agencies

which impacted the partners’ ability to attend SS/HS meetings. The PD reported that the

budget crisis also impacted the partners’ ability to commit to sustaining staff and/or

programs after the SS/HS grant ends.

Despite challenges, grantees identified a number of facilitators to sustainability. Some grantees

noted that having a strong project director facilitated sustainability. Grantees also indicated that

having knowledge of the local community, personal relationships with key decisionmakers, a

record of local achievement, and prior experience working within the school district are

important factors in achieving sustainability. In addition, having an effective local evaluator can

help to assess which programs are successful and determine which programs to sustain. A strong

local evaluator can also present data in an understandable way to stakeholders and potential

funders.


Strong partnerships at both the site and Federal levels were also important to achieving

sustainability. Having consistent meetings, where everyone kept each other accountable and

talked about sustainability, was important. Grantees found that celebrating project successes

facilitated partner buy-in. Grantees also reported that having effective Federal project officers,

and taking advantage of technical assistance, site visits, and SS/HS sponsored conferences, were

beneficial for sustainability.

Backing from the superintendent of schools may be an important factor in sustainability.

Superintendents can increase the SS/HS project’s visibility and harness key institutional and

political support throughout the school district and the greater community. Project directors in

the 2005 and 2006 cohorts were questioned in their no-cost extension year about the role of the

superintendent in their SS/HS project. (A no-cost extension year is an additional year of funding

provided to grantees after the main grant period, consisting of left-over money not spent during

the grant period.) The results showed that most project directors rated their superintendents as

providing high levels of support (59 percent), citing examples such as attending pushing for

funding to sustain SS/HS services, making budgetary changes to sustain funding, or giving the

project director an administrative role that provided credibility in the district. Only 5 percent of

project directors reported low levels of support.

SCHOOL CLIMATE

Data on school climate were not collected from the 2005 cohort and only began with the 2006

cohort in their third year of the grant due to delays in obtaining approval from the Office of

Management and Budget (OMB). Therefore, data on long-term outcomes on school climate are

not presented in this section. However, a detailed set of preliminary findings by school type and

by cohort is presented in Appendix F, along with additional details on the survey and its use by

grantees.


CHARACTERISTICS OF SAFE

SCHOOLS/HEALTHY STUDENTS

GRANTEES AND PROJECTS

he 175 grantees funded by the SS/HS Initiative in 2005 through 2009 are located in

urban, rural, suburban, and tribal areas in 41 States and the District of Columbia. They

serve more than 2.5 million students in 4,161 schools in 365 school districts. The

grantees received grant funding ranging from $372,520 to $2,902,497. The number of students

targeted by each grant, as measured by enrollment in schools with SS/HS programs and services,

ranged from 200 to 342,395, with a median of 8,763 students. Exhibit 4 on the following page

highlights the range of characteristics of the grantee communities, students, and schools served.

Additional information collected through the national evaluation included challenges to grant

implementation, the involvement of schools, and the importance of grant resources in the

schools. The grantees also reported on the implementation of comprehensive, coordinated, and

integrated programs and activities and enhanced services to provide an understanding of the

changes grantees made in their schools and communities using grant funds.

CHARACTERISTICS OF THE PARTNERSHIPS

Partnership History in the Community

The SS/HS Initiative requires the LEA to partner with the local mental health, law enforcement,

and juvenile justice agencies to develop and implement a comprehensive plan. The school

district(s) must have had a relationship with at least two of those three agencies for at least 6

months preceding the grant application, with a record of previous accomplishments. In fact, the

partnerships exhibited wide variation in both duration and level of agency involvement prior to

the grant award; some LEAs were just beginning to work with one of the required partners, while

others had worked with all three partners for an extended period of time. Exhibit 5 shows the

distribution of 2005 to 2009 grantees with respect to partnership history. More than 41 percent of

the grantees had longstanding, mature partnerships in place before they were awarded the grant,

while a smaller number (just under 10 percent) represented newer partnerships.

T


Exhibit 4. Grantee Characteristics

SS/HS COMMUNITY CHARACTERISTICSa

Percent of households (with children under the age of 18) below the poverty level

Mean 18.9%

Median 16.8%

Range 3.2% to 69.7%

Percent of population over age 25 with a high school degree or higher

Mean 78.9%

Median 81.3%

Range 29.7% to 96.6%

Percent of population over age 16 that is unemployed

Mean 5.7%

Median 4.5%

Range 1.3% to 39.8%

STUDENT CHARACTERISTICSb

Gender

Male 51.2%

Female 48.8%

Race and Ethnicity

African American 24.0%

Asian 4.4%

Hispanic 39.6%

Native American 0.8%

White 31.2%

SCHOOL CHARACTERISTICSc

Number of Targeted Schools

Total 4,161

2005 cohort 1,157

2006 cohort 321

2007 cohort 869

2008 cohort 1,327

2009 cohort 487

School Type *

Elementary 53.9%

Middle 18.1%

High 16.0%

Elementary/middle 5.2%

Middle/high 3.2%

K to 12 1.6%

Pre-K only 1.9% a Source: American Community Survey, U.S. Census Bureau.

b Source: National Center for Education Statistics. District-level data on race and ethnicity were limited to the

categories shown. c Source: Grant application and School-Level Survey (SLS).


Exhibit 5. Partnership History Prior to Grant

PARTNERSHIP STRUCTURE AND ACTIVITIES

The SS/HS Initiative encourages collaboration among agencies and organizations whose

programs and services can affect outcomes among children and youth. Schools and partner

agencies are expected to work together to coordinate services, and it is expected that the resulting

collaboration will fundamentally change how the organizations operate. Partnerships have also

been shown to improve delivery of needed school and community services (Florin, Mitchell,

Stevenson, & Klein, 2000). When carried out in combination with effective leadership, grant

activities are expected to result in improvements in individual student outcomes and school

climate and contribute to sustainability (Putnam, 2000; Wagenaar, Erickson, Harwood, &

O’Malley, 2006).

For partnership composition, research suggests that diversified membership in coalitions and

partnerships predicts a successful community collaboration (Cranwell, Kolodinsky, Anderson, &

Schmidt, 2004). As noted above, grantees were required to partner with local law enforcement,

mental health, and juvenile justice agencies to implement the grant. SS/HS grantees expanded

their partnerships beyond the required agencies to include faith-based groups, civic groups, local

businesses, chambers of commerce and other business groups, and other existing coalitions. The


partnerships also sought out organizations that worked with young people, were influential in the

community, or could help expand services. Most of the grantees built upon existing partnerships

and used the grant as a stepping stone to broader collaboration, bringing in community-based

organizations such as the following:

Boys and Girls Club

Big Brothers Big Sisters

YMCA

Emergency management services

Hospitals

Parks and recreation

Universities

Faith-based organizations

Substance use prevention

organizations

Chambers of commerce

The evaluation also assessed changes in the organization of the SS/HS partnerships, given the

central role of partnerships in the grant as well as guidance from the literature. For example,

Jasuja, Chou, Bernstein, Wang, McClure, & Pentz (2005) examined the structure of coalitions

and found that the presence of a steering board or committee and working subcommittees

predicted progress in adopting EBPs. Analyses by Kegler, Steckler, Mcleroy, & Malek (1998)

further supported the importance of the complexity of coalition structure (operationalized by the

number of functioning committees and task forces), which they found was associated with the

number of activities completed during the first year of implementation. The evaluation defined

SS/HS partnerships as having one, two, or all of the following three structures:

Single group (the entire SS/HS partnership)

Executive/management team (a group of individuals that serves in an executive or

steering committee function)

Committees/subcommittees assisting in implementation at schools; focusing on specific

content areas such as gang intervention, early childhood, or partnership operations and

bylaws; or serving in an advisory capacity, such as an advisory council board, student

advisory board, or existing community coalition

Most 2005–2009 SS/HS sites (59 percent) began as single-group partnerships, although more

than half (52 percent) of 2005–2008 sites changed their organizational configurations over time

(see Exhibit 6).7 Of those that did, most (61 percent) expanded their partnership between Years 2

and 3 of the grant, typically by adding committees. Most changes made by the 2006 and 2007

cohorts occurred during the first year of funding, which may have resulted from receipt of

technical assistance focused on partnership sustainability and the legacy wheel (a training tool to

help grantees plan for sustainability provided by the National Center for Mental Health

Promotion and Youth Violence Prevention). Anecdotal information collected through interviews

7 The 2005, 2006, and 2007, cohorts had 3 years of data, while the 2008 cohort had 2 years of data because of their

stage in the grant cycle.


with project directors and partners suggests that Federal project officers and technical assistance

specialists suggested expanding partnerships in an attempt to better engage high-level partner

agency decisionmakers and to increase partner participation in grant decisionmaking. Although

explicit data on the reasons for changing the organizational structure of partnerships are not

collected, some interview responses provided additional insight into the rationale underlying

expanding partnerships (e.g., adding a committee to focus on a particular content area such as

mental health, early childhood, or partnership operations).

Exhibit 6. Project Director Ratings of Partner Contributions

PARTNER TYPE TYPICAL RATING

School districts

Leading contributions to planning and implementation Significant to leading contributions to monitoring and tracking, and

policy change Significant contributions to sustainability

Juvenile Justice Moderate contributions to planning, implementation, monitoring and

tracking, policy change, and sustainability

Mental Health

Significant contributions to planning, implementation, and monitoring and tracking

Moderate to significant contributions to policy change and sustainability

Law Enforcement

Moderate contributions to planning Moderate to significant contributions to implementation Moderate contributions to monitoring and tracking, policy change,

and sustainability

Other Nonrequired Partners

Moderate to significant contributions to planning Significant contributions to implementation and monitoring and

tracking Moderate contributions to policy change Moderate to significant contributions to sustainability

Typically, for each cohort, project director ratings of partner contributions remained the same

across all years of grant operation. Partners made varying levels of contributions to five core

grant activities (planning, implementation, monitoring and tracking, formulating policy change,

and sustainability planning) over the course of the grant cycle as evidenced by project director

ratings. For example, in Year 1 of the grant, 2005, 2006, and 2007 cohort partners made the

strongest contributions to planning and implementation activities. In subsequent grant years,

contributions to implementation remained high, while contributions to planning tapered slightly.

In all three cohorts, partnership contributions to sustainability planning increased significantly

over time. The 2006 and 2007 cohorts reported significant increases in contributions to

monitoring activities; the 2005 cohort reported significant increases in policy change activities.

Exhibit 7 summarizes project director ratings of partner contributions to the five core grant

activities throughout the project.


Exhibit 7. Partnership Organization by Grant Year

PARTNERSHIP ORGANIZATION

Year 1 N=175

Single group 38%

Single group and executive team 22%

Single group and committees 17%

Single group, executive team, and committees 23%

Year 2 N=146

Single group 24%




Year 3 N=86

Single group 21%




* Note: Year 1 data include the 2005–2009 cohorts; Year 2 data include the 2005–2008 cohorts; and Year 3 data

include the 2005–2007 cohorts.

Perceptions of the local SS/HS partnership by participating organizations varied significantly by

grantee site. Sites with the most favorable perceptions of the partnership by partners reported few

barriers to implementation and collaboration and were able to obtain school and partner buy-in

for the grant. These sites were also able to articulate strategies for addressing barriers.

Conversely, sites with the least favorable perceptions of the partnership reported many

significant barriers and appeared to be overwhelmed by them. Sites with the most favorable

perceptions of the partnership were more likely to identify key facilitators to program

implementation and partner collaboration, whereas sites with the least favorable perceptions of

the partnership reported few to no facilitators. Sites with the most favorable perceptions of the

partnerships were more likely than sites with the least favorable perceptions to report factors

associated with effective partnerships such as participatory, consensus-based, decisionmaking

processes, and to include an advisory board in their partnership organization. Regarding

sustainability, sites with the most favorable perceptions were able to articulate concrete plans for

sustaining programs and were more likely than sites with the least favorable perceptions to have

concrete plans for continuing the partnership after the grant ends. Exhibit 8 shows the patterns

that emerged.


Exhibit 8. Factors Associated With Partnership Ratings

THEME SITES WITH MOST FAVORABLE

PERCEPTIONS OF PARTNERSHIP

SITES WITH LEAST FAVORABLE

PERCEPTIONS OF PARTNERSHIP

School and

Community Buy-In

Reported few barriers in relation to obtaining school buy-in

School-level administrators were included in the pre-grant planning

Reported multiple barriers in relation to obtaining school and/or community buy-in

Little evidence of school-level administrators included in the pre-grant planning

Barriers Reported few barriers Articulated proactive steps to

address identified barriers

Reported multiple unresolved barriers

Did not articulate strategies to address barriers

Partner Organization

Articulated a clearly organized partnership structure with clearly defined roles

Reported a shared vision of the partnership

Described a partnership that is loosely defined, not highly structured or organized

Rarely described a shared vision or goals for working together

Shared

Decisionmaking

Described a participatory, consensus-based decisionmaking process

Little evidence of shared decisionmaking

Data indicated low partner engagement, lack of partner buy-in, and difficulty collaborating with key partners

Sustainability

Articulated a commitment to sustain programs in Year 1

Articulated concrete plans for sustaining programming by Year 2

Articulated desire to sustain programs in Year 1 and 2

Did not describe concrete plans for achieving sustainability

CHARACTERISTICS OF THE OPERATING ENVIRONMENT

School-Level Involvement and Priorities

Individual schools reported varying levels of involvement in decisionmaking regarding planning

and implementing the grant. This variation may be related to organizational structure within the

school district(s) or district size. For example, a site with only a few small schools may be able to

actively engage many staff in grant decisions, unlike sites with more than 100 schools.

School staff were often involved in decisions related to:

Selecting programs and/or curricula to be implemented at the school

Program implementation

Ongoing, informal program monitoring and feedback


SS/HS-related protocols, such as crisis response plans

Long-term plans for enhancement or continuation of SS/HS programs at the school

Procedures/operations (e.g., steps to refer students to outside help)

In contrast, school staff were relatively less likely to be involved in decisions regarding funding

and/or resource allocations, regularly scheduled evaluation and data collection, and technical

assistance and training compared to other areas.

School involvement in the grant was not limited to decisionmaking regarding the planning and

implementation of grant activities. Schools are a key stakeholder in the Initiative, and therefore

their buy-in is critical to successful implementation of new programs and activities. Several

factors were identified as key in soliciting high levels of buy-in for SS/HS programs and

activities.

Administration. Grantees commonly reported that when the upper administration of the LEA

supported the grant, buy-in came quickly and more easily than when high-level administrators

were unsupportive or uninvolved. Several grantees noted that without the endorsement of

superintendents, school staff would have been more suspicious of the grant and less likely to

accept grant services. Many grantees made the following kinds of statements: ―Having the

district superintendent and the director of pupil services in full support of the program was the

biggest facilitator to school buy-in,‖ and ―The principal helped to boost staff commitment to

implementing the programs.‖ Several grantees reported that the superintendent’s efforts to

include principals in meetings with project partners to discuss issues such as truancy and drop-

out rates assisted with achieving school buy-in at the building level.

Recognition of need. Grantees commonly reported that assessing student and school needs and

sharing findings with school administrators and staff facilitated school buy-in. While some

grantees cited economic decline and school budget cuts as significant challenges, other grantees

reported that those concerns helped facilitate school buy-in because staff recognized the

importance of using grant funds to address declining resources. Some grantees reported that

school buy-in was facilitated by helping principals see how the activities associated with the

SS/HS grant can help schools meet their State-mandated school improvement goals.

Demonstrated success. Grantees noted that sharing local evaluation data on the success of grant-

related efforts with school administrators and staff was a key way to obtain school buy-in. For

example, some grantees noted:

The positive changes in the students’ growth and development, their positive academic

results, and increased feeling among students that there are options beyond high school

has facilitated buy-in.


The fact that the project and the partners have been able to accomplish some things in

the schools has been a facilitator.

Once the initiative got [implemented] at a few schools, and was doing good work…

principals began spreading buy-in by word of mouth.

Communication. Nearly all the grantees noted that communication between project staff, partner

agency staff, and school administrators and staff was a key facilitator of school buy-in. To

sustain school buy-in, grantees discovered the importance of communicating on a regular basis

about student needs, partner agency roles and responsibilities, problem-solving in the face of

challenges, SS/HS project status, and evaluation findings. For example, one mental health

partner noted that taking time to meet with each administrator to discuss the function of the

mental health agency and how services can benefit the students greatly facilitated buy-in across

the district. Many grantees identified a project coordinator in each school to improve

communication between SS/HS staff and school staff. Several grantees reported that housing the

mental health partner in the same building with the school district facilitated program

implementation, because it allowed daily, in-person communication between the mental health

partner and school staff.

School staff generally reported that the SS/HS grant helped to improve relations between district-

level and school-level staff, promote cohesiveness and respect, and facilitate combined resources

from a wide range of professionals in the school.

School staff were asked to indicate which SS/HS programs they felt were high priorities in their

schools.8 Exhibit 9 shows their responses for each type of activity in the final year of the grant.

8 Year 3 School Level Survey data are from the 2005 to 2006 cohorts.


Exhibit 9. School-Level High-Priority Activities

ACTIVITY PERCENT

Safe

ty a

nd

Vio

len

ce

Pre

ven

tio

n

Setting behavioral goals for preventing violence for students 82%

Communicating school's expectations on safe school environment 81%

Providing curricula or programs to students involving knowledge, skills, attitudes, or

values intended to prevent violence 79%

Communicating school’s expectations on violence issues 77%

Providing feedback or consequences to affect behavior related to prevention of

violence 76%

Setting behavior goals for preventing violence for students 74%

Tracking students’ behavior in achieving the goals on violence prevention 70%

Providing training to school staff on safe school environment issues 70%

Providing training to school staff on violence prevention issues 61%

Developing crisis preparedness plans, tools, or guidelines 71%

Tracking students' behavior in achieving the goals on promoting safe school

environment 68%

Providing outreach or service to families to improve their child management and

supervision practices 68%

Promoting student participation in resolving disputes or responding to problem

behavior 68%

Tracking students' behavior in achieving the goals on violence prevention 67%

Providing activities to establish and enforce classroom rules 66%

Conducting security assessment/search for security gaps 65%

Installing or replace safety equipment 61%

Providing training to school staff on violence prevention issues 61%

Su

bsta

nce

Use

Pre

ven

tio

n

Providing curricula or programs to students involving knowledge, skills, attitudes, or

values related to preventing alcohol or drug use 67%

Providing feedback or consequences to reinforce prevention of the use of alcohol and

other drugs 56%

Setting behavior goals for creating and maintaining alcohol- and drug-free students 54%

Tracking students’ behavior in achieving the goals on alcohol and other drug use 46%

Using outside resources in classrooms, such as parents or community volunteers, to

convey information on alcohol and other drug use 43%

Providing families with information on alcohol and other drug use prevention activities 49%

Providing training to school staff on alcohol and other drug issues 42%


ACTIVITY PERCENT M

en

tal H

ealt

h A

cce

ss

Providing referral and followup with local public mental health agencies when needed

services are not available at school 79%

Providing school-based mental health prevention and early intervention services for at-

risk students and their families 78%

Providing supportive services to families so that they may participate fully in the

educational, social, and healthy development of their children 77%

Supporting enhanced integration, coordination, and resource sharing among mental

health and social service providers in schools and other community-based programs 65%

Developing a formal arrangement between schools and public or nonprofit mental

health entities concerning the delivery of mental health services for children and

adolescents with more serious mental health problems

64%

Changing students' attitudes and beliefs on mental health 62%

Providing mental health screening and assessment in the school setting 61%

Providing training and consultation to school personnel to improve their knowledge and

skills on mental health promotion 54%

Earl

y C

hild

ho

od

D

evelo

pm

en

t

Overcoming barriers to identifying and serving families in need 52%

Providing mental health services and parenting programs for parents 47%

Training teachers or caregivers to work collaboratively with families to address the

social and emotional needs of children 43%

Providing activities or opportunities to preschool children to be ready for school 38%

Providing family support services to families with preschool children 36%

Providing case consultation, crisis intervention, or therapies to parents of preschool

children 33%

IMPORTANCE OF GRANT RESOURCES IN SCHOOLS

In each year of the grant, taking into account financial support and other kinds of resources, most

school staff surveyed thought the resources provided through the grant were very important in

improving safety and preventing problem behavior (see Exhibit 10).


Exhibit 10. School Staff Ratings of the Importance of SS/HS Resources in Relation to

Other Sources of Support in Schools*

*SLS data are from all available cohorts and years

IMPLEMENTING COMPREHENSIVE PROGRAMS AND ACTIVITIES

All 2005 and 2006 grantees documented the implementation of a comprehensive plan to address

the grant elements, as required. In general, the number of activities implemented increased with

each successive year of the grant, from an average of 14.3 activities per grantee in Year 1, to

22.0 in Year 2, to 24.1 in Year 3 (see Exhibit 11). Exhibit 12 identifies the kinds of activities

implemented.


Exhibit 11. Number of Activities Implemented by Grant Year

Exhibit 12. Types and Frequency of Activities Implemented

ACTIVITY

PERCENTAGE OF

GRANTEES THAT

IMPLEMENTED THE

ACTIVITY

Safe School Environment and Policies

Child and family support services 99%

Student conduct 98%

Crisis preparedness plan 95%

Community involvement 95%

Physical security of grounds and facilities 93%

School safety and security incident reporting 92%

Setting standards for student behavior 87%

Parental involvement 86%

Student discipline 85%

Reintegration of students from the juvenile justice system 68%

Substance Use and Violence Prevention and Early Intervention

Substance use and violence prevention services 99%

Parental and community involvement 96%

Social and recreational activities 88%

Mentoring 80%


ACTIVITY

PERCENTAGE OF

GRANTEES THAT

IMPLEMENTED THE

ACTIVITY

Mental Health Prevention and Intervention

Screening and assessment 100%

School-based mental health services 100%


Referral and followup in school 100%

Referral and followup outside of school 97%

Training school staff 87%

Early Childhood Development

Parenting programs and services 100%


School readiness 96%


Health services for mothers 64%

Supporting and Connecting Schools and Communities

Sharing resources with other agencies 95%

Afterschool programs 86%


Alternative education placement and programs 72%

Grantees generally implemented a broad range of activities and programs under the grant. In

some instances, they implemented comprehensive approaches to address specific needs. The

following sections summarize information on the programs and activities of grantees in the 2005

to 2007 cohorts reported during site visits and in project director and partner interviews.

Programs and Activities to Address Safety and Violence Prevention

Achieving safe learning environments and promoting a positive school climate are at the

forefront of the SS/HS Initiative. Grantees were tasked with first identifying needs and gaps for

safety and violence in their schools and communities, and then with proposing and implementing

various strategies to address these needs. Early in the grant period, many grantees identified

bullying at schools, on school buses, and in the community as major problems. Grantees also

identified the need to improve the physical safety of schools; to reduce physical fighting; and to

address gang issues, including recruitment, youth interaction with gang members, and witnessing

gang activity. Although each grantee faced unique challenges, sites used common strategies to

address these problem areas and to promote safe environments where teachers were able to teach

and students were able to learn.


Examples of commonly used strategies included:

Environmental modification for the safety of schools, which included installing

surveillance cameras in schools and on school buses and instituting ID badge systems

Rewriting and revising old school policies and codes of conduct or creating new ones

Developing districtwide emergency preparedness and safety plans

Safety assessments of schools and safety assessment trainings

Placement of School Resource Officers (SROs) in schools

Placement of Probation Officers (POs) in schools

Conferences, trainings, and workshops for school staff, students, and parents on topics

such as emergency preparedness, bullying, safety plans, and lock-downs

Implementation of EBPs and practices to address violent and aggressive behaviors, to

address gang-related issues, to screen youth at risk, and to promote a positive school

climate

Mentoring programs in schools

Dissemination of safety information to parents through forums, conferences, and DVDs

Safety and Security. Grantees focused on improving the physical security of schools and school

buses. They report installing security cameras and ID badge systems in addition to creating or

modifying existing emergency response plans and lock-down procedures. The types of activities

reported among SS/HS grantees are consistent with recent trends. Between the 1999–2000 and

2007–2008 school years, there was an increase in the percentage of public schools reporting the

use of security cameras to monitor school activities (from 19 percent to 55 percent, Dinkes et al.,

2009). Further, schools’ use of student and faculty ID badge systems also increased over the time

period (from 4 percent to 8 percent and from 25 percent to 58 percent, respectively, Dinkes et al.,

2009). The presence of SROs in schools also helped address the physical safety of schools. The

SROs developed relationships with the students and were frequently perceived as friends, a

paradigm shift from students’ more negative perception of police officers in the past.

Bullying. Sites also increased awareness regarding bullying. School staff were trained to

recognize and deal with bullying when it occurs, and students were taught how to recognize,

refuse, and report bullying. Antibullying and antiharassment policies were frequently instituted

by grantees and in some instances, the State requirements led grantees to create such policies in

schools. Dissemination campaigns and presentations to parents and local communities improved

awareness and buy-in of these policies. Olweus Bullying Prevention, Steps to Respect, and Stop

Bullying were the most commonly adopted bullying prevention programs.

Students are now more likely to report incidents of bullying. For example, there was an

incident in which two students were found to be cheating on standardized testing. When


the incident was investigated, it was found that the lesser academically prepared student

had bullied the more academically prepared student into giving him the answers to the

test. As a result of the information on bullying that is sent home by the SS/HS staff,

parents are also more aware of bullying and are more likely to report bullying incidents.

Prior to this initiative, the students and school district staff were unaware and

uninformed about bullying.

Gang Prevention. Among other contributions, sheriffs and law enforcement officials, including

SROs, played a key role in addressing gang issues in the schools through the provision of

trainings and workshops; through presentations to school staff, students, and parents; and

through the implementation of various evidence-based gang prevention programs including Gang

Resistance Education and Training (GREAT) and Gang Resistance Intervention Program

(GRIP). Probation Officers and social workers also collaborated with law enforcement, sheriff’s

departments, and schools to implement these programs, often attending trainings on the

programs.

The Police Department offers the Gang Resistance Intervention Program (GRIP) in two district

schools. The law enforcement partner has worked with school staff to target early indications of

gang-related behaviors, such as bullying, and worked with their SS/HS partners to connect those

youth to their school and community. The law enforcement partner said that this program focuses

on getting troubled youth to connect with their school community rather than connecting with a

gang community.

Implementation Considerations. Several factors facilitated program implementation. Grantees

reported the creation of committees and task forces that were specifically tasked with addressing

concerns regarding safety and violence. The groups consisted of representatives from partnering

agencies, parents, and community members and often helped select programming, organize

trainings and conferences, and assist with writing school safety policies and procedures.

During Year 3, a subgroup formed from the Consortium called the Gang Violence Prevention

Task Force. The task force organized a countywide gang conference with a national speaker

and a relational aggression conference. Representatives of each of the required partner

sectors—including mental health, juvenile justice, local police departments, county sheriff,

and school district representatives—participated in the gang task force. The partnership’s

gang prevention efforts have become more grassroots by getting community members

involved and coming up with action steps to reduce gang activity locally.

State and local budget cuts also provided challenges to the planned implementation of safety and

violence prevention programming. Required partners contributed what they could, as SRO and

PO positions were often funded partly or entirely by law enforcement and juvenile justice

agencies, respectively. Partnering agencies also applied for grants to help fund safety and

violence programs.


SS/HS grantees initiated a number of coordinated services and system changes in order to

address safety concerns and violence in their schools. SROs and POs provided joint trainings on

bullying, gangs, and emergency planning to students, parents, and other SS/HS partners. Some

sites incorporated the Balanced and Restorative Justice (BARJ) framework into their schools to

address violence in the school district. BARJ is a collection of practices and philosophies used in

transforming school environments by recognizing that in every conflict there are opportunities

for problems to be repaired and relationships to be strengthened.

Grantees used a variety of databases to identify, refer, and track violent and dangerous student

behaviors. The resultant data aided the partners in their efforts to coordinate services for

students. For example, one grantee developed a Risk Management System, which was used to

help track academic, behavioral, and mental health issues that may put a child at risk for school

failure, mental health disorders, violence, or other problematic behaviors. Once identified

through the system, students had access to specialized services. A student could be identified by

risk factors such as the student’s failing a grade, having juvenile justice involvement, engaging in

dangerous behavior, or any number of other factors. Students were also identified through

personal contact with a professional who might observe behaviors of concern. These risk factors

were weighted and students were given points. After accumulating a certain number of points the

student was flagged and sent to the guidance counselor, where jointly with mental health, a

determination was made regarding the necessary services. Development of the Risk Management

System involved the school district along with the partners, particularly mental health and

juvenile justice. Through the system, SS/HS partners were granted access to Department of

Juvenile Justice (DJJ) data that were unavailable in the past.

Overall, grantees made progress toward increasing the safety of their schools and reducing the

threat of violence at school. The results of the partners’ collaborative efforts were even felt by

the students themselves:

Bullying prevention programming was a huge success which affected many students. For

example, one little girl was transferred to another county and wrote a letter back to her

former school saying she wished she could bring the Olweus bullying program

implemented by the SS/HS project up to her school.

Prior to the SS/HS project, absenteeism was identified in the student surveys as an issue

because students did not feel safe at school, which relates to bullying and violence at

school. The SS/HS project has been extremely active in developing school safety climate

responses, which includes placing video cameras where teachers and other authority

figures could not be present to observe. The project staff is seeing a change in how the

students as a group respond when they are asked questions about safety in the school;

these improvements are the result of hard work from the project that did not happen

overnight.


Programs and Activities to Address Substance Use Prevention

Grantees were tasked with first identifying needs and gaps for substance use in their schools and

communities, and then with proposing and implementing various strategies to address these

needs.

SS/HS grantees identified several problems concerning substance use in their communities.

Although specific problems varied across sites, examples of problems reported by grantees

included:

Geographic location lends itself to increased drug problems and trafficking

Increasing rates of alcohol and drug use among students

Observed associations between substance use, gangs, and violence

Lack of appropriate services and resources for youths and families

SS/HS partner agencies provided substance use prevention and intervention services, programs,

and community outreach activities throughout the grant funding period. SS/HS grantees reported

implementing programs such as Too Good for Drugs and Too Good for Drugs and Violence,

Project Success, Project Towards No Drug Abuse, and Project ALERT most frequently. Some

substance use prevention programs used by grantees targeted students alone, while others

targeted parents or entire families. Staff members received training on the appropriate

implementation of programs selected, and training to increase their knowledge and

understanding of issues related to substance use. Mental health professionals also delivered

services such as individual and group counseling, and one grantee established a call-in center for

service referrals, for example. Examples of additional programs grantees implemented to address

substance use included:

Drug Abuse Resistance Education (DARE)

Life Skills

Students Against Destructive Decisions

Celebrating Families

Juvenile Alcohol and Drug Education

Guiding Good Choices

SS/HS grantees complemented substance use programming with a number of activities in

schools and local communities. For example, one grantee created an antidrug poster contest for

elementary school students, and posters were displayed in local businesses. At another grant site

students helped redesign an antialcohol logo that was printed on t-shirts and other promotional

materials. Other activities that grantees were involved in to promote awareness of alcohol and

drug abuse among youth included:


Partner presentations and workshops

Screening of family members to assess substance abuse service needs

Creation of ―Safe Home Pledges‖ that parents were asked to sign to commit to

maintaining a safe and alcohol-free environment (e.g., be present at their child’s parties,

not serve alcohol to minors)

Public forums and town hall meetings on drug or alcohol abuse

Marketing campaigns to increase public awareness of substance abuse among youth

Dissemination of a newsletter communicating relevant information (e.g., legislation

regarding the sale of alcohol)

Dissemination of a DVD containing substance abuse prevention messages

Although grantees described the successful implementation of various programs and activities to

prevent or reduce substance use among students, they also encountered several challenges that

affected the provision of relevant services. These challenges included:

Limited funds available for programs and services

Lack of awareness of substance abuse issues

Inability to get school or community buy-in for substance abuse prevention or

intervention activities

Community locale (e.g., proximity to Mexican border and drug trafficking, small

community with limited services available)

Poor student/parent participation in prevention/intervention programs or services

Staffing issues (e.g., recruitment and retention of qualified substance abuse staff, lack of

space to accommodate qualified staff)

Poor partner relationships

One grantee reported difficulty providing services to families living on a local naval airbase. The

grantee described reluctance from military families to seek substance abuse treatment due to fear

of the negative impact that treatment might have on their military career. To address this

challenge, mental health therapists provided services in places other than the airbase (e.g., family

home, local school).

Each of the four partner types (i.e., school district, juvenile justice, law enforcement, and mental

health) provided activities relevant to the agency’s branch of service. For example, most grantees

reported that juvenile justice agencies monitored substance use among students (e.g., provided

breathalyzers, drug testing), law enforcement agencies assessed the degree to which local

businesses were in compliance with the law (to ensure that tobacco and alcohol were not sold to


underage youth), and mental health agencies provided substance use counseling services. Unique

employment positions were incorporated into schools to address substance use among students,

such as an Alcohol, Tobacco, and Other Drug liaison and substance abuse coordinators.

Several factors facilitated partners’ efforts to reduce and prevent drug and alcohol use. Some

grantees reported that the colocation of staff in clinics and in the schools improved the

collaborative relationship between the school district and mental health agency. Grantees also

reported the creation of committees and task forces to address concerns regarding substance use.

The groups consisted of representatives from partner agencies, parents, and community

members, and they often helped select and implement substance use programming. Further,

some committees and task forces helped to create and modify school polices.

SS/HS grantees reported that their efforts to address substance use among students resulted in a

number of positive outcomes. For example, substance use among students declined during the

course of the grant. Grantees attributed the positive student outcomes to a number of factors,

including the prevention programs and activities, the efforts put forth by task forces and

committees, and the increased community involvement in substance use prevention activities.

Several grantees also reported increased access to and receipt of student services. Other

accomplishments described included expansion of prevention programming to schools that were

not targeted by the grant, and the award of Federal and State grants to support efforts to address

substance use among youth.

Activities to Address Access to Mental Health Services

A majority of the SS/HS grantees reported that, in general, there was a paucity of mental health

services in their communities prior to the grant. In addition, there were substantial barriers to

accessing mental health services, including lack of transportation (particularly for rural or

geographically isolated grantees), scheduling conflicts, and inability to pay. Several grantees also

reported that many students with significant, but not life-threatening, mental health needs are

ineligible for services due to recent reductions in State Medicaid funding.

Grantees implemented a variety of activities to improve mental health services and to improve

service coordination among mental health providers and other agencies. These include:

Mental health prevention and early intervention services

Screening and assessment

School-based individual, family, and group counseling

Referral and follow up with outside agencies

Training and consultation to school personnel and other agencies

Educational and supportive services to parents


Prevention and early intervention services. Grantees implemented numerous EBPs to teach

social and emotional skills or improve school climate. For example, sites implemented programs

such as Second Step (violence prevention); Reconnecting Youth (dropout prevention); and

Incredible Years (parenting education) as part of their comprehensive plan to prevent mental

health problems.

Screening and assessment. All of the grantees provided mental health screening and assessment

to students as part of their comprehensive plans. Some grantees developed new screening and

assessment protocols and procedures.

School-based individual, family, and group counseling. Most of the grantees initiated, or

expanded upon, the provision of school-based mental health services. While some grantees had

been providing school-based services for many years prior to the grant award, others hired all-

new staff and created entirely new positions for mental health providers to work in the schools.

School-based mental health providers were funded in a variety of ways: with SS/HS grant funds

or joint funding from the school district and mental health partner. Typically, school-based

mental health services included provision of individual and family therapy; on occasion, there

was also some medication management provided. Therapy could take place in the schools, in the

home, or in the provider’s office. Many grantees had multidisciplinary student assistance teams,

or student support teams, which the mental health provider attended regularly. Typically, these

teams were composed of school administrators, school social workers, other school staff,

probation officers, and sometimes law enforcement.

Referral and followup with outside agencies. Grantees provided mental health referral and

followup with outside agencies, including referrals to community-based mental health services,

as part of their comprehensive plans. Some grantees developed new referral and followup

protocols and procedures. The most common focus across sites for improving community-based,

mental health services included developing or refining the referral processes to mental health

agencies, developing or revising data collection and tracking mechanisms, and increasing

coordination and service integration with increased presence of mental health providers on

multidisciplinary teams. Often, SS/HS grantees reported seeking Medicaid funding in order to

provide school- and community-based services to students lacking insurance.

Training and consultation to school personnel and community-based mental health providers

and other agencies. All grantees provided training and consultation to school personnel and

other agencies on mental health issues (e.g., warning signs of depression, behavior management

techniques), mental health service options, and specific procedures, such as new referral

procedures.


Programs and Activities to Address Early Childhood Development

Grantees identified several gaps in programming and services for young children and their

families:

Many children are inadequately prepared to enter kindergarten.

There is a lack of a system to track programs implemented and their resulting outcomes

so that appropriate changes can be made.

Few local collaboratives focus on addressing the needs of teen parents.

There are inadequate mental health services available to young children.

Services to address children’s psychosocial and emotional development are limited.

Services needed by low-income families are often inaccessible.

High-quality school readiness programs have been associated with positive short- and long-term

outcomes in children, particularly when children and their families receive social, educational,

and health services as part of the program (Winter & Kelly, 2008). Although the individuals that

children come in direct contact with on a daily basis (e.g., parents, peers, teachers of high quality

programs) influence child development most substantially, the child’s community also affects

development (Eccles & Roeser, 1999). Therefore, SS/HS partnerships sought to implement

activities and identify community partners most appropriate in addressing the social, emotional,

and academic needs of young children so they could enter school prepared to learn.

Grantees implemented a range of activities to promote the positive development of young

children. These activities included the implementation of programs focused on young children’s

social/emotional development (e.g., Second Step, Incredible Years), aggressive or antisocial

behavior (e.g., First Steps to Success), physical health (e.g., Color Me Healthy), and school

readiness (e.g., Born to Learn). In addition to providing programming that targeted young

children, grantees also implemented activities and programs for parents so that they have the

tools necessary to support the development of their young children. For example, some sites

implemented Parents as Teachers, encouraged parent involvement in their child’s activities, and

provided parents with training on appropriate parenting techniques. Additionally, grantees

acknowledged the needs of teen parents by providing teen parenting programs and providing

prenatal services, for example. One site received additional grant funds and partnered with a

local nonprofit organization in an effort to build a childcare facility for teen parents.

Although grantees described the successful implementation of several activities to increase the

likelihood that children enter kindergarten prepared to learn, they also encountered several

challenges. For example, it can be difficult to get buy-in for early childhood programming from

staff at child care centers, from school boards and principals, and from early childhood

educators, who express concerns about implementation requirements (e.g., the time required for

program implementation, how services will be delivered, to whom services will be delivered). To


address these concerns, program implementers made sure to provide as much information as

possible about the programs and implementation process when SS/HS staff implemented a

program. SS/HS staff also took time to establish rapport with early childhood educators when

those educators were responsible for program implementation. Other concerns reported by

grantees related to struggles to build and maintain relationships with program staff and local

organizations. For example, sites reported turnover among early childhood program staff, and

difficulties establishing and maintaining relationships with local organizations delivering early

childhood programming. Other challenges included uncertainty about how to integrate grant

programs with preexisting initiatives, and concerns about how to maintain early childhood

activities after the grant ends.

Staff responsible for implementation of programs or services related to early childhood

development were either part of a team that jointly implemented activities, or acted as liaisons to

build partnerships between early childhood educators and the community. Early childhood staff

members engaged in a range of activities to meet the needs of their targeted population,

including:

Conducting various assessments (e.g., using a screening process to identify young

children who may have a cognitive delay, assessing school readiness)

Providing in-home services for parents and families

Providing staff training and consultation

Ensuring that families know how to access services in the community (e.g., how to locate

food banks, where to find healthcare information)

A number of agencies were reportedly involved with the implementation of early childhood

programming. Some grantees formed task forces or subcommittees focused on early childhood

issues. Agencies/organizations that SS/HS partners collaborated with include Head Start, local

colleges, Planned Parenthood, and social services. There has been representation on the task

forces and subcommittees from mental health providers, educators, school administrators, and

health care providers. These agencies and representatives collaborated with SS/HS partners to

achieve various goals, such as providing a continuum of mental health services, sharing

resources and data, providing ongoing training and support to early childhood educators, and

providing programming or services.

SS/HS grantees reported that their efforts resulted in a number of positive outcomes to aid young

children and their families. Grantees reported increased buy-in at the State level and from early

childhood educators. For example, by the end of the SS/HS funding period, one State-level

agency funded four of seven pre-K classes that were originally funded by the SS/HS grant.

Additionally, although the workload of preschool teachers at one site increased as a result of the


implementation of grant-related activities, the teachers expressed enthusiasm about program

activities and requested more program training. Grantees reported other key outcomes, including:

Increases each grant year in the percentage of preschool students demonstrating readiness

for kindergarten

Increases in the number of families served

Systems changes (e.g., policy change mandating prescreening kindergarteners; joint

planning between agencies serving young children)

IMPLEMENTING COORDINATED AND INTEGRATED SERVICES

Research indicates that prevention efforts are most effective when families, schools, community

organizations, and health care and service systems work together to implement programs and

activities to help students (Epstein, 1995; Weissberg et al., 2003). According to the literature, an

integrated service approach provides a comprehensive range of services in a geographic area

with the purpose of improving the effectiveness of service delivery (Kendrick, Jones, Bezanson,

& Petty, 2006). Examples of integrated/coordinated service delivery include resource mapping

and matching, mechanisms for resource coordination, confidential information-sharing

procedures, and training resource coordination teams (Adelman & Taylor, 1999; McMahon,

Ward, Pruett, Davidson, & Griffith, 2000). A main focus of SS/HS is on helping agencies

coordinate and integrate their services so they can improve outcomes among students and

schools.

A review of interview notes from a sample of 2005, 2006, and 2007 grantees revealed additional

descriptive information on the types of coordination and service integration activities that SS/HS

grantees implement. Data on these grantees suggest that most had some degree of

coordinated/integrated services. In some cases, the partners had worked together and offered

integrated services prior to the grant and enhanced these efforts during the grant period. Grantees

engaged in the following types of activities to enhance integrated/coordinated efforts:

establishing referral and information-sharing procedures, providing training across agencies,

establishing case management teams to follow students’ mental health services, and sharing

resources.

Referral Relationships. The SS/HS Initiative enabled grantees to develop and clarify referral

procedures for services such as counseling, parenting education, diversion programs for at-risk or

adjudicated youth, and other interventions. The referral procedures were often communicated

during trainings with school personnel and partners. Training session topics included review of

services available, populations targeted, criteria for admission, and possible sources for referrals.

Additional benefits of revamped referral processes included improved communication among

partners, the ability to follow up on students referred, and shorter waiting times for services.

Ultimately, improving referral procedures benefited everyone involved and contributed to


positive working relationships among grant staff, school personnel, and partners. Sites employed

the following types of referral mechanisms:

Streamlined or standardized referral processes, with a single point of contact, often a

mental health coordinator, to clarify the communication process

Standardized referral forms for all partner agencies

Referral to less punitive options, such as counseling or other social services, rather than

directing youngsters exhibiting problematic behaviors to the courts for adjudication

A committee of SS/HS project staff and partners to establish procedures for identifying

and referring pregnant teens to the Nurse Family Partnership

Skills enhancement as part of the referral procedures training, including identification of

the developmental levels of children referred for early childhood services

Case Management. Most sites offered case management services—often for particularly

challenging students. Case management was either coordinated by one staff person, such as a

case manager who served as a liaison between the various professionals involved with the

student, or a team responsible for linking students and families to needed services. The case

management and other multidisciplinary teams fostered better communication and working

relationships among professionals, eliminated gaps in service, avoided service duplication, and

provided more comprehensive services than would have otherwise been possible. Partners noted

that these multidisciplinary teams helped service providers operate more efficiently and

effectively, and contributed significantly to service integration and systems change. Examples of

case management efforts included:

A dedicated case management unit composed of professionals such as a licensed clinical

social worker, public health nurse, and Child and Family Services staff

Care coordination teams, typically consisting of a school administrator, counselor, care

coordinator, juvenile officer, and SRO

Case management teams to supplement the activities of an already established

Coordination of Services team, whose development was assisted by SS/HS staff to offer

more intensive services

Student Safety Assessment Teams composed of mental health, law enforcement, juvenile

justice, and other personnel to assess the threat level of especially troubled students and

provide interventions

Multidisciplinary teams to identify the most appropriate services and service plans for

students with emerging, or chronic, behavioral and/or psychological issues, e.g., truancy,

disruptiveness, violation of school rules, and criminal behavior; these multidisciplinary

teams identified the most appropriate services, and developed service plans

Case management activities focused on limiting juvenile court involvement, such as

assigning an at-risk intervention specialist to help students successfully transition from

juvenile custody to their neighborhood schools


Assignment of a juvenile court liaison/case manager to work with adjudicated youth and

families by offering a range of services, with the intent of promoting pro-social behaviors

and helping youth avoid further contact with the criminal justice system

Information-Sharing Procedures. Grantees implemented an array of activities to improve

information-sharing across partners and agencies, thereby enhancing integration of services.

Activities included information-sharing agreements and policies, database tracking systems,

communication feedback forms, and memorandums of agreements for confidentiality. In most

cases, several partners were involved (e.g., juvenile justice, law enforcement, schools, probation

officers, child protective services), with mental health and school partners being the most active

participants. Examples of information-sharing activities included:

Partners signed agreements to share information relating to out-of-home placements,

mental health issues, and gang/violence prevention.

Three mental health centers adopted a common communication feedback form for school

principals and data collection purposes.

Grantees tracked students receiving services and shared data with after-school club

sponsors (e.g., alcohol use data in order to reinforce alcohol prevention in the programs).

Mental health partners developed an electronic data system for clinical staff to access

student records and share information.

Partnership members signed a memorandum of agreement vowing to adhere to the same

confidentiality policies and procedures.

Resource Sharing. Grantees engaged in a range of resource-sharing activities that helped to

develop more comprehensive, coordinated programs and services. The most common activities

involved training school staff in SS/HS programs and training partner agencies on screening and

referral procedures. Other activities included resource mapping, crisis and emergency planning,

and development of multiagency programs, such as mentoring. Examples of resource sharing

activities included:

A districtwide counseling plan that identified all mental health resources and allocated

resources so that all schools in the districts had at least one mental health counselor

Mapping of available gang resources in order to focus intervention, prevention, and

suppression activities

Cross-trainings on topics such as gang awareness, school safety, and conflict resolution

Development of an attendance manual distributed to all school administrators by an

SS/HS attendance improvement team

A truancy reduction program that brought together partner agencies to establish truancy

child study teams, including all providers dealing with a particular student, to eliminate

duplication of services and treat students holistically


Training of teachers to identify and refer students for mental health assessments and

services

The 2005 and 2006 grantees made significant progress in coordinating and integrating services

based on survey data. The majority of grantees now have coordinated data systems:

More than 97 percent of the grantees established processes for sharing data to evaluate

activities.

More than 70 percent of the grantees established a process for monitoring the quality of

screening and assessments.

More than 70 percent of the grantees established a system for tracking outcomes.

More than 60 percent of the grantees established a treatment monitoring information

system shared across agencies.

The majority of grantees also have more integrated services than in Year 1 of the grant:

More than 98 percent of the grantees established processes for identifying and linking

students to services.

More than 76 percent of the grantees established service delivery teams that include

members from various systems.

The number of coordination and service integration activities implemented increased

significantly, from an average of 9.9 in Year 1, to 14.0 in Year 2, to 16.1 in Year 3. In Year 3, 63

percent of the grantees had implemented between 16 and 20 coordination and service integration

activities.

IMPLEMENTING ENHANCED SERVICES

The SS/HS grant strongly encourages sites to use EBPs, interventions and treatments with

empirical evidence supporting their effectiveness, as part of their comprehensive approach.

Thirty-six specific EBPs were used by two or more of the grantees. These EBPs ranged in scope

and complexity from specific curricula with proven outcomes in preventing substance abuse or

violence, to communitywide interventions that require the recruitment of private-sector

participants. Site visit and group interview data from the 2005, 2006, and 2007 cohorts suggest

that grantees selected specific EBPs based on reviews of their local area needs and through the

subsequent research on programs to address those needs. The reputation of the EBPs was a

significant factor in the decision to adopt a program.

“During the planning phase of this grant, school district staff worked with the required partners

to identify and select programs that best addressed the needs of the community. The grant writing

team spoke with the [student advisory] committee to discuss the needs assessment data and seek

their input on the most effective programs and strategies to address these needs. Programs


offered through SS/HS include a variety of evidence-based programs tailored to meet the cultural

and local needs of the area.”

“The two school district guidance counselors who are members of the core management team

suggested the use of this curriculum based on their positive experiences implementing the

curriculum.”

Exhibit 13 shows the interventions grantees used most frequently and the areas each intervention

addressed.

Exhibit 13. Most Frequently Implemented EBPs (2005 and 2006 Cohorts)

EBP AREAS ADDRESSED

PERCENTAGE OF

GRANTEES THAT

IMPLEMENTED THE EBP

Second Step Violence prevention 42%

Positive Behavioral Interventions and Supports

Academics and behavior 31%

Student Assistance Program Mental health promotion, substance use prevention

29%

Too Good for Drugs Substance use prevention 29%

LifeSkills Training Substance use and violence prevention, behavior

25%

Project ALERT Substance use prevention 20%

Parents as Teachers Early childhood development 19%

Strengthening Families Program

Mental health promotion, substance use prevention, and behavior

19%

Olweus Bullying Prevention Program

Violence prevention/bullying 19%

Although grantees implemented a number of different EBPs, grantees reported common barriers

to EBP implementation. Personnel issues were a major obstacle. Grantees experienced resistance

to outsiders and new people working on school grounds. Grantees reported lacking staff with the

necessary credentials to carry out the EBPs, having an already overburdened staff, and not

having enough time in the school day to carry out EBPs.

“The project director reportedly encountered some initial resistance at the school level. As described

by the project director, the school principals are accustomed to overseeing the activities and staff

within their buildings, and some were hesitant to allow outsiders to determine the programs that

would be implemented.”

IMPLEMENTING CULTURALLY AND LOCALLY RESPONSIVE PRACTICES

Grantees enhanced services by selecting and implementing programs and activities sensitive to

the needs of the target population and taking into account culture, socioeconomic status, or other

factors. Researchers and service providers agree that physical and mental health programs and


interventions tailored for specific populations are important and are more likely to succeed when

providers are knowledgeable about cultural differences and their impact on health-related

attitudes and behaviors, and respect the values of the target group. Often, a lack of sufficient

knowledge and awareness of the varying lifestyle patterns and needs of populations have resulted

in inadequate service delivery, lack of compliance with expected norms and standards of

behavior, and inconsistent or poor responses to caregivers and facilities providing prevention

services (Butler, 1996).” Research also suggests that it is important that there is congruence

between EBP requirements and the unique circumstances of the target population” (Green &

Mercer, 2001).

SS/HS grantees engaged in a range of activities to address the needs of various cultural groups

and other local area needs that included:

Providing information in multiple languages

Providing programming or services appropriate to the cultural groups or local needs

Including local community organizations or members from a specific cultural group in

program planning

Developing workgroups or subcommittees to address cultural or local needs

Providing staff training on cultural competency and other relevant topics

Maintaining culturally sensitive and diverse staff

Participating in community outreach activities

Among grantees that identified a specific group targeted for services, most targeted Latino or

Spanish-speaking students and parents, and low-income families or families living in poverty.

Many sites also targeted African American and Native American students and families, and

students in a gang or at risk of joining a gang.

Many SS/HS grantees addressed the needs of students and parents with limited English

proficiency by providing information in multiple languages. For example, several sites translated

program materials or information used to communicate with parents (e.g., documents published

by the school board, consent forms, automated message system) or members of the community

(e.g., newsletters, SS/HS marketing messages, fact sheets, Web sites) into languages other than

English. Interview respondents from some grantee sites reported that they had a translator on

staff, or they made sure that translation services were available upon request. Additionally, a few

grantees provided local evaluation materials (e.g., student surveys) and information on school

emergency procedures or other school policies in multiple languages.

Most SS/HS grantees reported that they implemented services or programs to address the needs

of cultural groups or local area needs. Several grantees implemented programs tested with

specific cultural groups, adapted a particular EBP so as to improve its applicability to a minority


population, developed a new program, or implemented a program developed to target a particular

cultural group (e.g., Effective Black Parenting, Nurturing Parenting).

In an effort to address the needs of minorities served through the SS/HS grant, agencies hired

staff reflective of the target population, or provided necessary staff training to ensure that

culturally competent services were delivered. For example, many service providers were

described as bilingual or bicultural or were members of a minority group. Additionally, several

grantees provided cultural competency trainings/workshops for staff. However, needs were not

limited to minority populations; many grantees also described local concerns, including poverty.

To address this need, trainings (e.g., A Framework for Understanding Poverty, Bridges Out of

Poverty) were provided to both direct-line staff and partners.

Other methods grantees used to address the needs of cultural groups or local area needs included

engaging community members or local organizations in the development and/or implementation

of programs, and developing workgroups or subcommittees focused on addressing their needs.

For example, several grantees engaged local organizations working directly with specific

populations (e.g., faith-based organizations working with African American families, agencies

providing programs for low-income families, partnerships with local Native American tribes).

Additionally, SS/HS partners developed or participated in workgroups or on subcommittees to

ensure students from minority groups, students in poverty, or other students with unique needs

had access to appropriate services; that their needs were being met; and that staff were provided

with necessary training to meet student needs. Examples of workgroups or subcommittees

identified included a racial overrepresentation (in social services) committee, a multicultural

committee, a cultural competency committee, and a gang prevention committee.

CONCLUSION

There is a good deal of evidence that SS/HS grantees are partnering and collaborating with their

communities and integrating services. The grantees expanded their partnerships beyond the

grant-required agencies to include a variety of community-based organizations or groups in

program planning, implementation, and resource sharing. These organizations worked with the

grant administration and schools to implement comprehensive services across the grant element

areas and incorporate EBPs and other enhancements to address specific local needs. Grantees

also made significant progress over the duration of the grant in regard to coordination and

service integration. Nearly all of the grantees established processes for identifying and linking

students to services, coordinated data systems, and formed or enhanced multidisciplinary service

teams to address student and community needs.

Interim Report of the SS/HS Initiative National Evaluation: 2005 and 2006 Grantees 47

FACTORS THAT INFLUENCE THE EFFECTIVENESS OF THE

SAFE SCHOOLS/HEALTHY STUDENTS GRANT

he SS/HS grantees are a highly diverse group. Each grantee faces its own unique

circumstances and has addressed these challenges using its own combination of

strategies. Nevertheless, evaluation and research to date suggest a limited number of

factors affect the ability of the SS/HS grantees to achieve the grant goals. When grantees

described factors contributing to the success of their SS/HS project, many cited the following:

Building strong partnerships, communicating regularly with partners and other

stakeholders, understanding each other’s needs and challenges, and getting buy-in early

Developing a clear partnership structure with defined roles, a clear mission and vision,

teams or committees, and shared decisionmaking

Getting strong support from the superintendent of schools and board of education

Hiring and retaining a project director with strong leadership skills and close ties to the

community

Collecting and using data to choose the right programs, make sure they are carried out

properly, monitor results, and market the project to stakeholders and potential funders

Developing a sustainability plan as early as possible

Helping partners integrate services by educating them about how to share information

and streamline referrals without violating privacy laws

Networking with counterparts from other grantee school districts

Being respectful of cultural differences in the community and using effective outreach

and hiring practices

The national evaluation of the grantees examined quantitative relationships between independent

factors (e.g., partnership characteristics and the number of activities implemented) and school

staff ratings of improvement in specific areas targeted by the grant. (See Appendix I for a

technical report on the analysis). Three areas of findings are presented below: the impact of the

pre-grant environment on staff ratings of improvement; the impact of comprehensive,

coordinated, and integrated services on staff ratings of improvement; and the impact of grant

T


operations and staff ratings of improvement. When possible, the findings are clarified using

qualitative information from the site visits and telephone interviews.

IMPACT OF THE PRE-GRANT ENVIRONMENT

No significant associations were found between the pre-grant environment and behavioral

outcomes. However, certain pre-grant characteristics did predict improvements reported by

school staff in a number of areas. Specifically:

Grantees in communities where fewer adults have a high school diploma reported greater

improvement across all grant elements, particularly safety and violence, than grantees in

communities with higher proportions of adult graduates.

Grantees in communities where more families with children are below the poverty line

reported greater improvement across all grant elements, particularly substance use

prevention, mental health services, and the school’s relationship with the community,

than grantees in communities where more families with children are above the poverty

line.

Grantees with more funding per targeted student reported greater improvement in mental

health services than grantees with less funding per targeted student.

In addition, qualitative data indicated that having a robust, longstanding partnership history

prior to the grant is another factor that predicted greater success with regard to perceived

improvements. Grantees that reported lower levels of improvement were more likely to report

they had used the grant as an opportunity to learn how to collaborate.

Grantees that saw relatively strong improvement across all grant areas also usually employed a

wide range of local resources prior to grant award, and many focused on expanding existing

resources rather than implementing new programs. Grantees that witnessed lower levels of

improvement consistently reported a significant lack of resources prior to grant award.

IMPACT OF COMPREHENSIVE, COORDINATED, AND INTEGRATED SERVICES

Among programs and activities designed to coordinate, improve, or expand services, a few types

in particular appeared to have more of an effect than others on staff-reported improvements.

Implementing a higher number of early childhood development-related programs and activities

was associated with higher school staff ratings of improvement across all areas of the grant. In

particular, these programs appeared to have a significant effect on safety, violence prevention,

improved mental health services, and improved school-community relationships.

IMPACT OF GRANT OPERATIONS

Grantees that reported more activities directed at coordination and service integration reported

greater overall staff perceptions of improvement in specific grant areas at the end of Year 3.


These grantees also differed in other key ways from those that reported fewer of those types of

activities. Exhibit 14 compares the characteristics of grantees that scored high and low in

coordination and service integration in three areas of grant operations: pre-grant planning,

communication, and sustainability planning.

Exhibit 14. Factors Associated With Higher and Lower Coordination and Service

Integration Ratings

THEME HIGHEST-SCORING SITES LOWEST-SCORING SITES

Pre-Grant Planning

All individuals or groups involved in planning for the grant and developing the application remained involved post-grant award

Only one reported continued involvement of individuals or groups involved in planning for the grant and developing the application post-grant award

Communication Reported consistent communication

about grant goals and activities

None reported good communication; all indicated communication was a barrier

Sustainability

Planning Reported taking some action toward

planning for sustainability in Year 1 None reported any sustainability

planning activity in Year 1

Aspects of grant operations also had apparent effects on staff-perceived improvement in specific

grant areas, as shown in Exhibit 15. The variables most strongly linked to perceived

improvement were higher average partnership functioning scores, higher perceived importance

of SS/HS resources in the school, and more positive perceptions of the implementation process

in the school.

Qualitative data indicated that grantees that saw greater improvements frequently had clear

partner roles, which involved having a clear implementation plan and a shared understanding

about roles. Conversely, grantees that saw lesser improvement indicated there was confusion

about the roles of partners.

Good communication and collaboration throughout the grant was another positive factor.

Grantees that saw lesser improvement usually reported poor communication or indicated only

late in the grant that communication was improving.

Making implementation changes appeared to have a negative effect on the grant’s

effectiveness. In Year 3, only one of the grantees reporting the highest levels of improvement

reported replacing a program or activity during the grant. In contrast, four of the five grantees

with the lowest levels of improvement said they added new programs or removed inappropriate

or unsuccessful programs during Years 2 and 3.


Exhibit 15. Factors Associated With More Staff-Reported Improvement in Grant Areas

GRANT AREA FACTORS POSITIVELY INFLUENCING PERCEPTIONS OF IMPROVEMENT

Violence and Substance

Use Prevention

Partnerships with higher average functioning scores Higher perceived importance of SS/HS resources More positive perceptions of the implementation process in the school

Mental Health

Greater key partner involvement in decisionmaking Fewer communication-related barriers to implementation Higher perceived importance of SS/HS resources More positive perceptions of the implementation process in the school

Early Childhood

Development

Partnerships with committees had more positive perceptions of improvements

Greater key partner involvement in decisionmaking More positive perceptions of the implementation process in the school

School, Family, and

Community Connections

Fewer barriers to collaboration among the partnership Higher perceived importance of SS/HS resources More positive perceptions of the implementation process in the school

Finally, results from a survey on sustainability provided an opportunity to rank grantees on the

degree of sustainability achieved. Four distinctions between the most successful and least

successful sites from the quantitative data were also highlighted during the focus group session

and group interviews.

Grantees that reported well-established partnerships at the beginning of the grant period

made more progress towards sustainability than those that reported new and developing

partnerships.

Grantees that made the most progress toward sustainability tended to have organizational

structures that included more committees than did grantees that made the least progress

toward sustainability.

Grantees that showed the most progress toward sustainability by the end of the grant

reported using diverse, creative strategies such as using Web-based training and applying

for nonprofit status so they could become eligible for specific types of funding. Grantees

that made less progress toward sustainability tended to rely on funding from the school

district.

Grantees that reported few barriers made more progress towards sustainability than those

that reported many barriers, particularly communication and financial barriers.


In addition to the patterns above, partnership functioning scores at Year 2 were positively and

significantly correlated with progress toward sustainability. For those sites that secured funding

compared to those that did not, partnership functioning scores at Year 2 were significantly

greater. In addition, grantees that scored high in sustainability were also those that reported well-

established partnerships at the beginning of the grant period and reported few barriers.


CONCLUSION

his report provides evidence of the effectiveness of the SS/HS Initiative based primarily

on findings from the 2005 and 2006 cohorts of SS/HS grantees. The results indicate that

the SS/HS Initiative is achieving its goals and supporting the development of

comprehensive plans and the delivery of enhanced, integrated, and coordinated services. The

activities the grantees implemented, combined with increased collaboration among the SS/HS

partners, were associated with improvements in school safety, violence and substance use

prevention, and access to mental health services. The improved long-term outcomes, including

the sustainability of SS/HS partnerships and activities after grant funding ends, highlight the

importance of expanding the Initiative’s reach into more communities and schools. The

improvements among SS/HS grantees since 2005 are in contrast to national trends. Data for the

same period from sources such as the Youth Risk Behavior Survey (CDC, 2010) showed no

significant improvements in violence, school safety, or current substance use.

The following findings from the national evaluation are particularly noteworthy:

Cross-site analyses of GPRA data did not find any pre-grant conditions or resources that

influenced behavioral outcomes achieved under the grant. This is encouraging because it

supports the assumption that any community can implement comprehensive, coordinated, and

integrated plans to address and meet the social, emotional, and behavioral needs of students and

families, and improve the connection between schools and the community, if appropriate funding

and guidance are provided. However, according to the SLS data, some school districts and

communities may value the grant more highly than others; school staff in communities

where fewer adults have a high school diploma and where more families with children are below

the Federal poverty level reported greater improvements across all grant elements than school

staff in communities with higher proportions of adult graduates and lower rates of poverty.

The success of grantees is strongly influenced by communication among project partners,

school administrators and staff, and the community. Grantees that clearly communicated

roles and responsibilities to partners and actively engaged school staff in the implementation of

the grant reported greater improvements in school safety, violence and substance use prevention,

early childhood development, and access to mental health services than those with unclear roles

and responsibilities and poor communication.

The value of the partnerships developed or enhanced through the SS/HS grant should not

be understated. Across all of the core grant areas (i.e., safety, violence prevention, substance

use prevention, mental health access, early childhood development, and supporting and

connecting schools), higher-functioning partnerships were associated with greater improvements

reported by school staff. Partnership functioning was measured by examining average partner

responses to a 12-item scale on the partnership inventory. In addition, greater partner

T


involvement in decisionmaking regarding grant-related activities was associated with greater

improvements in mental health and early childhood development reported by school staff.

Not surprisingly, the amount of funding and resources available to grantees influenced their

outcomes. School staff reporting grant resources were important in their school also reported

improvements in violence and substance use prevention; mental health; and school, family, and

community connections—they may have valued the grant more highly because of those

perceived improvements. The amount of grant funding per targeted student was significantly

related to improved access to mental health services, which may indicate that mental health

services cost more than other activities and therefore require more funding.

CMHS will continue to collect and analyze available data from the 2007 and subsequent cohorts,

whose grants are still in progress, and integrate it with analyses of data from the 2005 and 2006

cohorts to enhance understanding of the overall SS/HS Initiative and its results.


ACRONYMS

ATOD alcohol, tobacco, and other drug use

BARJ Balanced and Restorative Justice

CMHS Center for Mental Health Services

DARE Drug Abuse Resistance Education

DJJ Department of Juvenile Justice

DOJ U.S. Department of Justice

EBP evidence-based program

ED U.S. Department of Education

FY fiscal year

GPRA Government Performance and Results Act

GREAT Gang Resistance Education and Training

GRIP Gang Resistance Intervention Program

HHS U.S. Department of Health and Human Services

LEA local education agency

NET National Evaluation Team

PDI participating project director

PI partnership inventory

PI-PD Partnership Inventory-Project Director

PI-RP Partnership Inventory-Required Partner

PLS project-level survey

PO Probation Officer

SAMHSA Substance Abuse and Mental Health Services Administration

SCS School Climate Survey

SLS school-level survey

SRO School Resource Officer

SS/HS Safe Schools/Healthy Students

U.S.C. U.S. Code

draft final technical report of the safe schools/healthy ... · he safe schools/healthy students...

Documents