running head: promising artists in recovery program
TRANSCRIPT
Running Head: PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
Promising Artists in Recovery Program Evaluation
Michael J. Irons
A Capstone Project presented in partial fulfillment of the
requirements for the degree of
Master of Arts in Policy Studies
University of Washington Bothell
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
2
Abstract
The implementation of interventions designed to address the needs of juvenile justice
involved youth, while also conducting program evaluations, provides critical feedback to
program deliverers and policy makers. This study used a mixed-methods approach to
assess the impact of the Promising Artists in Recovery program (PAIR) on juvenile
justice involved youth participants diagnosed with a substance abuse problem. This
study relied upon an empirically based risk assessment, a post intervention youth survey
focusing on risk and protective factors, and qualitative data from interviews with program
deliverers. The use of risk and protective factors in assessing the likelihood of youth
substance abuse problems has been validated through meta-analysis research and this
study intends to answer whether there are significant differences in the dynamic risk and
protective factors of youth involved in PAIR, as measured by the Positive Achievement
Change Tool (PACT). A Wilcoxon Signed Rank Test and McNemar’s Test was used to
measure changes in PACT risk and protective factor scores from Time 1 (pre-PAIR) to
Time 2 (post-PAIR). Post intervention participant survey results and PACT scores were
analyzed to determine correlations between youth attendance and identified risk and
protective factors. The study findings indicated youth involvement in PAIR significantly
increased the number of positive adult non-family relationships, and interest and
involvement in pro-social structured activities in the study group. Findings also revealed
correlations in PAIR attendance and participant increases in encouraging adults, pro-
social community ties, and youth motivation to abstain from drug and alcohol use. The
results of this study have policy implications for juvenile justice stakeholders.
Keywords: juvenile justice policy, arts, risk and protective factors, and substance abuse
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
3
TABLE OF CONTENTS:
CHAPTER 1: PURPOSE OF THE STUDY 5 CHAPTER 2: REVIEW OF LITERATURE 18
CHAPTER 3: METHODOLOGY 26 EXPERIMENTAL & CONTROL GROUP 26
DESIGN & MATERIALS 27 MEASURES 28
DATA COLLECTION PROCEDURES 32
ANALYSIS STRATEGY 33
ETHICAL CONSIDERATION 35 CHAPTER 4: RESULTS AND DISCUSSION 38
RESEARCH QUESTION 1, PROTECTIVE FACTORS CHANGES 38
RESEARCH QUESTION 2, RISK FACTORS CHANGES 40
RESEARCH QUESTION 3, PRO-SOCIAL RELATIONSHIPS 41
DISCUSSION 49
CHAPTER 5: CONCLUSION 52 EXPECTED FINDINGS 52
LIMITATIONS & FUTURE WORK 54 APPENDICES 58
REFERENCES 58 APPENDIX A 64
APPENDIX B 69 APPENDIX C 70
TABLES 73
FIGURES 80
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
5
Chapter 1-Purpose of the Study:
The intent of this study is to examine the effects of the Promising Artists in
Recovery program (PAIR) on juvenile justice involved youth in Snohomish County,
Washington. A report published in 2011 by the Office of Juvenile Justice and
Delinquency Prevention (OJJDP) revealed that in 2008, 57% (322,900) of adjudicated
delinquency cases nationally resulted in formal probation being ordered by the court
(Puzzanchera, 2011). Nationally, the highest recorded juvenile arrest rate for the years
1980-2009 was 9,107 arrests per 100,000 youth, documented in 1996, while the arrest
rate dropped to 5,804 per 100,000 youth age 10-17 in 2009, the lowest level recorded in
29 years (National Center for Juvenile Justice, 2009). In 2008, within Snohomish
County, there were 1,954 delinquency petitions filed out of an estimated 77,200 youth
age 10-17 years old (OJJDP, 2008).
Inhabitants of Snohomish County are distributed throughout 2,087 square miles
between Puget Sound and the Cascade Mountains (QuickFacts, 2012). In 2010,
Snohomish County had an estimated population of 713,335 residents and the populace
was made up of 78.4% White, 8.9% Asian, 9.0% Hispanic or Latino origin, 4.6% two or
more races, 3.8 % Some Other Race, 2.5% Black, 1.4% American Indian and Alaska
Native persons, and 0.4% Native Hawaiian and other Pacific Islander persons (U.S.
Census Bureau, 2010). According to Puzzanchera (2013), 13.1% of all arrests (2,673
total arrests and 3,441 per 100,000 youth) in Snohomish County were of youth age 10-17
years old in 2010. Snohomish County’s arrest rate was 30% lower than the national
average of 4,889 per 100,000 youth during that same year (Puzzanchera, 2013). In 2010,
Snohomish County Juvenile Court had a total of 1,405 juvenile offender case filings,
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
6
which included 332 felony cases and 1,073 misdemeanor/gross misdemeanor offenses
(Washington Courts, 2013).
Juveniles who engage in drug and alcohol use are at increased risk for criminal
behavior with 60-85% of youth within the criminal justice population diagnosed with
substance use disorders (Barnoski, 2004). While drug and alcohol abuse affects a
significant proportion of juvenile offenders, evidence suggests that offering strength-
based community mentoring programs can reduce the frequency of drug/alcohol relapse.
A 2010 meta-analysis revealed that mentoring programs had the second largest positive
effect on recidivism reduction among eight generic program types, although all of the
programs showed positive effects (Lipsey, 2010). Rigorous program evaluation of
mentoring programs needs to be completed to establish best-practices and evidence-based
interventions designed to address substance abuse issues within the juvenile justice
population. The Washington Association of Juvenile Court Administrators has also
supported this rationale; “Additional research-proven programs are needed for other
groups of moderate to high-risk youth, programs for non-aggressive youth from stronger
families, and programs for youth with an alcohol/drug problem” (Barnoski, 2004, p. 4).
With limited resources available to juvenile justice agencies, efficient use of program
funding is necessary as is accountability of funds. This makes the study of newly
developed community-based mentoring programs a critical endeavor. As Greenwood
(2008) describes when referring to juvenile offender interventions, “Programs can no
longer be promoted for wide-scale dissemination until they have been proven effective by
a rigorous evaluation” (p. 206).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
7
One such program in need of evaluation is the PAIR program, a community-based
mentoring program currently underway in Snohomish County. As a pilot intervention
strategy that began in October 2011, PAIR was established to provide strength-based
mentoring opportunities to court involved youth diagnosed with a substance abuse
problem. Snohomish County Juvenile Court received a grant from the Robert Wood
Johnson Foundation in 2010 to implement the Reclaiming Futures model in an effort to
improve the coordinated response of juvenile justice, substance abuse agencies, and the
community (Hefley, 2010). The Reclaiming Futures model focuses on providing better
substance abuse screening and assessment, along with service coordination in the
community that promotes pro-social activities (Reclaiming Futures, 2012). A 2007
program evaluation report indicated the Reclaiming Futures model, “…is a potentially
effective method for improving a community’s response to delinquency and substance
abuse” (Butts, 2007, p. 25). The Butts (2007) Reclaiming Futures evaluation was based
on repeated surveys of stakeholders within the program sites from 2003-2006 although
the evaluation did not include client-based data (p. 3). The judges, probation counselors,
substance abuse providers and members of the local community have been working
together under the Reclaiming Futures model since 2011 in an effort to better serve
juvenile justice involved youth in Snohomish County.
Best-practices within the juvenile justice system of Washington State and
nationally require the adoption of empirically based risk and needs assessments focusing
on the static and dynamic risk and protective factors of juvenile offenders. Matching the
appropriate intervention to each individual youth’s criminogenic needs, as identified
through a risk and needs assessment, is important in evaluating and designing drug abuse
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
8
prevention strategies (Hawkins, 1992; Hoge, 2008; Vincent, 2012). Dynamic risk factors
for juvenile offenders include those areas of a youth’s life that are changeable, including;
current relationships, family dynamics, current drug/alcohol use, aggression, social skills,
attitudes and behaviors, and others (Barnoski, 2004). Static risk factors for juvenile
offenders are those factors that cannot be changed, including; criminal history, past
drug/alcohol use, history of running away, prior expulsions and suspensions from school,
mental health diagnosis, and more (Barnoski, 2004). Juvenile justice providers must
concentrate efforts and resources on influencing changes to the dynamic risk and
protective factors of juvenile justice populations.
Risk and needs assessments are delivered with fidelity when intensive training
and support are provided to probation counselors that administer the evaluation tool
(Vincent, 2012; Hoge, 2008). In 1998, Washington State courts, including Snohomish
County, adopted an empirically based risk needs assessment developed at the Washington
State Institute for Public Policy called the Washington State Juvenile Court Assessment
(WSJCA). This locally developed tool has been modified over the years and adopted by
courts througout the United States. The current interation of the assessment instrument,
provided by Assessments.com, is called the Positive Achievment Change Tool (PACT)
(Assessments.com, 2013). The PACT was developed from the latest research on the
correlates and causes of criminality in juvenile offenders (Hoge, 2008).
Juveniles receiving probation services in Snohomish County and Washington
State are assessed using the PACT assessment tool. The PACT contains 126 items of
measurement within 12 domains correlated to juvenile delinquency. The assessment tool
has an established statewide juvenile Assessment Quality Assurance Committee and full
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
9
time qualtiy assurance specialist, as well as on-going training and monitoring
administrative support in order to ensure assessment reliability (Barnoski, 2009). This
quality assurance component has implications for juvenile court interventions, as well as
the inter-rater reliability of the PACT (Barnoski, 2004). By ensuring probation
counselors in Washington State are properly trained, youth are referred to interventions
that match their criminogenic needs as identified by the PACT. In addition, the quality
assurance component allows courts to identify and implement appropriate interventions
within their jurisdictions, matching the needs of the local juvenile offender populations,
while utilizing the assessment tool to evaluate intervention effects on juvenile offender
populations over time (Vincent, 2012).
The results of a 2012 PACT query indicated a significant proportion of juvenile
offenders assessed in Snohomish County during the first six months of 2011 had room for
improvement in substance abuse related dynamic risk and protective factors (see
Appendix A). The PACT query results also revealed a substantial need for mentoring
opportunities amongst Snohomish County probation involved youth. The PAIR program
was established to meet the needs of juvenile court involved youth diagnosed with a
substance abuse problem.
The PACT assessment’s use of dynamic risk and protective factors as items of
measurement in each domain is based on the work of Hawkins, Catalano and Miller
(Barnoski, 2004). One protective factor supported by this research is youth having a
good relationship with a significant adult that models pro-social skills and encourages
engagement in positive activities (Barnoski, 2004; Hoge, 2008; Vincent, 2012). When
youth feel that there are people in their community that encourage them to stay out of
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
10
trouble and are willing to help them, this is considered a protective factor. A recent
review of the research on risk and protective factors relating to substance abuse in early
adulthood also suggests neighborhood instability may increase risk of alcohol and
marijuana use in young adults (Hawkins, 1992; Stone, 2012). Hawkins et al developed a
drug abuse prevention model suggests that when designing interventions to reduce the
negative effects of risk factors, it is important to look at the potential positive effects of
protective factors (Hawkins, 1992). Protective factors are social, behavioral and
environmental influences that can be added to a youth’s life experiences, serving as
resources to shield them from substance abuse relapse.
The inclusion of risk factors in the PACT assessment as a predictor for substance
abuse issues and criminal behavior by youth has also been validated by research of
Hawkins et al (Barnoski, 2004). Most studies show that more than half of girls and boys
involved in the juvenile justice system have a substance abuse problem (Hoge, 2008).
The results of a Snohomish County 2011 PACT query mirror Hoge’s findings with 51%
of youth placed on formal probation reporting a diagnosed substance abuse problem at
the initial PACT assessment while more than 75% indicated current drug and/or alcohol
use (See Appendix A, Items 8 and 9). These results suggest that one in four youth placed
on probation within Snohomish County during this time period could benefit from further
drug/alcohol assessment. A recent review of the literature that examines the evidence for
longitudinal predictors of substance abuse in young adults concludes that early adolescent
drug or alcohol use increases the likelihood of an individual using the same substance in
young adulthood as well as increasing the chances of other substance use (Stone, 2012).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
11
Risk factors are social, behavioral and environmental factors that contribute to potential
substance abuse problems in youth.
Although there is limited research on the impact of personal relationships between
group leaders and their mentees as a stand-alone intervention, there is evidence that high
quality interaction between leaders and youth participants within group settings can have
positive effects on pro-social adjustment (Denault, 2008). It is the closeness and
longevity of the mentor relationship that tends to contribute the most toward program
efficacy (Dubois, 2002). Study findings also suggest that successful art programs strive
to increase youth self-esteem and intentionally involve the community in which the youth
live (Farnum, 1998). As evidence continues to show that mentoring programs have
positive results, significant funding opportunities for these interventions are employed,
with 97 million dollars in federal funding for these program types in 2010 alone (Office
of Justice Programs, 2011).
Lipsey’s meta-analysis of effective programs showed that mentoring was second
only to group counseling, among the effective generic program types within the
counseling category (Lipsey, 2010). The form of meta-analysis performed by Lipsey and
Dubois is considered to be the best way to determine program effectiveness (Greenwood,
2008). Although there is a considerable amount of research, including meta-analysis on
1:1 mentoring programs, it is group mentoring similar to what is offered within the PAIR
program, that needs further study (Baldwin Grossman, 1997; Deutsch, 2009; Keating,
2002; Wilson, 2000; Rhodes, 2000; Dubois, 2002; Farnum, 1998; Jones-Brown, 1997;
Griffin, 2005).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
12
In addition to providing the PAIR participants with a positive adult role model,
the intervention seeks to increase the youth’s sense of attachment to their community
through pro-social activities (Reclaiming Futures, 2012; Butts, 2007). Again, research
suggests that pro-social community activity participation can help at-risk youth prevent
the development of substance abuse symptoms (Buu, 2009). The establishment of an
excellent mentor/child relationship within a pro-social community activity can protect
youth from risky behaviors such as drug and alcohol use (Dubois, 2002). This study
seeks to provide insight as to the effectiveness of the PAIR program in establishing
positive adult relationships within a pro-social activity for youth diagnosed chemically
dependent.
PAIR Intervention:
PAIR youth are often involved in multiple interventions before, during and
following their involvement in the workshop series, and there is evidence to suggest that
a comprehensive risk-focused approach incorporating multiple programs that address
various risk factors can be effective in preventing drug abuse (Hawkins, 1992). The
PAIR program seeks to enhance protective factors and address specific risk areas as an
adjunct, not a replacement for, Snohomish County interventions such as substance abuse
treatment, Aggression Replacement Training, Functional Family Therapy, Juvenile Drug
Court and others. It is this combination of interventions, “…promoting consistent
prevention principles across units of socialization” (Hawkins, 1992, p. 96), that have
shown positive effects on youth in relation to substance abuse prevention.
PAIR eligible youth are Snohomish County probation involved youth that have a
chemical dependency diagnosis, and youth do not have to pay to participate as funding is
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
13
provided by the Blanche Miller Trust Fund. The PAIR program consists of 7-9
consecutive workshop sessions and the workshops are offered several times throughout
the year. The juveniles meet with the workshop artist/artists, the program coordinator
and other mentors (judge, probation counselor and educator) for two hours each Tuesday
afternoon. The PAIR intervention is held in the community and away from the
courthouse complex. The pilot program finished it’s fifth workshop series in early 2013
with additional workshops planned and funded for 2013 and 2014. Thus far, youth have
engaged in PAIR workshops with the following themes; Creative Writing & Glass Art,
Mixed Media (Altered Books), Photography and Calligraphy. The community artists are
vetted and chosen by the PAIR Program Coordinator, and they come with a variety of
backgrounds and a passion for working with youth.
Purpose of the Study:
Given the above evidence derived from the PAIR program evaluation and the
PACT assessment results, this study will help answer the following research questions:
(1) Does PAIR significantly increase protective factors in juvenile justice involved youth
within Snohomish County as measured by the following PACT questions, including,
current interest and involvement in pro-social activities, number of positive adult non-
family relationships, and pro-social community ties?; (2) Does PAIR significantly
decrease risk factors in juvenile justice involved youth within Snohomish County,
including, current alcohol use and current drug use as measured by the PACT? And; (3)
Does the PAIR program provide opportunities for Snohomish County’s chemically
dependent juvenile justice population to build pro-social relationships with community
artists? Additional data will be collected on gender and the number of workshop sessions
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
14
attended by each participant, allowing for the analysis of dosage effect on the study
population.
Research question one intends to understand whether there is a relationship
between the PAIR program and participant interest and involvement in pro-social
activities, positive adult non-family relationships and pro-social community ties. The
null hypothesis for question one is, the Promising Artists in Recovery Program has no
effect on youth participant current interest and involvement in pro-social activities,
positive adult non-family relationships, and pro-social community ties. Appendix B, Part
A, contains a complete list of PACT risk needs assessment questions, and the
corresponding list of responses, used to answer question one.
Research question two intends to answer whether PAIR significantly decreases
risk factors in the juvenile justice involved youth within Snohomish County, including,
current alcohol use and current drug use. The null hypothesis for question two is, the
Promising Artists in Recovery Program has no effect on youth participant current use of
drugs and/or alcohol. Appendix B, Part B, contains a complete list of PACT risk needs
assessment questions, and the corresponding response options, used to answer question
two.
Research question three intends to answer whether the PAIR program
accomplishes its goal of providing opportunities for Snohomish County’s chemically
dependent juvenile justice population to build pro-social relationships with community
artists. The null hypothesis for question three is, the Promising Artists in Recovery
Program does not provide Snohomish County’s chemically dependent juvenile justice
involved youth with opportunities to build pro-social relationships with community
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
15
artists. The youth answers to the participant survey questions and interviews with
program stakeholders will help to answer question three. Appendix C contains a
complete list of the youth survey questions and the corresponding list of answers.
The purpose of this study is to explore the effects of the PAIR program on youth
referred to the intervention. An impact evaluation was used to determine whether the
PAIR program is having an effect on the youth participants due to it’s benefit in, “…that
it is suited to the needs of both program-level managers and policy designers, for it is
important for both to ascertain whether target populations are appropriately receiving
delivery of a program” (Theodoulou, 2004, p. 194). According to Theodoulou (2004),
the results of the policy analysis enables program deliverers to modify the intervention to
better achieve the stated goals, including identification of the following; the theoretical
and actual goals of the PAIR program, program objectives and results, as well as the
unintended, intended, positive or negative effects. Through the impact evaluation
process, PAIR program providers can improve the quality and effectiveness of the stated
goals of the intervention.
It is unlikely the research questions for this study can be best resolved by a strictly
qualitative or quantitative method alone so a mixed-method research design was
employed. Establishing causality from the application of the Wilcoxon Singed Rank
Test, McNemar’s Test and bivariate Spearman’s correlation alone would be problematic
at best, with many of the PAIR participants receiving multiple interventions while on
community supervision. Most youth receive one or more additional services while
attending the PAIR workshops. The following comprises a partial list of research-based
and best-practice programing that juvenile court involved youth participate in while on
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
16
community supervision; Case Management Assessment Process, Aggression
Replacement Training, Functional Family Therapy, Coordination of Services (Cocoon
House WayOUT Program), various substance abuse treatment therapies and specialized
education and/or job training (Snohomish County Juvenile Court, 2013; Washington
State Institute for Public Policy, 2013). As Fitzpatrick et al (2011) stated:
“Our recommendations are not for a type of method, but for choices that make
sense for the evaluation questions to be answered and the context of the study.
Typically, these are mixed methods because few questions can be answered by
only one strategy” (p. 384).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
18
Chapter 2-Review of Literature:
A useful book entitled Treating the Juvenile Offender discusses the importance of
implementing and studying the effectiveness of juvenile justice programs and
jurisdictional utilization of dynamic risk and protective factor theory (Hoge, 2008). The
book describes the importance of validated risk assessments as well as matching the
appropriate intervention to the individual youth’s criminogenic needs. The PAIR
program is utilizing Washington State’s empirically based risk assessment, an assessment
tool that was mentioned in this publication, to measure the effectiveness of the
intervention.
The Center for Juvenile Justice Reform studied the effectiveness of the
Standardized Program Evaluation Protocol (SPEP) (Lipsey, 2010). The study used meta-
analysis, “The technique for extracting and analyzing information about intervention
effects and the characteristics of the interventions producing those effects from a body of
qualifying research…” (Lipsey, 2010, p. 20). Lipsey concluded through meta-analysis
that mentoring programs are effective interventions for adolescent offender populations
like Snohomish County’s juvenile justice involved youth.
Another relevant study from the Office of Juvenile Justice and Delinquency
Prevention provides statistical information on the extent of the juvenile crime problem
and trends over time. The most recent reports and data show delinquency has decreased
over the past decade and Snohomish County’s crime rate is well below the national
average (Puzzanchera, 2012; Puzzanchera, 2013). Data for the 2012 report was retrieved
from the National Juvenile Court Data Archive and both sources include data submitted
by juvenile justice jurisdictions from across the United States.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
19
A handbook entitled, “YouthARTS Arts Programs for Youth at Risk: the
Handbook”, is a 217 page guide to developing arts programs for youth considered to be
at-risk (Farnum, 1998). The publication studied the effects of three art agency programs
on Oregon’s youth population. The researchers used an outcome evaluation that relied on
multiple data sources to measure attendance, youth benefits and changes in risk factors of
youth participants. The YouthARTS program concluded that intervention participants
made improvements in social skills and engaged in less delinquent behavior post
intervention. Another research journal article entitled, “The Effects of Cultural Arts
Programs on At-Risk Youth”, included an evaluation of five separate youth art programs
including the YouthARTS program (Rapp-Paglicci, 2007). A longitudinal study, “The
Arts and Achievement in At-Risk Youth: Findings from Four Longitudinal Studies”,
revealed promising results correlated to the positive educational achievement of at-risk
youth engaged in arts programming (Catterall, 2012). The youth art programs identified
in these studies are similar to the PAIR program in many respects.
The Robert Wood Johnson Foundation, Reclaiming Futures model program has
been studied and promising results were published in 2007. The Reclaiming Futures
program evaluation report, “Changing Systems: Outcomes from the RWJF Reclaiming
Futures Initiative on Juvenile Justice and Substance Abuse”, provides a comprehensive
overview of the model’s outcomes and significant findings from the program deliverer
perspective (Butts, 2007). The study involved ten different communities across the
United States and the findings revealed system improvements in twelve of thirteen
indices including; Pro-social Activities for Youth, Data Sharing, Treatment Effectiveness,
Assessment, Agency Collaboration, Availability of Client Information, Resource
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
20
Management, Cultural Integration, Access to Services, Targeted Treatment, Alcohol and
Other Drugs Assessment, and Family Involvement. The only area not showing a
statistically significant change was Partner Involvement and this area was rated the
highest from the beginning and consequently did not have very much room for
improvement from the outset. The Butts (2007) report provides insight into the model
responsible for the development of the PAIR program.
To provide historical perspective, “Promises and Pitfalls of Mentoring As a
Juvenile Justice Strategy” is reviewed (Jones-Brown, 1997). Written nearly fifteen years
ago, this paper discusses the promise of the Juvenile Mentoring Program (JUMP) and
C.O.R.E. mentoring programs. An OJJDP bulletin and November 2000 report outlines
the federal government’s rational for supporting mentoring programs and summarizes the
JUMP program and private public venture mentoring research (Baldwin Grossman, 1997;
Wilson, 2000). JUMP was developed to provide mentoring opportunities to at-risk youth
as a way to reduce delinquency and improve school achievement. The reports indicate
that positive relationships between young people and mentors can be developed through
structured programs. This promising approach to addressing juvenile delinquency was
hailed as a better way to deal with at-risk youth following the failed ‘get tough’ juvenile
justice policies of the 1990s.
The Office of Justice Programs fact sheet from the US Department of Justice
describes federal government sponsored mentoring work being done as of November
2011 (Office of Justice Programs, 2011). The US Department of Justice (2011) bulletin
describes mentoring as, “…crucial for individuals who are attempting to reenter the
community, especially if the person was processed as a young adult” (p. 1). An
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
21
evaluation of Big Brothers/Big Sisters, “Agents of Change: Pathways through Which
Mentoring Relationships Influence Adolescents’ Academic Adjustment”, showed
significant improvement in scholastic achievement in program youth (Rhodes, 2000).
This report adds to other research showing positive effects of mentoring programs on
youth outcomes.
The Keating mentor study showed significant changes from pre-intervention to
post-intervention on four of seven variables, where the control group studied did not
(Keating, 2002). A significant reduction in internalizing and externalizing behaviors, as
reported by teachers and mothers, were also reported. A 2009 journal article that
discusses the importance of developing a mentoring program that employs best-practices
was reviewed (Deutsch, 2009). The report describes the importance of duration,
connection, frequency of contact, consistency of contact, and mentor approach as being
important measures of the quality of a specific mentoring intervention. The study
recommended that dosage (level of exposure) be included as a measure in evaluating a
mentoring program.
Greenwood authored a journal article on the importance of utilizing proven
juvenile justice intervention programs as well as conducting research on new programs
(Greenwood, 2008). Although there are best-practices and evidence-based programs to
address juvenile delinquency, only 5% of youth considered appropriate candidates for
these interventions are given an opportunity to participate. The report emphasizes the
importance of evaluating programs prior to implementing them on a large scale so as to
avoid doing harm in addition to utilizing limited resources effectively.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
22
Griffin wrote a journal article that describes the Building Resiliency and
Vocational Excellence program (BRAVE), a substance abuse and violence prevention
program. The intervention included elements of mentoring as part of its Resiliency
Networking. The program also incorporates what it called, “The five best-practice
characteristics for mentoring” (Griffin, 2005, p. 81). The report emphasizes the
importance of addressing appropriate risk factors for the target population and utilizing a
strength-based approach.
A 2006 Corporation for National and Community Service publication provides
analysis of the Current Population Survey (CPS) to increase understanding around the
demographic, socioeconomic or other factors that show differences in those people that
volunteer to be youth mentors (Foster-Bey, 2006). This report can help stakeholders in
understanding the characteristics of those individuals that are more likely to volunteer to
be a mentor.
The Washington State Juvenile Court Assessment Manual provides the
background and origin for the PACT risk assessment (Barnoski, 2004). The risk needs
assessment is a tool to assist Washington State juvenile justice agencies in providing
appropriate interventions to local offender populations. The manual also assists
assessment tool users in implementing the evaluation tool with fidelity. The guidebook,
Risk Assessment in Juvenile Justice: A Guidebook for Implementation, offers additional
insight into risk assessments and the importance of addressing dynamic risk and
protective factors of youth engaged in the juvenile justice system (Vincent, 2012). The
guidebook also suggests that risk assessments can be used to provide stakeholders with
valuable data on intervention effectiveness.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
23
Secondary data to evaluate the need for the PAIR mentoring program was
collected in Snohomish County through a PACT query in 2011, 2012 and again in 2013.
The data from needs assessments was utilized to determine the need for this type of
program. Utilizing assessment data can, “…aid agency administrators in evaluating
resource availability throughout the jurisdiction and determining program gaps that need
to be filled” (Lipsey, 2010, p. 41). The PACT query covering initial risk assessments
from January 6, 2011 to July 6, 2011 indicated 277 youth received assessments during the
six-month period. Of the assessed youth, only 15% were involved in one or more
structured and supervised pro-social community activities, while 50% of responding
youth reported they have no positive adult non-family relationships. In addition, only 2%
(6 out of 277 youth) stated they have strong pro-social community ties. The 2011 PACT
query revealed important findings related to drug and alcohol use with Snohomish
County’s probation involved youth. The results showed 75% of respondents were
currently using drugs or alcohol at the time of the assessment. According to the PACT
data, 49% of youth reported past use of alcohol caused family conflict, while 57%
indicated drug use interfered with keeping pro-social friends. Of the 277 juvenile court
involved youth assessed, 51% were diagnosed abuse or dependent/addicted, 47% never
received a drug/alcohol assessment and only 2% of all youth were diagnosed as no
problem with drug/alcohol use. This query was an important step in identifying the need
and rationale for the conception and implementation of the PAIR program. The PACT
data revealed a significant need for Snohomish County’s juvenile justice stakeholders and
policy makers to address chemical dependency issues and the lack of pro-social activities
of probation involved youth.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
24
The work of Hawkins et al discusses the importance of addressing both risk and
protective factors within interventions designed to address drug abuse. Specifically, they
recommend interventions should be developed that take into account the empirical
evidence around risk and protective factors (Hawkins, 1992). The Hawkins et al (1992)
study indicates at-risk youth should be treated through the utilization of a variety of
interventions targeted at various, “…units of socialization” (p. 96). It is vital for juvenile
justice agencies and policy makers to focus on the implementation of carefully designed
programs addressing salient risk and protective factors of at-risk youth populations. In
addition, the work of Stone et al provides meta-analysis on risk and protective factors and
their effects on early adolescent substance use (Stone, 2012). Stone et al is a recent study
adding to the research of Hawkins et al, concluding that additional longitudinal and non-
college attending young adult studies are needed in this area. The report provides further
evidence of the influence of risk and protective factors on adolescent, as well as
subsequent adult, substance abuse disorders.
Crime statistics from the Office of Juvenile Justice and Delinquency Prevention
and the National Center for Juvenile Justice are used to measure the extent of the problem
and cover the years 1995 to 2009. These crime statistics allow researchers to consider
juvenile crime rates and juvenile justice response trends over time. Historical data that
covers incarceration, probation, parole, demographics, judicial waiver, referral source,
juvenile court case processing and offense category are explored to give a sense of the
problem facing juvenile justice practitioners. Suggestions for improving juvenile justice
program effectiveness by utilizing and evaluating best-practices in mentoring programs,
as presented by the Center for Juvenile Justice Reform are also examined (Lipsey, 2010).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
26
Chapter 3-Methodology:
Experimental Group:
The experimental group consists of Snohomish County juvenile court-involved
youth diagnosed with a substance abuse problem and included At-risk Drug Court,
Probation and Offender Drug Court youth. Some youth were referred to PAIR multiple
times since the program began in October 2011. Participants volunteer for the PAIR
program (limitations of this study include self-selection bias as a result of not utilizing
random assignment) and their assigned probation counselor or the Drug Court program
refers youth to the intervention. The experimental group included a total of 37 individual
youth referred, with 10 of those youth being referred to PAIR two or more times resulting
in 54 total youth referrals (see Table 1). The highest attended workshop had 12
participants while the smallest cohort included 7 participants. A total of 30 youth
attended the PAIR program, with 8 of the 30 youth participating in two or more
workshops during the pilot study period.
The PAIR participants are encouraged to maintain contact with the artists and
mentors in the community through texting, interactions at the courthouse and taking part
in opportunities for additional community art projects as they present themselves.
Beyond the workshop sessions, youth participants had further contact with artists and
pro-social community members through art shows and exhibits, individual commissioned
creative arts opportunities, teen art programming and classes, panel participation in a
Juvenile Justice Conference, supplemental glassblowing workshops and multiple PAIR
program participation.
Control Group:
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
27
A control group was not included in this study. The control group that was
initially identified included probation-involved youth diagnosed with a substance abuse
problem that don’t participate in PAIR, however, the data could not be used. The control
group was drawn from youth receiving an initial PACT assessment during the same time
period as the PAIR involved youth, but there were significant differences in the length of
time between assessment and re-assessment as well as differences in gender. There were
substantial differences in the PAIR population’s make up as well, with a larger proportion
of Drug Court youth in the experimental group, making the use of the control group in
any analysis and results problematic at best. Gender differences between the
experimental group and the larger Snohomish County juvenile justice population will be
discussed later on in this report.
Design and Materials:
The PAIR program evaluation included participant surveys, pre and post
intervention PACT risk assessments, stakeholder interviews and direct observation. This
study was formulated as a mixed-methods design utilizing descriptive statistics,
Wilcoxon Signed Rank Test, McNemar’s Test, bivariate Spearman’s correlations and
qualitative data. This method was used in an attempt to determine whether the PAIR
intervention significantly impacted previously mentioned youth participant risk and
protective factors. The design was also chosen because random assignment was not
possible, as a tenant of the PAIR program is to have eligible youth volunteer to
participate and a suitable comparison group could not be established. The mixed-
methods design of this study was preferred in an effort to enhance the evaluation results
and the program provider’s understanding of the PAIR intervention (Fitzpatrick, 2011).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
28
Through the combined analysis of PAIR participant surveys, PACT assessment
results and program deliverer’s perception of the interventions effectiveness, the strength
of the findings is enhanced. A mixed-methods approach is utilized to triangulate the
measures and, “…thus, increase the validity of (our) measurement of the construct”
(Fitzpatrick, 2011). The mixed-methods approach is deemed more effective when
dealing with what J. C. Green calls contemporary social inquiry and the critical issues
that comprise such studies (Greene, 2012). Critical issues cited by Greene (2012)
include:
• the complex character of human phenomena,
• the location of context in human action,
• the role of values in social inquiry, and
• the role of inquiry in society (p. 758)
Interviews were conducted with artist mentors, the PAIR Coordinator, the
Juvenile Drug Court Judge and other stakeholders to add additional insight beyond the
PACT and participant survey data. The mixed-methods approach is recommended when
conducting mentoring evaluation and Deutch (2009) recommends quality program
appraisals include; focus groups, direct observation, logs and participant check-in,
administrative records, surveys and interviews (pp. 62-63).
Measures:
Utilizing the PACT risk assessment, the dependent variables in this study are
three protective factors (all ordinal) and two risk factors (both categorical) related to
juvenile offender behaviors. Again, these dependent variables include participant interest
and involvement in pro-social structured recreational activities, number of positive adult
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
29
non-family relationships, pro-social community ties, current drug use and current alcohol
use. Independent variables include the participant’s age, gender and the number of PAIR
sessions attended.
To evaluate research question one and two a Wilcoxon Singed Rank Test and
McNemar’s Test was conducted to evaluate the impact of the intervention on PAIR
participant scores in the risk and protective factors (dependent variables) included in the
PACT. The bivariate Spearman’s correlation analysis was also used to help describe the
relationship of PAIR attendance and these identified risk and protective factors. The
PAIR program will be further evaluated using the PACT assessment results and
descriptive statistics from participant surveys. The youth’s assigned probation counselor
administered a PACT assessment to the youth participant within 90 days of the start of
the PAIR workshop (independent variable). Following the youth’s completion of the
PAIR program, the same probation counselor completed a follow-up PACT assessment
within 30 days following the final workshop session.
A participant survey was developed and administered at the conclusion of the last
two PAIR workshops. The analysis of the participant survey responses will be
descriptive for each item. Bivariate Spearman’s correlation analysis was employed to
describe the strength and direction of the relationship between youth attendance and the
youth perception of programs effects as reported in the survey. The last two questions on
the survey are related to youth participant suggestions for changes in the workshops and
art mediums that interest them. The survey questions are also used to validate the PACT
risk assessment information gathered, as well as evaluate the third research question.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
30
In order to analyze question three the Wilcoxon Signed Rank Test and
McNemar’s Test will be informative, and the post intervention youth survey can be used
to validate the PACT assessment results. The participant survey was developed because
currently the Juvenile Probation Counselor is reporting on data from question one and
two, and in order to determine if there is a more parsimonious relationship, the participant
survey was created to solicit youth responses to these questions as well. The survey was
developed in mid 2012 and administered to the two final PAIR workshop cohorts.
Observations can also tell us whether specific activities occur that may enhance
the effectiveness of the intervention. To gain additional insight beyond the Wilcoxon
Signed Rank Test, McNemar’s Test and bivariate Spearman’s correlation analysis,
additional descriptive statistical examination of the PAIR program was also conducted.
As mentioned earlier, the program does not allow for random assignment, as a tenant of
the workshops is to have youth ‘volunteer’ to attend the intervention leading to self-
selection bias, a limitation that will be noted in the findings. The study design collected
PACT data from the entire participant population, as the program group is small (38
individual youth referred to one or more workshops from October 2011 to January 2013).
The surveys were distributed to 19 participants at the conclusion of the last two PAIR
programs (see Appendix C for a sample survey questionnaire and a complete list of the
survey questions and response categories). The evaluator conducted interviews with
youth, program deliverers and probation counselors to elicit stakeholder interpretation
and exploration of program effectiveness. This additional qualitative information
provides the context for consideration of the intervention and its outcomes from the
stakeholder perspective.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
31
From the participant survey, the dependent variables in this study are ten
measures related to the youth’s motivation to abstain from drug/alcohol use, youth
feelings toward the PAIR program and mentors, and youth feelings toward their
community. These dependent variables include participant perception of the PAIR
Coordinator’s enthusiasm, PAIR artist enthusiasm, youth connection to their community,
number of positive encouraging adults, interest and involvement in pro-social community
activities, perceived support from the community, attachment to the community in which
they live, and motivation to abstain from drug use and motivation to abstain from alcohol
use (see Appendix C for a sample survey and complete list of questions and response
categories). Independent variables include the participant age at the time of PAIR
orientation, gender and the number of PAIR sessions attended.
Key stakeholders will be involved in the interpretation of the data to facilitate the
practical use of results while also adding validity to the conclusions. The initial purpose
of the evaluation will be formative rather than summative, with the evaluation resulting in
program improvement and promoting new ways of thinking through the evaluation
process. Beyond this study, recidivism data may be extracted using the Washington State
Juvenile Correction Database custom BOXI reports providing re-offense rates of program
youth.
The study took place in Snohomish County, Washington and utilized the Positive
Achievement Change Tool (PACT), an Assessments.com risk needs assessment to
measure changes in risk and protective factors. Additional measures included post
intervention participant surveys, direct observation and stakeholder interviews. Survey
measures included youth perception of positive adults in their lives, pro-social
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
32
community activities, positive attachment to their community, and motivation to abstain
from drug and alcohol use. The Wilcoxon Signed Rank Test will tell us whether or not
there were significant differences in the youth protective factors following the PAIR
intervention. The McNemar’s Test will tell us whether or not there were significant
differences in youth participant drug or alcohol use following the PAIR intervention.
Surveys in conjunction with direct observation and stakeholder interviews will also tell us
whether or not youth participants developed positive relationships with program
providers.
Data Collection Procedures:
Post-participant surveys were administered to the last two cohorts of youth
following their completion and within 30 days of the final PAIR workshop (n=19).
Careful consideration of the sequencing of the survey questions was made, and questions
are clear and worded correctly while also being structured to elicit cooperation (Fink,
2009). Special attention was also directed at the wording and order of the survey
questions in an effort to eliminate bias and promote respondent participation.
Development of a unique twelve question survey was completed, as existing surveys did
not address this specific program evaluation. The first ten questions relate to the PACT
assessment risk and protective factors and were developed using a five point Likert-scale.
The final two survey questions were tailored in a way to elicit participant responses and
determine whether program changes need to be made, as well as gather feedback on the
perception of what the best elements of the program are.
The youth survey questions are also used to validate the PACT assessment
information gathered. The post intervention survey was distributed, explained and then
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
33
collected from the youth participants by this researcher. The youth participant’s Juvenile
Probation Counselor conducted pre and post intervention PACT assessments (n=47).
The initial PACT assessment interview was completed within 90 days of the intervention
start date and the data was then entered into the Assessments.com data warehouse. No
more than 30 days following the final PAIR workshop, the Juvenile Probation Counselor
completed another PACT assessment with the youth and entered the data into
Assessments.com.
The author of this report entered the pre and post intervention PACT data and the
post intervention survey data into Microsoft Excel. The random assignment of
identification numbers to youth participants was completed in Excel and the data was
cleaned prior to being transferred to the Statistical Package for the Social Sciences
(SPSS) by the author of this report. Within SPSS the variables were then categorized for
analysis.
The author of this study also conducted informal interviews with key stakeholders
(i.e. youth participants, probation counselors, artists, PAIR Coordinator, and a Superior
Court Judge). Direct observation of the PAIR intervention has been ongoing over the
past 18 months and information has been recorded in a chronological log. Recorded
interviews were conducted with PAIR artists, a Superior Court Judge and the PAIR
Coordinator to solicit feedback from the program deliverers/stakeholders. To further
validate study findings, the chronological log and recorded stakeholder interviews were
reviewed to determine the perceived influence of the intervention on dynamic risk and
protective factors.
Analysis Strategy:
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
34
Statistical Package for the Social Sciences (SPSS) was used to analyze the
relationship between the study variables. Again, analysis includes five measures of
PACT risk and protective factors and ten measures of participant perception of the PAIR
program. Using appropriate frequencies, I computed descriptive statistics on the
population as calculated percentages of survey responses, changes in risk/protective
factor scores by gender and PAIR sessions attended.
Interviews with the PAIR artists, PAIR Coordinator and Juvenile Drug Court
Judge all reported their experiences with the intervention in recorded interviews. The
program deliverers were interviewed to provide additional qualitative information, adding
to the PACT assessment and post-PAIR survey analysis.
For research questions one and two, bivariate correlation using Spearman’s Rank
Order Correlation was used to determine whether there was a relationship with PAIR
program participation and PACT risk and protective factors. It was expected that
protective factor scores would be strengthened as youth attendance in the PAIR program
increased, and it was expected that risk factors would decrease as youth attendance in the
intervention increased.
A Wilcoxon Signed Rank Test was used to measure median PACT protective
factor scores of PAIR participants on two different occasions (pre and post PAIR
intervention). The McNemar’s Test was used to compare the proportion of youth who
reported drug use or alcohol use prior to and following the PAIR program. Analysis
includes assessment of each participant’s risk and protective factors at time one (within
90 days of the start of the intervention) and time two (within 30 days of the final PAIR
session) by the youth’s assigned probation counselor. A Wilcoxon Signed Rank Test was
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
35
used to evaluate the impact of the PAIR program on participant interest and involvement
in pro-social structured recreational activities, positive adult non-family relationships,
pro-social community ties, while the McNemar’s Test was used to evaluate the impact of
the program on current drug use and current alcohol use.
For research question three, bivariate correlation using Spearman’s Rank Order
Correlation was used to determine whether there was a relationship with PAIR program
participation and post intervention participant feelings toward their community,
perception of artist and coordinator enthusiasm, interest and involvement in positive
activities, and motivation to abstain from drug and alcohol use. Analysis of the survey
responses, PACT scores and stakeholder observations are also interpreted collectively so
as to validate the results through triangulation.
Ethical Considerations:
This project included the study of human subjects and the protection of their
rights was considered paramount. The PAIR program was funded by a grant from the
Blanche Miller Trust and an internal review was conducted with Snohomish County
management, court administration, program deliverers and representatives from the
Miller Trust Board prior to the study taking place. Care was taken to ensure that the
confidential information and anonymity of individual youth was maintained during the
data collection of assessments, surveys and interviews. Confidentiality and anonymity
was preserved, by assigning random I.D. numbers to each participant prior to the data
entry into SPSS and subsequent analysis. Care was taken so the identity of the people
providing the information will not be linked to any of the individual data included in the
study nor disclosed to others. The subject’s participation in the study was voluntary and
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
36
program participants were not required to participate in the research activities as a caveat
to PAIR program involvement.
Informed consent was obtained from program deliverers (Judge, PAIR
Coordinator and artists) participating in the case study recorded interviews and data
collection, as the anonymity of the personal interview data and the quotes included in the
study could not be ensured. The University of Washington IRB application was
considered and a waiver was granted.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
38
Chapter 4-Results and Discussion:
There were a total of 47 pre intervention and post intervention PACT assessments
completed by the participant’s Juvenile Probation Counselor. Youth attended an average
of 4.19 PAIR sessions per workshop over the entire study. There were a total of 19 youth
that participated in the post PAIR survey, with these youth attending an average of 6.11
sessions per workshop over the final two workshops.
Demographics:
This study draws on merged data from youth referred to the PAIR intervention.
The participant ages varied from 14 to 19 years old (see Table 2). A total of 37
individuals were referred to the study, but with many youth being referred to the program
multiple times there were 35 females and 19 males referred over time (see Table 3).
There was a significant difference in the gender of the study population as
compared to the general juvenile offender population of Snohomish County. Among all
respondents in the study group, 65% of referred youth were female and 35% were male
(see Figure 1). In contrast, a PACT query revealed that, among 671 offender youth
assessed between September 2011 and January 2013, 26% were females and 74% were
males (see Figure 2). The mean age of the participants referred to the program was 16
years old; matching the average age of PACT assessed youth in Snohomish County (see
Table 4).
Research Question 1:
Does PAIR significantly increase protective factors in juvenile justice involved
youth within Snohomish County as measured by the following PACT questions,
including, current interest and involvement in pro-social activities, number of positive
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
39
adult non-family relationships, and pro-social community ties? Results of the Wilcoxon
Signed Rank Test analysis of PAIR participant PACT scores in pro-social activities,
number of positive non-family adults and pro-social community ties are presented in
Tables 5, 6 and 7. The Wilcoxon Signed Rank Test was conducted to evaluate the impact
of the PAIR intervention on participant protective factor scores as measured by the
PACT. The test revealed a statistically significant increase in youth interest and
involvement in pro-social activities, z = -3.407, p < .001, with a medium effect size (r =
.35) using Cohen (1988). The median PACT score increased from pre-PAIR (Md = 1) to
post-PAIR (Md = 2). The results indicate that youth interest and involvement in pro-
social activities was significantly higher following participation in the PAIR program.
The Wilcoxon Signed Rank Test also revealed a statistically significant increase
in the number of positive adult relationships following youth participation in the PAIR
program, z = -3.346, p < .001, with a medium effect size (r = .35) using Cohen (1988).
The median PACT score increased from pre-PAIR (Md = 1) to post-PAIR (Md = 2). The
results indicate that the number of pro-social adults was significantly higher following
youth involvement in the PAIR program.
The Wilcoxon Signed Rank Test revealed no significant difference in youth pro-
social community ties, z = -1.667, p < .096. The median PACT score remained (Md = 1)
for youth at both pre and post-PAIR. The results indicate no significant differences in the
participant’s pro-social community ties.
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention PACT scores on youth involvement in pro-social
activities, number of positive non-family adults, and pro-social community ties are
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
40
presented in Table 8. There is a weak positive correlation in the data set between the
PAIR sessions attended and youth interest and involvement in pro-social activities as
measured by the final PACT assessment (Spearman’s r = .223, n = 47, p < .13), however
the results did not reach significance at the p < .05 level. For number of PAIR sessions
attended and the number of pro-social adult non-family relationships following the PAIR
program, there is a moderate positive but significant correlation in the data set
(Spearman’s r = .305, n = 47, p < .04). Finally, there was a strong but significant positive
correlation in the number of PAIR sessions attended and participant pro-social
community ties (Spearman’s r = .546, n = 47, p < .01). Therefore, we can report strong
positive correlations in perceived pro-social community ties associated with increases in
attendance in the PAIR program. We can also report moderate positive correlations in
the number of pro-social adult relationships with increases in youth attendance in the
PAIR program. There was not sufficient evidence to support a correlation between PAIR
attendance and pro-social activity involvement among the study population.
Research Question 2:
Does PAIR significantly decrease risk factors in juvenile justice involved youth
within Snohomish County, including, current alcohol use and current drug use as
measured by the PACT? The McNemar’s Test (see Table 9) conducted on participant
alcohol use and revealed no significant change in participant alcohol use from time one to
time two, as measured by the PACT, (p < 1.000). This suggests there was no significant
change in the proportion of youth using alcohol following PAIR (34.0%) when compared
with the proportion of participants using alcohol before PAIR (31.9%).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
41
The McNemar’s Test (see Table 10) was conducted on current drug use and
revealed no significant change in participant drug use from time one to time two, as
measured by the PACT, (p < .454). This suggests there was no significant change in the
proportion of participants using drugs following PAIR (36.2%) when compared with the
proportion of youth using drugs before PAIR (44.7%).
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention PACT scores on current drug use and current alcohol
use are presented in Table 11. There was no correlation in the data set between the
number of PAIR sessions attended by the respondent and the respondent’s drug use as
measured by the PACT assessment (Spearman’s r = -.03, n = 47, p < .83), and the results
did not reach significance at the p < .05 level. Also, there was a weak negative
correlation in the data set between the PAIR sessions attended by the respondent and the
respondent’s alcohol use (Spearman’s r = -.13, n = 47, p < .40), however the results did
not reach significance at the p < .05 level. Therefore, we can report there is not enough
evidence to determine whether or not participant attendance in PAIR workshop sessions
is associated with changes in drug or alcohol use as measured by the PACT assessment.
Research Question 3:
Does the PAIR program provide opportunities for Snohomish County’s
chemically dependent juvenile justice population to build pro-social relationships with
community artists? Results of the bivariate Spearman’s correlation analysis of PAIR
participant attendance and post intervention survey results on youth perception of the
PAIR Coordinator enthusiasm and PAIR artist enthusiasm are presented in Table 12.
There was a strong but significant positive correlation in the data set between increased
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
42
PAIR sessions attended and the respondent’s perception of the PAIR Coordinator’s
enthusiasm (Spearman’s r = .67, n = 19, p < .01). Also, there was a weak positive
correlation in the data set between the number of PAIR sessions attended by the
respondent and the respondent’s perception of the PAIR artist’s enthusiasm (Spearman’s
r = .18, n = 19, p < .45), however the results did not reach significance at the p < .05
level. Therefore, there was a strong positive correlation in PAIR attendance and
participant perception of the PAIR Coordinator’s enthusiasm. There was not enough
evidence to determine a correlation between PAIR attendance and the respondent’s
perception of the artist’s enthusiasm, although all surveyed youth reported the artists
were ‘very’ or ‘extremely’ enthusiastic.
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention survey results of participant feelings of more
encouraging adults and youth interest in community activities are also presented in Table
12. There was a moderate but significant positive correlation in the data set between
PAIR sessions attended by the respondent and the respondent’s post PAIR feelings of
more encouraging adults (Spearman’s r = .47, n = 19, p < .04). Also, there was a strong
but positive correlation in the data set between PAIR sessions attended and the
respondent’s interest in community activities (Spearman’s r = .59, n = 19, p < .01).
Therefore, we can report a strong positive correlation with participant attendance and
youth interest in community activities, and there was also a moderate positive correlation
with increased attendance and youth feeling they have more encouraging adults in their
life.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
43
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention survey results on participant involvement in positive
community activities and participant feelings about the community in which they live are
presented in Table 13. There was a moderate positive correlation in the data set between
PAIR sessions attended and the respondent’s involvement in community activities
(Spearman’s r = .43, n = 19, p < .08), however the results did not reach significance at the
p < .05 level. Also, there was no correlation in the data set between PAIR sessions
attended and respondent feelings about the community they live in (Spearman’s r = .08, n
= 19, p < .74), however the results did not reach significance at the p < .05 level.
Therefore, we can report that there is not enough evidence to determine a correlation
between PAIR attendance and youth feelings about the community they live in. With
PAIR attendance and youth involvement in positive activities, although there was not
enough evidence to determine correlation, the results were very close to being significant
at the p < .05 level.
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention survey results on participant perception of community
support and feelings of connection to community are presented in Table 13. There was a
weak positive correlation in the data set between PAIR sessions attended and the
respondent’s perception of community support (Spearman’s r = .14, n = 19, p < .57),
however the results did not reach significance at the p < .05 level. Finally there was a
strong positive correlation in the data set that did reach significance between PAIR
sessions attended and the respondent’s feelings of connection to community (Spearman’s
r = .55, n = 19, p < .02). Therefore, we can report that there is a strong positive
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
44
correlation in participant feelings of connection to community and attendance. There was
not enough evidence to determine a correlation between PAIR attendance and participant
perception of community support.
Results of the bivariate Spearman’s correlation analysis of PAIR participant
attendance and post intervention survey results on participant motivation to abstain from
drug use and alcohol use are presented in Table 14. There was a moderate positive
correlation in the data set that did reach significance between sessions attended and
participant motivation to abstain from drug use (Spearman’s r = .45, n = 19, p < .05).
There was a moderate positive correlation in the data set that did reach significance
between sessions attended and motivation to abstain from alcohol use (Spearman’s r =
.50, n = 19, p < .03). Therefore, we can report that there is a moderate positive
correlation with youth participation in PAIR workshops and participant motivation to
abstain from both drug use and alcohol use.
Interviews conducted with PAIR artists, a Superior Court Judge and the PAIR
Coordinator also provided anecdotal evidence that the intervention was impacting the
dynamic risk and protective factors of the study youth. Artists were asked about their
observations regarding the program and participants. One artist reported she was
deliberate about establishing trust with the youth in the workshops by following through
on commitments and communicating often with participants, stating, “You could see
them bonding together, and more trust was developed with each other and with us”
(recorded interview, December 6, 2012). Another artist also indicated the program
increases the chance youth will make a connection with someone outside the courthouse,
reporting, “If youth don’t get this extra bump, they won’t be as likely to have the
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
45
opportunity to make meaningful connections in the community” (recorded interview,
December 6, 2012). Another artist also reported that trust was intentionally developed
with the youth and she tried to get participants success in small increments in order to
build their confidence (recorded interview, December 6, 2012). One Superior Court
Judge was interviewed and he felt the artist and Program Coordinator made a difference
for many of the youth coming from Drug Court by helping the participants connect with
an adult and establishing a trusting relationship, indicating, “A lot of (youth) have
traumatic histories with adults. The more positive connections they have with adults the
less they fear them, (and) the more they trust them” (recorded interview, December 24,
2012). The establishment of a trusting relationship, as an important positive and
deliberate proviso, was emphasized by all of the program deliverers that agreed to
recorded interviews and this assertion is supported by research. In the Garringer (2007)
mentoring guide, trust is emphasized and it is recommended mentors, “Take the time and
effort necessary for your mentee to develop trust in you. While you know that your
mentee should trust you, the reality is that you have to earn the trust” (p. 17).
The PAIR artists reported they saw pro-social community ties developed with
youth participants during the program. Another program deliverer re-counted how she
spoke with community members who commented on the articles in the local newspaper,
stating, “They were amazed at the artwork produced by PAIR youth” (recorded
interview, December 19 2012). One artist saw connections going on between kids and
community members viewing the artwork. He felt the display of art in the local
newspaper, public library, art shows and public spaces was a way to validate youth’s
work and complete, “…the circle that this is their community” (recorded interview,
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
46
December 6, 2012). One of the stakeholders indicated the biggest surprise was,
“Realizing how many of these kids have artistic abilities, (and) how talented they are”
(Recorded Interview, December 24, 2012). These observations suggest additional
research might be warranted, with a future study focusing on how PAIR youth’s artwork
and their affirming life stories might impact the community and/or the community
perception of how to best deal with juvenile delinquency.
Program deliverers reported additional activities youth became connected with
beyond PAIR. A conversation with a program deliverer, revealed added opportunities for
PAIR youth, such as a local government commissioned art projects, a community park art
project and one youth was asked to create a piece for a local charity auction (recorded
interview, December 19, 2012). According to one artist, “…pro-social activities were
encouraged and youth were referred to free art classes, college courses in the community
and many participants were connected to glassblowing opportunities beyond the
workshop” (recorded interview, December 6, 2012). The artists encouraged youth to
participate in pro-social activities, but one artist reported the PAIR program could do
better at finding opportunities to connect kids to other resources like local college courses
(recorded interview, December 6, 2012). He went on to say additional conversations
about how to improve the program can help deliverers and artists expand their scope in
what and who the kids can be connected to. One adult stakeholder observed youth
become more open to other positive activities in the community and he saw participants,
“…had positive connections with each other, (and) grow more motivated to re-engage
with activities they had fun and success with prior to their addiction” (recorded interview,
December 24, 2012).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
47
Recorded interviews with the stakeholders revealed anecdotal evidence that youth
motivation to abstain from drug and alcohol use was increasing as interest and
involvement in the PAIR program developed. When asked if motivation to abstain from
using drugs or alcohol increased, one artist indicated she thought PAIR helped youth stay
motivated, but it was only a part in a bigger puzzle. She reported some youth did relapse
during the program, but the artists promoted sobriety while other programs like Drug
Court, support meetings, and formal substance abuse treatment provided relapse
assistance to youth (recorded interview, December 6, 2012). An adult stakeholder
described an observed cognitive dissonance take place when one youth realized her
behavior was having a negative effect on her siblings after feeling the positive influence
of PAIR artists and mentors. The youth’s desire to be a ‘positive role model’ for her
siblings further motivating her to commit to positive behavior change (recorded
interview, December 24, 2012). A program deliverer, saw youth add sober support
meetings to their resource list, as youth would network and invite one another to attend
sober support meetings, further expanding support networks (recorded interview,
December 19, 2012). She went on to say, “It felt like they were being decisive about
staying clean”, and participants encouraged each other to engage in PAIR and not use
drugs or alcohol.
Direct observation of PAIR workshops and informal discussion with program
deliverers revealed additional anecdotal information regarding program outcomes. The
program deliverers, artists and the other direct service providers (probation counselors,
Judge, educators) supported youth participants beyond PAIR by staying available to
youth well after the PAIR program and Drug Court ended. Many of the youth continued
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
48
to be involved in additional PAIR workshops and supplementary glassblowing
opportunities beyond their initial involvement and completion of the PAIR workshop.
Many youth were very appreciative of the opportunity to have their work
displayed publicly, and one youth commented that he hoped the art might inspire others
to overcome their past. During the workshops youth would support one another with
positive affirmations of the artwork being created. The artists were consistently
purposeful in establishing a positive atmosphere, encouraging youth with praise, and
demonstrating patience. There were instances where youth would be struggling with
court compliance while participation in PAIR continued, allowing the workshop
atmosphere to serve as a safe place to troubleshoot problem issues. The encouraging
milieu seemed to give some youth more confidence and motivation to stay engaged in the
program even when they were struggling in other aspects of their lives. The stakeholders
were also deliberate about cultivating youth efficacy, saying, “These kids can do anything
and I try to help them to see that possibility, (and) it’s a struggle for them to believe they
can succeed, but they really can” (recorded interview, December 6, 2012). Another
program stakeholder also commented, “Giving youth the PAIR opportunity got them
thinking, ‘if I can do this, then I can have fun when I’m not high’, and it opens them up to
doing more opportunities”, lending evidence to support the claim that youth motivation to
participate in pro-social activities beyond PAIR increased alongside surges in self-
efficacy (recorded interview, December 24, 2012). Interim updates on program outcomes
and participant survey aggregate results were provided to program deliverers, motivating
them to continue to provide PAIR workshops the youth found interesting and to make
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
49
program improvements based on youth suggestions. Youth were also given education
credit hours by some schools for participating in the PAIR program.
Discussion:
As mentioned earlier in this report, prior research has shown that youth
involvement in structured pro-social activities is a protective factor against current and
future substance abuse, and this study provides evidence youth attendance in PAIR is
correlated to youth interest and involvement in structured pro-social activities (see Figure
3). Youth participated in additional pro-social activities beyond PAIR including, art
shows and exhibits, individual commissioned creative arts opportunities, teen art
programming and classes, panel participation in a Juvenile Justice Conference, multiple
PAIR program participation (8 youth participated in two or more workshops), and
supplemental glassblowing workshops where twelve youth participated in one or more
glass workshops. This study also provides evidence youth participation in the PAIR
program is correlated to an increase in the number of non-family pro-social adults in the
participant’s lives, a research validated protective factor (see Figure 4). Consequently,
the preceding results have implications for stakeholders as they consider continued
support and the dedication of limited resources to this intervention.
This study provides evidence female juvenile-justice involved youth were more
likely to participate in the PAIR program when compared to their male counterparts.
While the majority of participants were female, the survey results indicate 100% of male
respondents felt ‘much more motivated’ to abstain from drug and alcohol use compared
to 71% of the female respondents following the intervention (see Figure 5). These results
have implications for program deliverers as they move forward with referring youth to
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
50
the program and, as of this writing, stakeholders are engaged in the process of
establishing specific art mediums that would encourage more male youth to participate.
Program deliverers reported many of the male participants have expressed interest in
graffiti art and more hands on activities like metal and wood sculpture. An ‘activities
survey’ is also currently being utilized throughout the Snohomish County Juvenile Court
to gauge youth interest in a variety of community activities, and by grouping the data by
gender the program deliverers might better serve the entire population through careful
consideration of the results as they plan for future PAIR workshops.
Of the 19 youth that completed the post PAIR survey, 84% of youth felt the PAIR
Coordinator was very or extremely enthusiastic while 100% of respondents felt the artists
were very or extremely enthusiastic (see Figure 6). Results show 84% of surveyed youth
felt there were more encouraging adults in their lives while the same percentage also had
more interest in community activities (see Figure 7). Results show 79% of youth were
more involved in positive activities, liked the community more, and perceived more
community support at the conclusion of the PAIR program (see Figure 8 and 9). With
respect to respondent motivation to abstain from drug and alcohol use, the survey results
indicated 79% of youth were more motivated to not use drugs and not use alcohol post
intervention (see Figure 10).
This study only takes into account gender, pro-social community attachment, the
strength and number of pro-social adults in the youth’s life, pro-social community
activity interest and involvement, drug use and alcohol use. Additional demographic
information including race, ethnicity, income, etc. could prove to be critical variables that
may or may not be correlated to these measures of participant risk and protective factors.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
52
Chapter 5-Conclusion:
Expected Findings:
The PAIR program evaluation provides stakeholders with findings and insight
into the effectiveness of this pilot intervention. The study provides evidence youth
engagement in PAIR increases protective factors, including, interest and involvement in
pro-social activities and the number of positive adults in participant’s lives. There is not
enough evidence to support a claim PAIR impacts pro-social community ties with
program youth, although there was a strong positive correlation with youth’s increased
attendance and youth reporting a positive connection to community.
The program was specifically designed to serve youth diagnosed with a substance
abuse problem. Current drug and alcohol use was studied and there was insufficient
evidence to support a claim the PAIR intervention impacted participant drug or alcohol
use. There was however a moderate positive correlation with increased PAIR attendance
and youth motivation to abstain from drug or alcohol use, suggesting that, as youth
engagement and participation was strengthened, motivation to abstain from drug and
alcohol use increased. The long-term effect of this increased motivation in PAIR youth
merits further study, as it is important to increase motivation when addressing substance
abuse issues with youth. Research suggests motivation is dynamic, and it can be
modified and influenced by pro-social interactions (Miller, 1999; Prochaska, 1995).
Miller, (1999) describes how motivation to abstain from drug and alcohol use is
influenced by social interactions and community support, and these external factors help
to provide the ‘conditions of change’ (p. 3).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
53
The gender of youth being referred to the PAIR program was significantly
different from the overall juvenile offender population and program deliverers should
reflect on the disparity found in this study. Stakeholders should continue to evaluate
whether possible changes need to be made in the program and actively consider new
ways to engage male juvenile court involved youth.
Making a significant impact on the number of positive adults and pro-social
activities with program youth was the primary goal of the PAIR program from inception.
Born out of the Reclaiming Futures model to integrate service coordination and
intervention plans that include involvement in pro-social activities and ‘natural helpers’,
the PAIR program is achieving promising results. The program deliverers have
embraced their role as being ‘part of a bigger puzzle’ that includes court involvement and
formal drug and alcohol treatment. The full impact of the PAIR program on youth
participant substance abuse issues, within this coordination of care, deserves further
research.
The PAIR program is currently funded by the Blanche Miller Trust Fund on a
year-to-year basis. Providing quality and sustainable programing requires stakeholders to
continue to collect data and evaluate program effectiveness. Through this evaluation
process policy makers might pursue diverse funding opportunities, such as the
Snohomish County .01 percent sales tax revenue for funding of chemical dependency
specific services, and Washington State juvenile justice funding requiring interventions
be research-based, evidence-based or promising programs with rigorous research criteria
(WSIPP, 2010).
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
54
This study is one of the first pilot program evaluations conducted in Snohomish
County using PACT assessment risk and protective factor data from inception to
evaluation. The specific risk and protective factors were selected as PAIR was being
developed, assisting stakeholders in delivering the program with purposeful intent. This
purposeful intent appears to have encouraged professionals to incorporate specific
protective factor elements into the program. For example, when changes to PAIR were
considered, the emphases remained on how the modifications would positively or
adversely influence the program goals to increase participant pro-social engagement with
adults, positive connection to community and grow pro-social activity involvement. As
Vincent (2012) describes how policy makers might use risk assessments in decision
making and outcomes; the PACT assessment results can assist PAIR deliverers and
juvenile court personnel to ‘improve practice’ and, “Staff morale also could be enhanced
by awareness that the agency’s goals are being met (as demonstrated through data that
came out of their direct efforts)” (p. 87).
Limitations and Future Work:
The future work of policy makers should include the establishment of a separate
research partner, not connected to the PAIR program or the Juvenile Court. The research
partner, possibly from academia, another governmental organization or a research firm,
would develop the tools to conduct a rigorous program evaluation. The author of this
report is an employee of the Court, while the PAIR Coordinator is a pseudo-employee of
the Court, and both individuals have worked together since the creation and
implementation of the intervention. Consequently, the perception of bias within this
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
55
study is a concern, as these evaluators appear to be assessing the quality of their own
work.
Although the salary of the author of this report is not directly tied to the
achievement or perceived success of the PAIR program evaluation results, the significant
time, energy, investment in development, involvement with implementation and
maintenance of the program creates an impression of bias, as “...the potential for
organizational pressure is greater when the evaluator is employed by the organization
whose program is being evaluated than when the evaluator is employed by an outside
agency” (Fitzpatrick, 2011, p. 102). A completely separate research partner could be
contracted with to create an independent assessment tool, conduct data analysis, author
reports and document research findings at ‘arms length’ from the Court and PAIR
Coordinator, mitigating any real or perceived conflict of interest. Funding opportunities
exist for this type of research and, as an example, the Office of Juvenile Justice and
Delinquency Prevention (OJJDP) has grants available for ‘Mentoring Best Practices
Research’ (U.S. Department of Justice, 2012).
This evaluation focuses on the impact of a specific set of risk and protective
factors and excludes the possible impact of other components of the program on a
number of empirically based risk and protective factors. A review of the literature on art
programming effects on at-risk youth indicates that involvement in the arts, whether in
school or as an extra curricular activity, is correlated to higher academic achievement and
civic engagement, reduction in delinquency and increased pro-social peer relationships
(Catterall, 2012; Rapp-Paglicci, 2007; Farnum, 1998). Although the effects reported in
these other studies were beyond the scope of this analysis, it would be worthwhile to
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
56
examine possible correlations between youth involvement in the PAIR program and
increases in protective factors or decreases in risk factors related to education,
employment, current living situation, social skills, pro-social peer relationships,
aggression and others.
With respect to program deliverer interpretations of program effects and
observations, response bias is also a concern. One might argue these professional
providers could be casting the program in a positive light as a result of substantial
interactions and investment in the program. The study may also be questioned due to the
Hawthorne effect, as youth and program deliverers were aware they were participating in
a study, affecting the validity of survey and interview responses (Fink, 2009). These
considerations need to be taken into account and provide evidence there is a need for
future research on the PAIR program. Additional insight can also be gained through
further analysis of recidivism rates over time, race and ethnicity, and finding a suitable
control group for comparison.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
58
APPENDICES
References:
Assessments.com. (2013). Assessments.com. Retrieved April 5, 2013, from
Assessments.com: https://www.assessments.com/purchase/detail.asp?SKU=5197
Baldwin Grossman, J. & Garry, E. (1997, April). Office of Juvneile Justice and
Delinquency Prevention. Retrieved from www.ojjdp.gov:
https://www.ncjrs.gov/pdffiles/164834.pdf
Barnoski, R. (2009, December). Providing Evidence-Based Programs With Fidelity in
Washington State Juvenile Courts: Cost Analysis. Retrieved April 5, 2013, from
Washington State Institute for Public Policy:
http://www.wsipp.wa.gov/rptfiles/09-12-1201.pdf
Barnoski, R. (2004, March). Washington State Juvenile Court Assessment Manual,
Version 2.1. 3-4, 85. Olympia, Washington, U.S.: Washington State Juvenile
Court Administrators Association.
Butts, J. & Roman, J. (2007). Changing Systems: Outcomes from the RWJF Reclaiming
Futures Initiative on Juvenile Justice and Substance Abuse. Portland State
University. Portland: Reclaiming Futures National Program Office.
Buu, A., DiPiazza, C., Wand, J., Puttler, L., Fitzgerald H., & Zucker R. (2009). Parent,
Family, and Neighborhood Effects on the Development of Child Substance Use
and Other Psychopathology From Preschool to the Start of Adulthood. University
of Michigan, Department of Psychiatry and Addiction Research Center. Ann
Arber: National Institute on Alcohol Abuse and Alcoholism.
Catterall, J., Dumais, S., & Hamden-‐Thompson, G. (2012). The Arts and Achievement in
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
59
At-‐Risk Youth: Findings from Four Longitudinal Studies. National Endowment
for the Arts. Washington DC: Office of Research and Analysis.
Denault, A., & Poulin, F. (2008). Associations Between Interpersonal Relationships in
Organized Leisure Activities and Youth Adjustment. The Journal of Early
Adolescence, 28 (477), 498-499.
Deutsch, N. & Spencer, R. (2009). Capturing the Magic: Assessing the Quality of Youth
Mentoring Relationships. New Directions for Youth Development (121), 47-70.
Dubois, D., Holloway, B., Valentine, J., & Cooper H. (2002). Effectiveness of
Mentoring Programs for Youth: A Meta-Analytic Review. American Journal of
Coummunity Psychology , 30 (2), 157-197.
Farnum, M., & Schaffer, R. (1998). YouthARTS. (K. Carlson, Editor) Retrieved
September 12, 2012, from Americansforthearts.org:
http//www.americansforthearts.org/youtharts/pdf/youtharts.pdf
Fink, A. (2009). How to Conduct Surveys: A Step-‐by-‐step Guide 4th Edition.
Washington D.C.: SAGE Publications, Inc.
Fitzpatrick, J., Sanders, J., & Blaine, W. (2011). Program Evaluation: Alternative
Approaches and Practical Guidelines 4th Edition. Upper Saddle River, New
Jersey, U.S.: Pearson Education.
Foster-Bey, J., Dietz, N., & Grimm, R. (2006, May). Corporation for National &
Community Service. Retrieved September 14, 2012, from nationalservice.gov:
http://www.nationalservice.gov/pdf/06_0503_mentoring_report.pdf
Garringer, M., & Jucovy, L. (2007, September). National Criminal Justice Reference
Service. Retrieved September 13, 2012, from www.ncjrs:
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
60
http://gwired.gwu.edu/hamfish/merlin-
cgi/p/downloadfile/d/20699/n/off/other/1/name/foundationspdf/
Greene, J. (2012). Engaging Critical Issues in Social Inquiry by Mixed Methods.
American Behavioral Scientist , 56 (755), 755-774.
Greenwood, P. (2008). Prevention and Intervention Programs for Juvenile Offenders. The
Future of Children , 18 (2), 185-210.
Griffin, J. (2005). The Building Resiliency and Vocational Excellence (BRAVE)
Program: A Violence-Prevention and Role Model Program for Young, African
American Males. Journal of Health Care for Poor and Underserved , 16 (4), 78-
88.
Hawkins, D., Catalono, R., & Miller, J. (1992). Risk and Protective Factors for Alcohol
and Other Drug Problems in Adolescence and Early Adulthood: Implications for
Substance Abuse Prevention. Psychological Bulletin , 112 (1), 64-105.
Hefley, D. (2010, October 21). HeraldNet. Retrieved November 3, 2012, from
Heraldnet.com: http://www.heraldnet.com/article/20101021/NEWS01/710219870
Hoge, R., Guerra, N., & Boxer, P. (2008). Treating the Juvenile Offender. New York,
New York, U.S.: Guilford Press.
Jones-Brown, D., & Weston Henriques, Z. (1997, Winter). jstor.org. Retrieved
September 13, 2012, from JSTOR: http://www.jstor.org/stable/29767049
Keating, L., Tomishima, M., Foster, S., & Alessandri, M. (2002). The Effects of a
Mentoring Program on At-Risk Youth. Adolescence, 37 (148), 717-734.
Lipsey, M., Howell, C., Kelley, R., Chapman, G., & Carver, D. (2010, December).
Georgetown.edu. Retrieved February 17, 2012, from Georgetown University:
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
61
http://cjjr.georgetown.edu/pdfs/ebp/ebppaper.pdf
Miller, W. (1999). Enhancing Motivation for Change in Substance Abuse Treatment.
Rockville, MD, U.S.: U.S. Department of Health and Human Services.
National Center for Juvenile Justice. (2009). U.S. Departement of Justice Office of Justice
Programs Statistical Briefing Book. Retrieved October 21, 2012, from OJJDP:
http://www.ojjdp.gov/ojstatbb/dat.html
OJJDP. (2008). Easy Access to State and Juvenile Court Case Counts. (Office of Juvenile
Justice and Delinquency Prevention, Producer) Retrieved October 21, 2012, from
ojjdp.gov: http://www.ojjdp.gov/ojstatbb/ezaco/
Office of Justice Programs. (2011, November). Office of Justice Programs. Retrieved
September 13, 2012, from US Department of Justice:
http://www.ojp.usdoj.gov/newsroom/factsheets/ojpfs_mentoring.html
Prochaska, J., Norcross, J., & Diclemente, C. (1995). Changing For Good: A
Revolutionary Six-‐Stage Program for Overcoming Bad Habits and Moving Your
Life Positively Forward. New York, New York, U.S.: Harper Collins.
Puzzanchera, C., & Kang, W. (2013). Retrieved May 11, 2013, from Easy Access to
FBI Arrest Statistics 1994-‐2010 Online: Available:
http://www.ojjdp.gov/ojstatbb/ezaucr/
Puzzanchera, C. A. (2012, May). Office of Juvenile Justice and Delinquency Prevention.
Retrieved September 12, 2012, from National Juvenile Court Data Archive:
http://www.ojjdp.gov/ojstatbb/njcda/pdf/jcs2009.pdf
QuickFacts, U. S. Census Bureau State & County. (2012). U.S. Department of
Commerce: United States Census Bureau. Retrieved January 31, 2012, from
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
62
United States Census Bureau Web site:
http://quickfacts.census.gov/qfd/states/53/53061.html
Rapp-‐Paglicci, L., Ersing, R., & Rowe, W. (2007). The Effects of Cultural Arts
Programs on At-‐Risk Youth. Journal of Social Service Research , 33 (2), 51-‐56.
Reclaiming Futures. (2012). Reclaiming Futures. (R. W. Foundation, Producer)
Retrieved November 3, 2012, from reclaimingfutures.org:
http://www.reclaimingfutures.org/model_how_it_works
Rhodes, J., Grossman, J. & Resch, N. (2000, Nov.-Dec.). JSTOR. Retrieved September
13, 2012, from www.jstor.org: http://www.jstor.org/stable/1132506
Snohomish County Juvenile Court. (2013). Snohomish County Superior Court. Retrieved
April 5, 2013, from Snohomish County:
http://www1.co.snohomish.wa.us/Departments/Superior_Court/Juvenile_Services
/Services/Offender_Services/Supervision_Services.htm
Stone, A., Becker, L., Huber, A., & Catalano, R. (2012). Review of Risk and Protective
Factors of Substance Use and Problem Use in Emerging Adulthood. Addictive
Behaviors , 37, 747-775.
Theodoulou, S., & Kofinis, C. (2004). The Art of the Game. Belmont, CA, U.S.: Clark
Baxter.
U.S. Census Bureau. (2010). 2010 Demographic Profile Data for Snohomish County.
(U. S. Department of Commerce, Producer) Retrieved May 28, 2013, from U.S.
Department of Commerce: U.S. Census Bureau:
http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xh
tml?src=bkmk
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
63
U.S. Department of Justice. (2012, December 21). OJJDP FY 2013 Mentoring Best
Practices Research. Office of Juvenile Justice and Delinquency Prevention Grant
Application . U.S. Department of Justice.
Vincent, G., Guy, L., & Grisso, T. (2012). Risk Assessment in Juvenile Justice: A
Guidebook for Implementation. Models for Change System Reform in Juvenile
Justice. John D. and Catherine T. MacArthur Foundation.
Washington Courts. (2013). Washington State Courts. Retrieved May 28, 2013, from
Washington State Courts Web site:
http://www.courts.wa.gov/caseload/content/archive/superior/Annual/2010.pdf
Washington State Institute for Public Policy. (2013, January). Inventory of Evidence-‐
Based, Research-‐Based, and Promising Practices. Retrieved May 14, 2013,
from Access Washington:
http://www.wsipp.wa.gov/rptfiles/E2SHB2536-‐2i.pdf
Wilson, J. (2000). Evaluation of the Juvenile Mentoring Program: JUMP. Office of
Juvenile Justice and Delinquency Prevention, U.S. Department of Justice. OJJDP.
WSIPP. (2010, December). Washington State Institute for Public Policy. Retrieved
April 14, 2013, from www.wsipp.wa.gov:
http://www.wsipp.wa.gov/rptfiles/10-‐12-‐1201.pdf
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
64
Appendix A
PACT 2.0 Full Risk Assessments Completed from 1/6/11 to 7/6/11 in Snohomish County
Retrieved on 4/7/12
1. History of structured recreational activities: Within the past 5 years, minor has participated in structured and supervised pro-social community activities, such as religious group/church, community group, cultural group, club, athletics, or other community activities. Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
Involved in 2 or more structured activities 72
25.99%
Involved in 1 structured activity 100
36.1%
Never involved in structured activities 105
37.91%
2. Current interest and involvement in structured recreational activities: Minor participates in structured and supervised pro-social community activities, such as religious group/church, community group, cultural group, club, athletics, or other community activity. Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
Currently involved in 2 or more structured activities 6
2.17%
Currently involved in 1 structured activity 35
12.64%
Currently interested but not involved 95
34.3%
Not interested in any structured activities 141
50.9%
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
65
3. History of positive adult non-family relationships not connected to school or employment: This includes adults, who are not teachers and not part of the minor's family, who can provide support and model pro-social behavior, such as religious leader, club member, community person, etc. Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No positive adult relationships 132
47.65%
1 positive adult relationship 99
35.74%
2 positive adult relationships 32
11.55%
3 or more positive adults relationships 14
5.05%
4. Current positive adult non-family relationships not connected to school or employment: Adults, who are not teachers and not part of the minor's family, who can provide support and model pro-social behavior, such as religious leader, club member, community person, or probation officer, etc. Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No positive adult relationships 138
49.82%
1 positive adult relationship 84
30.32%
2 positive adult relationships 37
13.36%
3 or more positive adults relationships 18
6.5%
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
66
5. Current pro-social community ties: Minor feels there are people in his or her community who encourage him or her to stay out of trouble and are willing to help the minor. Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No pro-social community ties 102
36.82%
Some pro-social community ties 169
61.01%
Has strong pro-social community ties 6
2.17%
6. History of minor's alcohol use: Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No use of alcohol 39
14.08%
Past use of alcohol 238
85.92%
Alcohol disrupted education 97
35.02%
Alcohol caused family conflict 137
49.46%
Alcohol interfered with keeping pro-social friends 113
40.79%
Alcohol caused health problems 22
7.94%
Alcohol contributed to criminal behavior 83
29.96%
Minor needed increasing amounts of alcohol to achieve same level of intoxication or high
54
19.49%
Minor experienced withdrawal problems 25
9.03%
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
67
7. History of minor's drug use: Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No past drug use 35
12.64%
Past use of drugs 242
87.36%
Drugs disrupted education 151
54.51%
Drugs caused family conflict 183
66.06%
Drugs interfered with keeping pro-social friends 158
57.04%
Drugs caused health problems 35
12.64%
Drugs contributed to criminal behavior 119
42.96%
Minor needed increasing amounts of drugs to achieve same level of intoxication or high
90
32.49%
Minor experienced withdrawal problems 67
24.19%
8. History of referrals for drug/alcohol assessment: Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
Never referred for drug/alcohol assessment 118
42.6%
Referred but never assessed 12
4.33%
Diagnosed as no problem 6
2.17%
Diagnosed as abuse 10
3.61%
Diagnosed as dependent/addicted 131
47.29%
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
68
9. Minor is currently using alcohol or drugs: Get Detail
277 - Total Participants
277 - Total Participants Who Responded
100% - Percentage Who Responded
Answer Responses Visual Percentage Percentage
No, do not compete Domain 8B 68
24.55%
Yes, must complete Domain 8B 209
75.45%
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
69
Appendix B
PACT Risk Needs Assessment Questions
Part A:
• Current interest and involvement in pro-social structured recreational
activities:
o Not interested in any structured activities
o Currently interested but not involved
o Currently involved in 1 structured activity
o Currently involved in 2 or more structured activities
• Current positive adult non family relationships:
o No positive adult relationship
o 1 positive adult relationship
o 2 positive adult relationships
o 3 or more positive adults relationships
• Current pro-social community ties:
o No pro-social community ties
o Some pro-social community ties
o Has strong pro-social community ties
Part B:
• Current alcohol and drug use:
o No
o Yes
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
70
Appendix C
(Sample survey for PAIR youth participant) I.D. #_____
Dear students, we would like you to fill out this survey to provide feedback to the PAIR program. This is important so that we know your thoughts. All of your individual responses are confidential and will not be shared with your probation counselor. Please answer the following twelve questions that deal with your feelings about the PAIR program. Thank you for taking the time to complete this survey. Name:
1. How enthusiastic was the PAIR coordinator (Henri) when working with you?
a. Extremely enthusiastic b. Very enthusiastic c. Moderately enthusiastic d. Slightly enthusiastic e. Not at all enthusiastic
2. Compared to three months ago, do you feel more connected to your
community? a. Much more connected b. Sort of connected c. Not sure d. Slightly less connected e. Not at all connected
3. How enthusiastic was the PAIR artist when working with you?
a. Extremely enthusiastic b. Very enthusiastic c. Neutral d. Slightly enthusiastic e. Not at all enthusiastic
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
71
4. Compared to three months ago, do you feel like there are adults in your life that are much more encouraging?
a. A lot more encouraging adults b. Somewhat more encouraging adults c. No change with encouraging adults d. Slightly less encouraging adults e. No encouraging adults
5. Compared to three months ago, how INTERESTED are you in positive
activities within the community you live? a. Much more interested b. Sort of more interested c. Same amount of interest as before d. Slightly less interested e. Not at all interested
6. Compared to three months ago, how INVOLVED are you in positive activities
in your community? a. Much more involved b. Sort of involved c. Same amount of involvement as before d. Slightly less involved e. Not at all involved
7. Compared to three months ago, how do you feel about the community where
you live? a. Like the community a lot b. Like a little bit c. Neither like nor dislike d. Dislike a little bit e. Strongly dislike
8. Compared to three months ago, how much support does the community in
which you live provide? a. A great deal more support b. Some more support c. Not sure d. Less support e. No support at all
9. Compared to three months ago, how motivated are you to not use drugs?
a. Much more motivated b. Somewhat more motivated c. Same as three months ago d. Slightly less motivated e. Not at all motivated
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
72
10. Compared to three months ago, how motivated are you to not use alcohol?
a. Much more motivated b. Somewhat more motivated c. About as motivated d. Slightly less motivated e. Not at all motivated
11. What art mediums would you suggest for future PAIR workshops?
12. What changes would most improve the PAIR program?
Thank you for your participation and feedback.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
73
Table 1 Summary of Youth Referred to PAIR Workshops. Workshops Number of
Youth Percent Valid Percent Cumulative
Percent 1st PAIR Workshop
8 14.8 14.8 14.8
2nd PAIR Workshop
9 16.7 16.7 31.5
3rd PAIR Workshop
14 25.9 25.9 57.4
4th PAIR Workshop
13 24.1 24.1 81.5
5th PAIR Workshop
10 18.5 18.5 100.00
Total 54 100.0 100.0 Table 2 Summary of Referred Youth Age at Start of PAIR Workshop. Number of
Youth Percent Valid Percent Cumulative
Percent 14 Years Old 2 3.7 3.7 3.7 15 Years Old 10 18.5 18.5 22.2 16 Years Old 21 38.9 38.9 61.1 17 Years Old 14 25.9 25.9 87.0 18 Years Old 6 11.1 11.1 98.1 19 Years Old 1 1.9 1.9 100.00 Total 54 100.0 100.0 Table 3 Summary of Referred Participant Gender. Number of
Youth Percent Valid Percent Cumulative
Percent Female 35 64.8 64.8 64.8 Male 19 35.2 35.2 100.0 Total 54 100.0 100.0 Table 4 Summary of Referred Youth Average Age and Standard Deviation. Total Youth 54 Missing 0 Mean 16.28 Median 16 Mode 16 Standard Deviation 1.071
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
74
Table 5 Summary of Wilcoxon Signed Rank Test on Interest and Involvement in Pro-Social Activities. N Mean Rank Sum of Ranks Negative Ranks 0* .00 .00 Positive Ranks 14** 7.50 105.00 Ties 33*** Total 47 *Post Interest & Involvement in Pro-Social Activities < Pre Interest & Involvement in Pro-Social Activities. **Post Interest & Involvement in Pro-Social Activities > Pre Interest & Involvement in Pro-Social Activities. ***Post Interest & Involvement in Pro-Social Activities = Pre Interest & Involvement in Pro-Social Activities.
Descriptive Statistics
N Percentiles
25th 50th
(Median) 75th
Pre Interest & Involvement in Pro-Social Activities
49 1.00 1.00 2.00
Post Interest & Involvement in Pro-Social Activities
47 1.00 2.00 3.00
Test Statistics Wilcoxon Signed Ranks Test
Post Interest & Involvement in Pro-Social Activities – Pre Interest & Involvement in Pro-Social Activities
Z Asymp. Sig (2-tailed)
-3.407* .001
*Based on negative ranks.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
75
Table 6 Summary of Wilcoxon Signed Rank Test on Positive Adult Non-Family Relationships. N Mean Rank Sum of Ranks Negative Ranks 1* 6.50 6.50 Positive Ranks 15** 8.63 129.50 Ties 31*** Total 47 *Post Positive Adult Non-Family Relationships < Pre Positive Adult Non-Family Relationships. **Post Positive Adult Non-Family Relationships > Pre Positive Adult Non-Family Relationships. ***Post Positive Adult Non-Family Relationships = Pre Positive Adult Non-Family Relationships.
Descriptive Statistics
N Percentiles
25th 50th
(Median) 75th
Pre Positive Adult Non-Family Relationships
49 1.00 1.00 2.00
Post Positive Adult Non-Family Relationships
47 1.00 2.00 3.00
Test Statistics Wilcoxon Signed Ranks Test
Post Positive Adult Non-Family Relationships – Pre Positive Adult Non-Family Relationships
Z Asymp. Sig (2-tailed)
-3.346* .001
*Based on negative ranks.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
76
Table 7 Summary of Wilcoxon Signed Rank Test on Pro-Social Community Ties. N Mean Rank Sum of Ranks Negative Ranks 2* 5.00 10.00 Positive Ranks 7** 5.00 35.00 Ties 38*** Total 47 *Post Pro-Social Community Ties < Pre Pro-Social Community Ties. ** Post Pro-Social Community Ties > Pre Pro-Social Community Ties. *** Post Pro-Social Community Ties = Pre Pro-Social Community Ties.
Descriptive Statistics
N Percentiles
25th 50th
(Median) 75th
Pre Pro-Social Community Ties
49 1.00 1.00 1.00
Post Pro-Social Community Ties
47 1.00 1.00 1.00
Test Statistics Wilcoxon Signed Ranks Test
Post Pro-Social Community Ties – Pre Pro-Social Community Ties
Z Asymp. Sig (2-tailed)
-1.667* .096
*Based on negative ranks. Table 8 Summary of Bivariate Spearman’s Correlation for Post Intervention PACT Scores of Protective Factors and Participant Attendance in PAIR Workshops. PAIR
Sessions Attended
Interest & Involvement in Pro-‐social Activities
Positive Adult Non-‐family Relationships
Pro-‐social Community Ties
PAIR Sessions Attended
-‐ .223
.305*
.546**
* P < .05; ** P < .01
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
77
Table 9 Summary of McNemar’s Test on participant alcohol use as measured by the
PACT.
Post Alcohol Use
Not Currently Using Currently Using Total
Count 26 6 32
Not
Currently
Using
% Within Initial
Alcohol Use
81.3% 18.8% 100%
Initial
Alcohol
Use
% Within Post
Alcohol Use
83.9% 37.5% 68.1%
Count 5 10 15
Currently
Using
% Within Initial
Alcohol Use
33.3% 66.7% 100%
% Within Post
Alcohol Use
16.1% 62.5% 31.9%
Total Count 31 16 47
% Within Initial
Alcohol Use
66.0% 34.0% 100%
% Within Post
Alcohol Use
100% 100% 100%
Chi-Square Tests
Value Exact Sig. (2-sided)
McNemar’s Test
N of Valid Cases
47
1.00*
*Binomial distribution used.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
78
Table 10 Summary of McNemar’s Test on participant drug use as measured by the
PACT.
Post Drug Use
Not Currently Using Currently Using Total
Count 20 6 26
Not
Currently
Using
% Within Initial
Drug Use
76.9% 23.1% 100%
Initial
Drug
Use
% Within Post Drug
Use
66.7% 35.3% 55.3%
Count 10 11 21
Currently
Using
% Within Initial
Drug Use
47.6% 52.4% 100%
% Within Post Drug
Use
33.3% 64.7% 44.7%
Total Count 30 17 47
% Within Initial
Drug Use
63.8% 36.2% 100%
% Within Post Drug
Use
100% 100% 100%
Chi-Square Tests
Value Exact Sig. (2-sided)
McNemar’s Test
N of Valid Cases
47
.454*
*Binomial distribution used.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
79
Table 11 Summary of Bivariate Spearman’s Correlation for Post Intervention PACT Scores of Risk Factors and Participant Attendance in PAIR Workshops. PAIR Sessions
Attended Drug Use Alcohol Use
PAIR Sessions Attended
-‐ -‐.03 -‐.13
Table 12 Summary of Bivariate Spearman’s Correlation for Post PAIR Survey and Participant Attendance. PAIR
Sessions Attended
Coordinator Enthusiasm
Artist Enthusiasm
Encouraging Adults
Interest in Community Activities
PAIR Sessions Attended
-‐ .67** .18 .47* .59**
*P < .05 ** P < .01 Table 13 Summary of Bivariate Spearman’s Correlation for Post PAIR Survey and Participant Attendance. PAIR
Sessions Attended
Positive Community Activities
Feelings About Community
Community Support
Connection to Community
PAIR Sessions Attended
-‐ .43 .08 .14 .55*
*P < .05 Table 14 Summary of Bivariate Spearman’s Correlation for Post PAIR Survey and Participant Attendance. PAIR Sessions
Attended Abstain from Drug Use
Abstain from Alcohol Use
PAIR Sessions Attended
-‐ .45* .50*
*P < .05
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
80
Figure 1. This figure shows the gender of all PAIR referred youth in percentages.
Figure 2. This shows the gender of all other assessed youth during study period.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
81
Figure 3. Scatterplot showing the relationship between PAIR sessions attended and participant interest in community activities.
Figure 4. Scatterplot showing the relationship between PAIR sessions attended and the feelings of encouraging adults.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
82
Figure 5. These two figures show PAIR participant motivation to abstain from drug use (top) and alcohol use separated by gender.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
83
Figure 6. These two figures show the youth perception of coordinator (top) and artist enthusiasm, as measured by the post intervention survey.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
84
Figure 7. These two figures show the youth interest in community activities (top) and feelings about encouraging adults, as measured by the post intervention survey.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
85
Figure 8. These two figures show the youth involvement in positive activities (top) and feelings about community, as measured by the post intervention survey.
PROMISING ARTISTS IN RECOVERY PROGRAM EVALUATION
86
Figure 9. This figure shows the youth perception of community support, as measured by the post intervention survey.