pccyfs 2012 annual spring conference the berks county juvenile justice system improvement project...

PCCYFS 2012 Annual Spring Conference

The Berks County The Berks County Juvenile Justice System Juvenile Justice System Improvement ProjectImprovement Project

Presented by:

Robert N. Williams: Berks County Juvenile Probation Office

Richard D. Steele: Juvenile Court Judges’ Commission

PCCYFS 2012 Annual Spring

Conference

2

Evolution of an Evidence Based Probation Department• The search for structured decision-

making and assessment devices• The “JJSES” is born• Initial EBP training• Utilizing MacArthur funding• Motivational interviewing• Georgetown invitation


Conference

3

Georgetown University’s Center for Juvenile Justice Reform• December 2010 release of “Improving

the Effectiveness of Juvenile Justice Programs: A New Perspective on Evidence-Based Practice”

• March 2011 call for letters of interest to participate in the Juvenile Justice System Improvement Project (JJSIP)

• Berks partners with JCJC and PCCD to apply


Conference

4

JJSIP

• May 2011: Berks/PA one of four sites selected nationally

• Other sites selected:– Pinellas County, Florida– Maricopa County, Arizona– Hartford, Connecticut


Conference

5

JJSIP Goal

• The reduction of crime and delinquency and improved positive outcomes for youth in the juvenile justice system through the implementation of efficient and effective juvenile justice administration.


Conference

6

JJSIP Expected Outcomes

• Reduced recidivism rates for juvenile offenders

• Reduced utilization of detention and out-of-home placement, particularly for lower risk offenders

• Reduced racial and ethnic disparities and disproportionality in juvenile justice processing


Conference

7

JJSIP Expected Outcomes

• Increased probation completion rates among juvenile offenders

• More efficient use of resources• Decreased school dropout, increased

school attendance, performance and stability of school placement among juvenile offenders

• Reduced mental health symptoms and substance abuse among juvenile offenders


Conference

8

JJSIP Components

• OJJDP’s Comprehensive Strategy for Serious, Violent and Chronic Juvenile Offenders: James C. Howell and John Wilson

• The Standardized Program Evaluation Protocol: Dr. Mark Lipsey


Conference

9

Comprehensive Strategy for Serious, Violent and Chronic Juvenile Offenders

• Developed by Howell and Wilson for OJJDP in the 1990’s

• The “probation” aspect of the JJSIP

• Suggests a comprehensive framework and continuum of services from prevention to aftercare


Conference

10

CS Core Principles

• Strengthen the family in its primary responsibility

• Support core social institutions such as schools, religious institutions and community organizations

• Promote delinquency prevention as the most cost-effective approach

• Intervene immediately and effectively when delinquent behavior occurs

• Identify and control the small group of SVC juvenile offenders


Conference

11

Research Basis for the Comprehensive Strategy for Serious, Violent and Chronic Juvenile Offenders

• We know the key features of juvenile offender careers

• We understand how offender careers develop

• We have effective programs for reducing the risk of re-offending

• We have “forward-looking” administrative tools for managing offender risk and matching youth to programs


Conference

12

Non-Serious Non-ViolentNon-Chronic64%

Serious34%

Chronic15%

Violent8%

C,S & V 4%

Maricopa Co. Study (N=151,209)

Sixteen Cohorts of Juvenile Offender Court Careers


Conference

13

Maricopa County Study Key Findings

• Almost two-thirds (64%) of juvenile court careers were nonchronic (less than four referrals) and did not include any serious or violent offenses; 18% of all careers had serious (but nonviolent) offenses, 8% had violent offenses, and 4% of the careers had serious, violent, and chronic offenses.


Conference

14



Conference

15


• The CS is a two-tiered system for responding proactively to delinquency

• The first tier (prevention, youth development and early intervention) focuses on reducing the likelihood that at-risk youth will enter the juvenile justice system

• The second tier (interventions and graduated sanctions) addresses risk factors of system youth with a focus on SVC offenders


Conference

16


• But we aren’t funded or don’t have the time to address prevention!

• At the very least, you need to be involved in community prevention efforts to ensure that risk factors observed in kids referred to the court are being addressed

• Do you have diversion programs in place?


Conference

17

CS Intervention and Graduated Sanctions Highlights• Build a seamless continuum of services

with graduated placement levels• Utilize structured decision making

instruments across the continuum• Match offenders to the appropriate level

of supervision and services based on the R-N assessment

• Place low risk offenders in community programs with minimal supervision


Conference

18

CS Intervention and Graduated Sanctions Highlights• Medium risk offenders are typically

placed in more structured community-based programs with greater supervision

• Very high risk offenders are typically placed in residential settings

• SVC offenders make up <10% of the offending population


Conference

19

CS Intervention and Graduated Sanctions Highlights• Most juvenile offenders enter the

system with minor offenses and low recidivism risk

• Some minor offenders are on pathways toward serious, violent and chronic offending

• Risk assessment devices are available that measure risk well enough to guide allocation of system response and effort

• To be effective, interventions should address priority treatment needs as identified through assessment


Conference

20

Standardized Program Evaluation Protocol (Spep)

• The “provider” aspect of the JJSIP• Based on Dr. Mark Lipsey’s meta-

analysis work with nearly 700 research studies over the past twenty years

• Juveniles aged 12-21 in programs aimed at reducing delinquency

• Focus on the programs’ effects on recidivism (reoffending)


Conference

21

SPEP Defined• The SPEP is a validated, data driven

rating scheme for determining how well an existing program matches research evidence for the effectiveness of that particular program for reducing the recidivism of juvenile offenders

• More simply put, the SPEP is a tool for comparing juvenile justice programs to what has been found effective for reducing recidivism in the research

• Effectiveness = the ability to reduce the recidivism of juveniles receiving a program compared to a control group that does not receive the program


Conference

22

Effects on recidivism analyzed as a function of program characteristics

© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt University. Not to be copied or used without explicit permission.


Conference

23

Program types sorted by general approach: Average recidivism effect

Multiple services

Counseling

Skill building

Restorative

Surveillance

Deterrence

Discipline

-10 -5 0 5 10 15

% Recidivism Reduction from .50 Baseline



Conference

24

Further sorting by intervention type within, e.g., counseling approaches

Mixed w/referrals

Mixed

Peer

Group

Family crisis

Family

Mentoring

Individual

0 5 10 15 20 25




Conference

25

Further sorting by intervention type within, e.g., skill-building approaches

Job related

Academic

Challenge

Social skills

Cognitive-behavioral

Behavioral

0 5 10 15 20 25 30




Conference

26

Many types of therapeutic interventions thus have evidence of effectiveness … but there’s a catch:

• Though their average effects on recidivism are positive, larger and smaller effects are distributed around that average.

• This means that some variants of the intervention show large positive effects, but others show negligible or even negative effects.



Conference

27

Example: Recidivism effects from 29 studies of family interventions

-.40 -.30 -.20 -.10 .00 .10 .20 .30 .40 .50 .60

Family InterventionsCovariate-Adjusted Recidivism Effect Sizes (N=29)

Effect Size (zPhi coefficient)

>.00

Average recidivism reduction of 13%

Median



Conference

28

-.40 -.30 -.20 -.10 .00 .10 .20 .30 .40 .50 .60

Where are the brand name model programs in this distribution?

Family InterventionsCovariate-Adjusted Recidivism Effect Sizes (N=29)

Effect Size (zPhi coefficient)

>.00

Median

MST

FFT



Conference

29

SPEP BASICS

• Lipsey’s analysis looked at how a program’s components are related to its impact on recidivism

• He identified the primary characteristics of effective interventions with delinquents

• The SPEP creates a metric by assigning points to service programs according to how closely their characteristics match those programs of as given type that show the best recidivism outcomes


Conference

30

SPEP BASICS

• Lipsey found that the effects of juvenile delinquency interventions are mainly related to four key aspects of an intervention:– Type of program/service– Quantity or dosage amount– Service quality– Juvenile risk level


Conference

31

SPEP BASICS• The SPEP is used only to evaluate

therapeutic services (services oriented mainly toward facilitating constructive internalized sustained changes in behavior)

• Therapeutic services include:– Restorative services– Counseling and its variants– Skill-building services– Services for special populations

• Control-oriented services and services for which there is insufficient research are not able to be rated


Conference

32

SPEP BASICS

Many types of programs for diverse needs are supported by evidence:

– Interpersonal skills– Family and parenting

– Individual counseling

– Life Skills – Sex offender

treatment – Cognitive-behavioral

– Challenge programs

– Milieu therapy

Victim-offender mediationDrug & alcohol treatmentBehavioral & incentiveGroup counselingSocial caseworkEducationalVocationalPsychodynamic


Conference

33

SPEP PROCESS

• Identify programs being provided to offenders

• Break programs down into services that can be matched with research-based categories

• Collect demographic, risk, quality and quantity data for each service

• Enter data into the SPEP model to generate SPEP score

• Evaluate performance based on SPEP components and overall score


Conference

34

SPEP SCORING

• Points are assigned to each category based on the findings of Lipsey’s research

• Points are proportionate to the contribution of each rated factor to recidivism reduction

• Target values are set from either the meta-analysis (generic) or program manual for blueprint programs

• The SPEP is configured to so that the maximum total score for any service is 100 points


Conference

35

Standardized Program Evaluation Protocol (SPEP)for Services to Juvenile Offenders©

Recalibrated version, 2012

PointsPossible

PointsReceived

Primary and Supplemental Service Types [Identified according to definitions derived from the research]Primary Service Type for Program Being Rated

Group 1 services (5 points) Group 4 services (25 points)

Group 2 services (10 points) Group 5 services (30 points)

Group 3 services (15 points)

30

Supplemental Service TypeQualifying supplemental service used: Yes (5 points) No (0 points)

5

Quality of Service Delivery[Determined from a systematic assessment of the relevant features of the provider and provider organization]Rated quality of services delivered:

Low (5 points)Medium (10 points)High (20 points)

20

Amount of Service[Determined from data for the qualifying group of service recipients]Duration [Target number of weeks specified for each service type] % of youth who received at least the target weeks of service:

0% (0 points) 60% (6 points)20% (2 points) 80% (8 points)40% (4 points) 99% (10 points)

10

Contact Hours [Target number of hours specified for each service type] % of youth who received at least the target hours of service:


10

Risk Level of Youth Served[Determined from risk ratings on a valid instrument for the qualifying group of service recipients] % of youth with at least the target risk score set for the JJ system:


25

Provider’s Total SPEP Score

100

(Insert Score)

• Points assigned proportionate to the contribution of each factor to recidivism reduction

• Target values from the meta-analysis (generic) OR program manual (manualized)


Conference

36

Understanding A Spep Score

• The SPEP score is configured so that the total score indicates how closely the key characteristics of a rated program match those of the programs of that type that showed at least average positive effects in the corresponding research

• The difference between the scores for the individual SPEP components and the maximum values possible for each provide a diagnostic information about where the rated program can improve


Conference

37

Understanding A Spep Score

• SPEP scores have been shown to relate to programs’ effects on recidivism and can thus provide useful guidance for program evaluation and improvement

• However, there are almost certainly other factors related to a program's effectiveness that should be taken into consideration, most notably the clinical quality of the interaction between juveniles and their service providers


Conference

38

I’ve Got My Score, Now What?

• Scores should be considered a baseline and a road map for program improvement

• What can I do to increase my score?

• How can I work with probation and the courts to improve my effectiveness?


Conference

39

Hypothetical SPEP Score


Conference

40

Program Improvement Strategies

• PRIMARY SERVICE (5-30 points)• Are there any circumstances where a

provider might consider changing the primary service?

• Dosage requirements can’t be met due to caseload sizes

• Service not meeting the needs of the target population

• A more potent service?


Conference

41


• SUPPLEMENTAL SERVICE (0 or 5 points)• Can a supplemental service be added

that fits (reinforces or augments) with the primary service?


Conference

42


• AMOUNT OF SERVICE• DURATION (0-10 points)• CONTACT HOURS (0-10 points)• Are youth receiving the target amount

of service?• How can retention be improved?• Are youth removed beyond your

control?• Are therapist caseload sizes preventing

the meeting of dosage requirements?


Conference

43


• QUALITY OF SERVICE (5-20 points)• What are the quality indicators?• Manualized protocols• Training, coaching and feedback• Monitoring to maintain fidelity• How is corrective action taken when

staff departs from protocol or quality lapses?


Conference

44


• RISK LEVEL (0-25 points)• More points given for the % of high-risk

youth involved in the program• Consider inclusionary and/or

exclusionary admission criteria with preferred risk level indicated

• At the mercy of the courts?


Conference

45

Hypothetical Second Year Score


Conference

46

Why Do Some Programs Fail to Respond to the Researcher’s Call for Change?

• Strong skepticism about the applicability of research to practice.

• A strong belief across systems (counselors, program administrators, judges, POs) that the program already works.

• A belief that the program delivers other more important outcomes beyond recidivism reduction.

• Legitimate funding challenges associated with change.

• Pride!


Conference

47

SPEP BEST PRACTICE GUIDELINES

• Therapeutic approaches are more effective

• Larger effects will be seen with higher risk youth

• The amount of service should at least match the average of effective programming according to the research

• High quality implementation, including treatment protocols and monitoring for adherence, is important


Conference

48

JJSIP IMPLEMENTATION

• 18 month project

• Berks is the pilot site for an anticipated statewide rollout

• Elements of the JJSIP have already been incorporated into PA’s Juvenile Justice System Enhancement Strategy


Conference

49

History of JJSES

• Concept of JJSES “born” in June 2010 at JCJC/Chiefs Annual Strategic Planning Meeting

• Designed to organize / sustain “lessons learned” from Models for Change

• Risk/Need Assessment (YLS) activities were increasing – needed to move to the next stage

• EBP Exposure through Mark Carey Training • Leadership Team created

Keith Snyder, Coordinator (JCJC) Bob Tomassini (Adams) Bob Williams (Berks)Sam Miller (Cumberland) Beth Fritz (Lehigh)Mike Pennington (PCCD) Rick Steele (JCJC)

• Statement of Purpose Developed


Conference

50

JJSES Statement of Purpose

• We dedicate ourselves to working in partnership to enhance the capacity of Pennsylvania’s juvenile justice system to achieve its balanced and restorative justice mission by:– Employing evidence-based practices, with

fidelity, at every stage of the juvenile justice process;

– Collecting and analyzing the data necessary to measure the results of these efforts; and, with this knowledge,

– Striving to continuously improve the quality of our decisions, services and programs.


Conference

51

“Statement of Purpose” Endorsements

• Juvenile Court Judges’ Commission (JCJC)• Juvenile Court Section of the Pa. Conference

of State Trial Judges• Pa. Council of Chief Juvenile Probation

Officers• Juvenile Justice and Delinquency Prevention

Committee of the Pa. Commission on Crime and Delinquency

• County Juvenile Probation Departments/Juvenile Courts

• Over 30 service provider organizations and agencies (including JDCAP, PCCYFS, PCPA)


Conference

52

Leadership Team’s Initial Activities• Identification of various initiatives /

activities

• Who’s “in charge”?

• Where is the “home” of each initiative / activity?

• What’s the status of each initiative / activity?

• Is there a sustainability plan?


Conference

53

Elements of Pennsylvania’s Models for Change Initiatives


Conference

54

Challenges: • The pieces of the puzzle were at various

stages of implementation among jurisdictions

• How do we “transform” the pieces of the puzzle into a comprehensive strategy?

• What is the recommended sequence of activities for probation departments, providers, and others?

• Do we have the necessary infrastructure to support implementation of each element?

• Communications strategy

• Stakeholder involvement

• What does “evidence-based” really mean?


Conference

55

Programs / interventions can be placed along a “proof of effectiveness” continuum

How confident are we that a program will improve outcomes?

Not much

confidence

Best Practices“We’ve done it

and we like it”

Promising Practices“We really think this

will work… but we need time to prove it”

Research-based“This program is based on sound theory informed by research”

Evidence-based“This program has been rigorously evaluated and shown to work”

Very much

confidence

Bumbarger, B. K. (2009). Promoting the Use of Evidence-based Prevention: Application in the Real World. Available online at https://breeze.psu.edu/p22215525


Conference

56

Intermediate Goals / Activities• Work with The Carey Group to develop a

framework for an implementation strategy• Create a JJSES Workgroup to achieve

broader juvenile justice system representation and involvement

• Develop a JJSES “Monograph”• Develop infrastructure to support activities• Integrate “lessons learned” from PA’s

participation in the Juvenile Justice System Improvement Project (JJSIP)


Conference

57

JJSES FrameworkAchieving our Balanced and Restorative Justice Mission


Conference

58

What Can We Expect?

“Research has shown that

the risk of recidivism is greatly reduced (10-30% on average)

when attention is paid

to criminogenic needs.”

SOURCE: D.A.Andrews, I. Zinger, R.D. Hoge, J. Bonta, P. Gendreau and F.T. Cullen,Does correctional treatment work? A clinically relevant and psychologically informed meta-

analysis,Criminology, 28 (1990); Andrews (2007)


Conference

59

Goals for 2012

• PCCD planning/implementation grants to support local implementation of JJSES

• 6 Regional training programs for county teams

• Development of JJSES Monograph and “How To” Guide for Chief Juvenile Probation Officers

• County survey to determine extent to which elements of JJSES Framework have been implemented

• Development of Leadership Academy / management training curricula

• Carey Guide training


Conference

60

Goals for 2012

• Phase IV (Final) YLS County Training• Rollout of Standardized Case Plan tied to

YLS assessments• Development of online “Program and

Practice Effectiveness Tool Kit” to clarify definitions and list evidence-based curricula

• Increased capacity of Resource Center for Evidence-based Programs and practices

• Increase capacity of PaJCMS data analysis infrastructure

• Development of infrastructure to support statewide implementation of SPEP


Conference

61

Contact Information

• Robert N. Williams• Chief Juvenile Probation Officer• Berks County Juvenile Probation Office• [email protected]

• Richard D. Steele• Director of Policy & Program

Development• Juvenile Court Judges’ Commission• [email protected]

pccyfs 2012 annual spring conference the berks county juvenile justice system improvement project...

Documents