juvenile justice system enhancement strategy ...€¦ · organizational change research has helped...
TRANSCRIPT
August 4, 2012
Juvenile Justice System Enhancement
Strategy Implementation
Manual
A “How To” Guide
JJSES | STAGE ONE
1
Introduction Most of the early adopters of evidence-based practices have “muddled through” a set of
planning activities culminating in sometimes-effective, sometimes-ineffective decisions.
Fortunately, much has been learned through these early, pioneering efforts. In addition,
organizational change research has helped guide successful change strategies. The Juvenile
Justice System Enhancement Strategy (JJSES) Implementation Manual “How-To” Guide was
developed to shorten the learning curve for evidence-based practices (EBP) planners and
reduce the muddling through events. This Manual was written for those individuals within
Pennsylvania’s juvenile probation departments who serve in a leadership or coordinating role
around evidence-based practices. The work by these leaders and coordinators in aligning
policy and practice with risk reduction research evidence requires a thoughtful, strategic plan
and process. Pennsylvania’s juvenile justice system seeks to take advantage of the lessons
learned from early adopters, research, and JJSES resources to take a more direct route for
achieving system enhancements.
This How-To Guide provides local probation leaders and coordinators with a step-by-step
process through each of the four recommended Stages of Implementation (Readiness,
Initiation, Behavioral Change and Refinement). Each Stage is divided into activities to be
conducted in the beginning, middle and end of the Stage and contain the following:
Outcomes: Key outcomes to be achieved as a result of completion of the Stage
activities
Competencies: Staff competencies to be acquired as a result of the training conducted
within the Stage
Activities: Recommended activities to be employed or initiated by the probation
department during the corresponding Stage
Support: The type of Stage supports and resources made available to assist
probation agencies implement JJSES successfully
Proficiency: A checklist to determine if the department has met the proficiency
benchmarks to be ready to engage in the activities of the next Stage
Performance A set of core performance measures to measure progress in achieving
Measures: JJSES goals
Resources For A list of additional resources related to the Stage activities
More Information:
JJSES | STAGE ONE
2
I. STAGE ONE
Outcomes by the End of Stage One1
1. 90% of the probation staff will become knowledgeable about evidence-based practices
based on an 80% or better score on an EBP pre/post test
2. 75% of the local justice system stakeholders will become knowledgeable about
evidence-based practices based on an 80% or better score on an EBP pre/post test
3. The department will have completed a human, fiscal, and data cost analysis of JJSES
4. The department will have assessed its organizational readiness for JJSES
implementation and adequately addressed any shortcomings as determined by the
pre/post Likert Readiness to Change survey
5. The department will have completed an eighteen month action plan
Competencies Achieved in Stage One
Knowledge about research findings on risk reduction
Awareness of human, fiscal, and data costs in implementing EBP through JJSES
Leadership and organizational change skills
1 The outcomes listed throughout the Chief’s How-To Guide should be viewed as drafts. That is, until baseline data is
collected it is not clear what is reasonable to expect or what a plausible target should consist of. The outcome figures are best estimates at this stage of JJSES development.
Stage One: Readiness
As a result of this stage, the jurisdiction will acquire knowledge about risk reduction research and analyze the fiscal, human resource, and data costs
required to implement Pennsylvania’s Juvenile Justice System Enhancement Strategy. The jurisdiction will examine its own capacity for change by
assessing its readiness.
JJSES | STAGE ONE
3
Activities and JJSES Support
Summary of Stage Beginning: The first series of activities are designed to get the
probation department and stakeholder leadership on board with the fundamental research that
has guided changes in juvenile justice risk reduction practices. Key to this stage is helping
juvenile justice system administrators fully understand why aligning the juvenile probation
department’s policies and practices with research evidence is consistent with their public safety
mission. If juvenile justice professionals are not convinced that research evidence can improve
public safety outcomes, then the JJSES will not make sense or be sufficiently supported.
Activities. The first step in Stage One is to
understand the framework of the JJSES model
and to conduct initial EBP training. The
recommended activities include:
1. Review the results of the County EBP
Implementation Survey conducted in the
summer of 2012. The survey describes
the manner in which departmental
policies and practices are aligned with
EBP, as well as what resources and
technical assistance may be needed.
2. Communicate with a JJSES Point of
Contact (POC) if there is a question
regarding JJSES implementation or a
need for technical assistance. A POC list
has been established that enables a
county to contact individuals who can
respond to questions or connect counties
to available resources or technical
assistance. The POC list was developed
to give counties access to consultation with colleagues with technical expertise, with
whom they may have a shared collaborative history, or from a county similar in size,
demographics or location.
3. Train probation staff on EBP. This training provides every staff member with the
foundational EBP knowledge on which future EBP training will be built.
4. Train stakeholders, service providers, and victim advocates on EBP. In addition to
department staff, others involved with the department will need to be familiar with the
JJSES Support for Beginning of Stage One
Monograph on JJSES
Intro to EBP training curriculum (one day) for probation staff
Intro to EBP training curriculum (two
hours) for stakeholders
EBP training for new staff on EBP at the orientation trainings
A listing of data inputs and outcome
data needed to be collected locally for this Stage (forthcoming)
Technical assistance
JJSES | STAGE ONE
4
same foundational EBP knowledge. This shared knowledge base will help build support
for future policy and practice changes.
5. Conduct an EBP knowledge test before and after the training. Administering the pre-
post EBP knowledge test (see enclosed) will increase the confidence of the department
that core EBP information has been retained (as measured by pre and post scores) or to
develop a plan to address deficiencies (in the event of unsatisfactory scores).
6. Deliver or communicate with a POC to obtain the Staff Introduction to EBP Training
PowerPoint (6 hours) and a “Stakeholder Introduction to EBP Training” PowerPoint (2
hours).
Additional Resources. For more reading on the core principles for evidence-based
practices, the following resources are available
Implementing Evidence-Based Practice in Community Corrections: The Principles of Effective
Interventions, National Institute of Corrections. See http://nicic.gov/Library/019342
Tools of the Trade: A Guide to Incorporating Science Into Practice, National Institute of
Corrections. See http://nicic.gov/Library/020095
Coaching Packet: Implementing Evidence Based Practices, Center for Effective Public Policy.
See http://www.cepp.com/documents/Implementing Evidence Based Practices.pdf
Evidence-Based Practice to Reduce Recidivism: Implications for State Judiciaries. See
http://community.nicic.gov/media/p/6977.aspx
Colorado Probation and Evidence-Based Practices: A systemic view of the past, present &
future of EBP in Colorado Probation. See
http://www.courts.state.co.us/userfiles/file/Administration/Probation/APPA_Presentation_Materia
l/EBP_Report.pdf
JJSES | STAGE ONE
5
Summary of Middle of Stage One: The next set of activities in the Readiness Stage of
JJSES is to examine the department’s organizational capacity to undergo a long-term re-
engineering process. This self-examination spans a number of areas such as leadership and
staff alignment with the values and expectations of an EBP department, resource capacity,
timing, organizational robustness, stakeholder support, and management commitment. It is not
advised that a department begin an EBP initiative
until it is confident that it has sufficient stamina,
resiliency, and desire to successfully implement the
required changes. If the organization is not fully
prepared or the existing environment is not
conducive to this change effort, the JJSES will not
take hold and efforts to re-engage staff and
stakeholders in the future will be more challenging.
Activities. The activities in the middle of Stage
One are designed to build on those completed at
the beginning of Stage One. Once the department
is fully aware of the demands of an EBP
environment it can better determine whether it
wants to proceed. Therefore, the activities in the
middle of Stage One are designed to help the
department assess whether it is prepared to move
ahead. This part of the Stage includes an
assessment process to help answer the readiness
question and to assist in preparing the department for the type of changes that follow. The
activities to be completed include:
1. Conduct an organizational readiness survey. This survey assesses the department’s
“health” which is directly tied to likelihood of achieving a successful change initiative.
The survey, to be taken by all department personnel, measures perceptions of effective
communication, staff alignment with department mission, trust in leadership, etc. The
survey is enclosed.
2. Complete a cost benefit analysis. EBP will likely require the shifting of resources and
implementing policy decisions that will result in “trade-offs.” For example, the
department may need to terminate some activities in order to provide resources for
another; establish a continuous quality improvement process; and ensure that personnel
will have the time to integrate EBP activities into their day-to-day work. The cost benefit
analysis will allow the department to examine what it hopes to gain through JJSES and
determine whether it is prepared to make the cost-related decisions to achieve these
gains. The cost benefit analysis worksheet is enclosed.
JJSES Support for Middle of Stage One
Organizational Readiness survey
(enclosed)
Cost Benefit Analysis Worksheet
(enclosed)
Technical assistance
JJSES | STAGE ONE
6
3. Determine level of technical assistance support needed, if any. Counties may elect to
either: a) conduct a self-administered organizational readiness survey, cost/benefit
analysis, and an action plan for implementing EBP (see enclosed); or b) request an
independent EBP readiness and/or alignment assessment to be conducted by a team
assembled from the POC list. This assessment will examine existing policy and practice
to provide the agency with a sense of how its operation is currently prepared for, and in
alignment with, EBP and where it needs to change in order to be consistent with
research evidence. The self-administered tools are enclosed and an independent
assessment can be accessed by contacting a POC.
4. Develop an action plan. An action plan can be developed once the organizational
readiness and cost benefit analyses have been completed and areas of concern
identified by the readiness survey have been addressed. Anticipated costs will need
to be identified and addressed by the department and stakeholders. A “roadmap”
will need to be constructed to give stakeholders and all juvenile court personnel a
clear sense of where the department intends to go, what the activities will look like,
and when they will be accomplished. An action plan template is enclosed.
Additional Resources.
For more reading on implementation and planning issues, the following resources will be made
available on the JJSES website:
1. Implementation Research: A Synthesis of the Literature, authored by Fixsen, D. L.,
Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Tampa, FL: University
of South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHI Publication #231). Available at
http://ctndisseminationlibrary.org/PDF/nirnmonograph.pdf
2. The Change Book; a Blueprint for Technology Transfer, Addiction Technology Transfer
Center Network. Published in 2010 by the Addiction Technology Transfer Center
(ATTC) National Office University of Missouri-Kansas City, Missouri through a
cooperative agreement from the Substance Abuse and Mental Health Services
Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT). See
http://attcnetwork.org/explore/priorityareas/techtrans/docs/TheChangeBook2010.NOfl.pdf
3. Roadmap for Evidence-Based Practices in Community Corrections, Crime and Justice
Institute at Community Resources for Justice (2010). Commonwealth of Virginia. Report
supported by Grant No. 2007-DD-BX-K011, Bureau of Justice Assistance, Office of
Justice Programs. Available at http://cjinstitute.org/files/Roadmap_Final.pdf
4. A Ten-Step Guide to Transforming Probation Departments to Reduce Recidivism.
Council of State Governments Justice Center, Fabelo, T., Nagy, G., and Prins, S.
Report supported by the Bureau of Justice Assistance (BJA), Office of Justice Programs,
JJSES | STAGE ONE
7
U.S. Department of Justice (under grant # 2009-DD-BX- K139); the Pew Center on the
States Public Safety Performance Project; the Texas Department of Criminal Justice–
Community Justice Assistance Division
(TDCJCJAD); and the Travis County
Community Supervision and Corrections
Department (CSCD) Adult Probation
Department. Available at
http://knowledgecenter.csg.org/drupal/content/te
n-step-guide-transforming-probation-
departments-reduce-recidivism
5. Step-by-Step Planning Guide; Six Phases
Toward Implementing Evidence-Based
Practices for Risk Reduction, The Carey
Group. Available at
http://www.thecareygroupinc.com/complimentary
resources
Summary of Stage End: The final set of
activities in the Readiness Stage involve acting on
the information collected in the beginning and
middle portions of Stage One. For example, the
organizational readiness survey and initial EBP
trainings will likely reveal a set of issues that
require immediate attention. The alignment and cost benefit analyses will likely lead to activities
for the action plan. These Stage End steps are designed to put in motion the preparation for
activities to be commenced in Stage Two.
Activities. The following activities are derived from those prompted by the Middle Stage (i.e.,
surveys and analyses) and include:
1. Follow up to readiness survey. It is critical that department staff who took the survey see
the results, understand the department strengths, and be given assurance that areas of
concern will be addressed. Often, agencies will create workgroups at this stage to review
survey results and develop an action plan. By addressing these organizational climate
issues the department will be in a better position to implement EBP change initiatives and
garner staff buy-in and support.
2. Stakeholder Involvement. Jurisdictions will differ in the level of desired stakeholder
involvement. For example, in some cases, the attorney for the Commonwealth and
defense counsel will want to be immersed in the policy-related discussions at the juvenile
probation department level and in other counties they will only want to be informed. Each
department will need to determine the level of stakeholder involvement desired at this
JJSES Support for End of Stage One
A listing of local data needed to be
collected for this Stage (forthcoming)
Action planning template (enclosed)
Organizational change and project management leadership training
Proficiency checklist to advance to
Stage Two
Technical assistance
JJSES | STAGE ONE
8
stage of the JJSES implementation. As significant policy decisions are made over time,
stakeholder involvement will likely need to increase.
3. Develop an eighteen month action plan. To address the issues identified in Stage One,
the department should initiate an action planning process with the goal of identifying the
concrete action steps that should be taken over the course of eighteen months. This
action plan will provide juvenile court personnel and stakeholders with a roadmap
regarding the what, who, and when of JJSES implementation. It will also give the staff a
clear picture of what JJSES entails and how their job functions will likely be affected over
the course of JJSES implementation. The action plan template is enclosed.
4. Administer a Stage One proficiency checklist. To help determine readiness to participate
in Stage Two of the JJSES the department is encouraged to complete a proficiency
checklist. This self-administered checklist includes a list of the activities to be completed
in the Initiation Stage and an assessment as to whether the department is ready to move
to Stage Two. The Stage One proficiency checklist is enclosed.
5. Request assistance for Stage Two. Once the proficiency checklist is completed,
department leadership should indicate its commitment to move ahead to Stage Two and, if
needed, request technical assistance (training, access to curricula/documents,
consultation services, etc.).
6. Organizational change and project management leadership training. JJSES will be
delivering a leadership training curriculum to assist Chief Probation Officers with practical
advice on how to successfully implement the JSES. This training will involve hands-on
workshops that include short knowledge building sessions and follow-up work sessions
where local JJSES issues are integrated into the curriculum exercises. This process will
make the training practical, in that it has real-time application and learnings that are
immediately transferrable to the JJSES effort.
JJSES Statement of Purpose
We dedicate ourselves to working in partnership to enhance the capacity of
Pennsylvania’s juvenile justice system to achieve its balanced and restorative
justice mission by:
• Employing evidence-based practices, with fidelity, at every stage of the
juvenile justice process;
• Collecting and analyzing the data necessary to measure the results of
these efforts; and, with this knowledge,
• Striving to continuously improve the quality of our decisions, services and
programs.
JJSES | STAGE ONE
9
Additional Resources.
None
Performance Measures
Forthcoming
JJSES | STAGE ONE
10
Cost Benefit Analysis Worksheet
Purpose: A department that makes a commitment to align its practice with research evidence
for risk reduction will likely experience significant challenges regarding how they invest their
resources. Certain functions and activities will need to be added, others dropped or modified.
Personnel requirements will likely change including how personnel are assigned, what staff
qualifications are needed, determining caseload and workload size, etc. In some cases,
activities and resources will need to be added such as those required when adopting screening
and assessment instruments or cognitive behavioral groups. Policies and practices with
stakeholders could change such as altering the format of predisposition reports to reflect
assessment information or modifying the expectations in service provider contracts. The cost-
benefit worksheet is designed to help administrators move ahead with their commitment toward
evidence-based practices with their “eyes wide open.” In this way, the department can plan for
resource shift or acquisition. It examines three primary areas: human, fiscal, and data
resources.
Instructions: It is recommended that the Chief Juvenile Probation Officer complete the cost
benefit analysis worksheet in conjunction with a team of department representatives (i.e.
supervisor, line juvenile probation officer, and support staff). Each representative will perceive
the items on the worksheet differently from their work experience perspective and will be able to
provide insight as to possible implications. It is important to fully explore these implications (i.e.,
costs and benefits) from each representative’s point of view, drawing upon lessons learned from
other counties and other jurisdictional experiences. If the department is just beginning to
embark on an EBP implementation process it may not be able to quantify some of the costs or
benefits. If this is the case, the department is urged to contact a Point of Contact (POC) to
request technical assistance.
As noted, it is likely not practical for most counties to quantify costs or benefits uniformly. An
urban county may view what is “costly” very differently than a rural county and vice versa. A
department whose caseload sizes are small might be able to transfer cases or staff more readily
than one with large caseloads. Therefore, the worksheet provides a subjective measure (“Low,
Moderate, High, or Not Applicable”) for both cost and benefit. The department will need to
determine what these terms mean to them. To assist in this interpretation, a brief description for
Low, Moderate, High or Not Applicable is provided below. Departments should consider only
local costs when completing the worksheet. While they may not be able to identify precise
dollar or time costs, it is important that departments enter an EBP planning process with full
awareness of what is involved and how it might impact resources.
The following table provides a definition of Cost and Benefit for purposes of completing the
cost/benefit worksheet.
JJSES | STAGE ONE
11
Cost (Human, Fiscal, and Data) Benefit (to Mission Outcomes) N/A
Low: Can be done with minimal
additional or existing resources
Description: The change will
have minimal or no impact
on a person(s) workload or
department budget
Low: Minimal or no real benefit to the
department’s mission will be achieved if
this activity is conducted
Description: The benefit is minimally
associated with the mission, or cannot
be articulated with clarity
Is perceived
to be not
applicable to
your
department’s
efforts to
implement
evidence-
based
practices.
Moderate: Will take some
additional resources, but current or
future resources or restructuring can
be used without requiring a
significant resource acquisition or
burden
Description: Can readily acquire
a grant or shift existing human
or fiscal resources
Moderate: Benefits can be articulated
but are not significantly contributory to
public safety/risk reduction or are
abstract and hard to quantify
Description: There is a
distinguishable benefit to the mission
and the benefit slightly outweighs
the cost
High: Will require a concerted effort
to bring on additional resources
Description: Will require the
securing of additional funding
from a source that has other
competing interests and/or the
approval of one or more
stakeholders to move
resources
High: Clearly, this activity will greatly
aid in achieving BARJ/JJSES outcomes
Description: It is very clear that the
benefit far outweighs the cost,
appears to be achievable, and would
deliver a message that the agency is
adopting an EBP approach
The cost benefit analysis team should gain a consensus regarding the cost and benefit of
implementing various EBPs. After the worksheet is completed, a discussion should ensue as to
whether the department is prepared to proceed with JJSES and, if so, how to plan for the
upcoming costs. If it is determined that the department is not prepared to proceed with
implementation of the JJSES, a plan should be developed to address what needs to occur
before the department is able to move forward with implementation.
JJSES | STAGE ONE
12
Stage One - Readiness (Prepare for JJSES Implementation)
Objective
Recommended Activities Costs
(low, moderate or high)
Benefit (low, moderate or
high) or N/A
Human
Fiscal
Data
The department
will have assessed
its organizational
readiness for EBP
and adequately
addressed any
shortcomings so
as to improve the
likelihood that a
change initiative
regarding JJSES
will succeed
-Conduct Likert readiness to change
survey
-Develop a response plan around
any item on the survey that showed
a significant gap score
-Secure a firm commitment in the
form of, for example, a letter of
support from leadership for JJSES
Statement of Purpose
The benefit should
be rated high if the
department has a
number of staff-
perceived issues
that are affecting
staff energy and
commitment (e.g.,
communication,
incompatibility with
mission, etc.)
A high knowledge
level about
evidence based
practices among
probation staff and
stakeholders will
be obtained
-Conduct intro to EBP training for
probation staff
-Train stakeholders, service
providers and victim advocates on
EBP
-Conduct pre/post EBP tests to
measure knowledge gained
This should be
rated high.
Without the
necessary
foundational
knowledge by all,
progress will not
be made.
The department
will develop a
detailed plan to
move ahead with
EBP
-Set up an action plan workgroup
(diagonal slice of department) and
complete an 18 month action plan
This should be
rated moderate. It
is very useful but
progress can be
made without it.
Begin process of
stakeholder
engagement
-Provide ongoing EBP education and
information to stakeholders
-Acquire a “letter of intent” to work
with the department on EBP from
key service providers
This should be
rated high (if
stakeholders are
reluctant to accept
JJSES changes;
moderate if less
reluctant).
JJSES | STAGE ONE
13
What is needed to accomplish Stage One?
Human
Resources
________________________________________________________________
______________________________________________________
Fiscal
Resources
________________________________________________________________
______________________________________________________
Data
Resources
________________________________________________________________
______________________________________________________
Stage Two – Initiation (Put in Place Assessment and Case Planning Processes)
Objective Recommended Activities
Costs (low moderate or high)
Benefit (low, moderate or
high) or N/A
Human Fiscal Data
Engage and
motivate the
juvenile
-Train all direct service staff on
Motivational Interviewing (MI) consistent
with recommended MI Training
protocols
-Conduct booster sessions and quality
assurance processes
High benefit. MI
is a core skill
needed to
improve
outcomes
Assess juveniles
for risk/need per
county policy
and at
recommended
intervals using
actuarial
screening and
assessment
instruments
Conduct
specialized
screening and
assessments, if
needed
-Train staff and utilize actuarial
screening and assessment instruments.
The instruments may include:
1. YLS/CMI
2. MAYSI~2
3. Detention Risk Assessment
-Assess juveniles per county policy and
at recommended intervals using
actuarial screening and assessment
instruments
-Establish a process to ensure
assessment inter-rater reliability
-Determine if additional screening and
assessment instruments are needed
(e.g., sex offender, domestic violence,
reading, learning disability, responsivity,
etc.) and, if so, provide staff training
and/or develop referral protocols with
providers of specialized assessments
High benefit.
Risk/need
assessments
(and the other
actuarial tools)
provide the
foundation by
which key
decisions are
made. The
same is true for
inter-rater
reliability
processes.
The additional
screening and
assessment
instruments are
of moderate
benefit.
JJSES | STAGE ONE
14
Develop a
comprehensive
diversion policy
for low risk
offenders
-Educate staff and stakeholders on the
“Guide to Developing Pre-Adjudication
Policy and Practice in PA”
-Establish protocols to divert
appropriate low risk offenders away
from the juvenile justice system
-Establish minimal supervision levels for
low risk offenders who remain involved
the juvenile justice system
High benefit.
Diversion
prevents
unnecessary
penetration into
the system and
retains probation
services for
higher risk
youth.
Complete case
plans on
medium and
high risk youth,
meeting SMART
guidelines
Address at least
the top three
criminogenic
needs prior to
discharge
Engage and
involve family in
case plan
-Train direct service staff on case
planning utilizing recommended case
plan template
-Create and distribute criminogenic
need response guidelines
-Conduct case audits
-Train staff on family involvement
strategies
High benefit.
Research has
shown that the
use of case
plans improves
outcomes. It
also provides
the department
with a means of
determining
whether the
criminogenic
needs are being
addressed.
Develop
professional
alliance
-Train direct service staff on
professional alliance (done in
conjunction with core competency
training)
-Administer the DRI-R, “Working
Alliance Inventory,” or exit survey
High benefit.
Without
professional
alliance, risk
reduction cannot
occur. The use
of the DRI, WAI,
or exit survey is
moderate
benefit.
Establish
caseload/
commensurate
with risk
-Develop policies to achieve caseload
size and supervision requirements
consistent with level of risk
High benefit.
JJSES | STAGE ONE
15
Prepare
supervisors for a
teaching and
coaching role
regarding EBP
-Train supervisors on effective coaching
and leadership on EBP with an
emphasis on the two competencies of
professional alliance and case planning
-Supervisors lead booster sessions on
the core competencies of professional
alliance and case planning
-Establish a “Transfer of Learning”
process
-Revise/align relevant policies and
human resource procedures to coincide
with stage changes
High benefit
(most of the
activities). One
of the key
reasons EBP
has not taken
hold in probation
departments is
because
supervisors have
not had the
knowledge or
comfort to
support and
guide staff.
Analyze key
data to monitor
progress
-Collect EBP process and outcome data
measures on Stage Two and distribute
in user friendly way
High benefit.
You cannot
know if you are
meeting
intended
outcomes
without
collecting the
data.
What is needed to accomplish Stage Two?
Human
Resources
_______________________________________________________________________
_______________________________________________________________________
Fiscal
Resources
_______________________________________________________________________
_______________________________________________________________________
Data
Resources
_______________________________________________________________________
_______________________________________________________________________
JJSES | STAGE ONE
16
Stage Three – Behavioral Change (Put in Place Processes That Reduce Rearrest)
Objective Recommended Activities Costs
(low, moderate or high)
Benefit (low, moderate or high)
or N/A
Human
Fiscal
Data
Youth learn skills
that help them stay
out of the justice
system
-Train staff on the skill practice and
rewards/sanctions competencies
-Train staff on the skill building
tools
-Train staff on cognitive behavioral
interventions
-Establish/expand cognitive
behavioral programming if needed
(recommend minimum of two CBT
groups - one for boys/one for girls)
-Develop strategies and
techniques to support gains made
by youth through cognitive
behavioral interventions (e.g.
Aggression Replacement Training,
Thinking for Change)
-Establish a QA process that
includes direct observation or tape
review of staff interactions
High benefit. While
community service
providers can
greatly assist in
achieving risk
reduction the
research is clear
that the one-on-one
sessions with
probation staff can
likewise achieve
these outcomes. In
addition, probation
staff can reinforce
the skills taught by
service providers.
Established a
structured decision
making process for
rewards and
sanctions
-Establish a planning process to
establish the research, values,
policies, and procedures for a
structured decision making
process (including stakeholder
collaboration)
Moderate benefit. It
definitely adds
value to outcomes
but perhaps slightly
less than the other
techniques noted
above.
Further engage
stakeholders
-Determine how court related
process may need altering (e.g.,
disposition report, admissions,
violations, discharges)
High to low benefit
depending on the
nature of existing
processes. If these
processes are in
alignment with
JJSES then the
JJSES | STAGE ONE
17
-Provide additional training if
needed
benefit would be
low; if not, it would
be high benefit.
Align community
based services with
EBP
-Conduct a service gap analysis
using risk/need information
-Enter into an arrangement to
improve and/or add contracted
services per the gap analysis
-Collaborate with providers
regarding SPEP scores and
program improvement plans
High to low benefit
depending on the
alignment of current
community based
services. High if
the effort would
improve services;
low if services are
largely in alignment
with EBP.
Prepare supervisors
for a teaching and
coaching role around
EBP
-Train supervisors on effective
coaching and leadership on skill
practice and rewards/sanctions
-Supervisors lead booster sessions
on the core competencies of skill
practice and rewards/sanctions
High benefit. See
above discussion of
supervisors’ role.
Establish an EBP
culture
-Provide artifacts (checklists,
posters, etc.) that support skill
practice and EBP activities
-Revise/align relevant policies and
human resource procedures to
coincide with stage changes
Moderate benefit.
Analyze key data to
monitor progress
-Collect EBP process and outcome
data measures on Stage Three
and distribute in user friendly way
High benefit. You
cannot know if you
are meeting
intended outcomes
without analyzing
the data.
What is needed to accomplish Stage Three?
Human
Resources
_______________________________________________________________________
_______________________________________________________________________
Fiscal
Resources
_______________________________________________________________________
_______________________________________________________________________
Data
Resources
_______________________________________________________________________
_______________________________________________________________________
JJSES | STAGE ONE
18
Stage Four – Refinement (to Solidify Department Use of Knowledge to Inform Decisions)
Objective Recommended Activities Costs
(low, moderate or high)
Benefit (low, moderate or high)
or N/A
Human
Fiscal
Data
Become an
enduring “Learning
Organization”
-Develop a plan to continuously
acquire and disseminate new
research findings
Moderate Benefit.
Refine existing
policies to ensure
alignment with EBP
-Ensure personnel are recruited
with the types of EBP
competencies the department
seeks - consistent with the 14
Professional Alliance Traits
-Ensure future department
training is directly linked to the
required EBP competencies
-Ensure mission statement,
policy and procedures, job
descriptions, performance
evaluations, promotions, and
staff rewards are linked to EBP
related performance (e.g.,
addressing criminogenic needs)
instead of process only (e.g.,
meeting contact standards)
High Benefit. Few
things are more
important than
ensuring that existing
staff attitudes/beliefs
and skills are in
alignment with the
tasks required under
EBP. A department
must also ensure that
its reward system and
other infrastructure
are consistent with
EBP or else staff will
tend to gravitate to
that which has
traditionally been
valued.
Ensure information
system capacity to
evaluate and
research local
practices
-Analyze information system for
needed improvements
-Develop and disseminate
scorecard for purpose of
individual, department, and
system wide feedback
(customized to the target
audience)
-Evaluate key programs for
effectiveness (possibly with local
University assistance)
High benefit. You
cannot know if you
are meeting intended
outcomes without
analyzing and
distributing the data.
JJSES | STAGE ONE
19
What is needed to accomplish Stage Four?
Human
Resources
_________________________________________________________________
_____________________________________________________
Fiscal
Resources
_________________________________________________________________
_____________________________________________________
Data
Resources
_________________________________________________________________
_____________________________________________________
JJSES | STAGE ONE
20
EBP Knowledge Pre/Post Test
Instructions: The EBP knowledge test is designed to be given prior to the Intro to EBP training
and immediately after the training. The same test is administered both times in order to
compare the number of questions answered correctly on the pretest to those answered on the
post test. Participants should be told to not put their names on the test, as it is anonymous. It is
more important that participants answer to the best of their ability and give a true picture of their
knowledge than for them to feel any pressure to “get it right.” The testing is designed to identify
- Whether the training had its intended impact (increased knowledge); and
- What knowledge gaps exist for which booster sessions may be necessary
Participants should be given the pretest before the trainer starts teaching. The facilitator or
trainer should explain the purpose for the testing and that it is anonymous. Test results should
be collected before the training begins. After the training ends, participants should be re-
administered the same test. At the end of the training, the facilitator or trainer should remind
participants to check the “post-test” box on the test to distinguish given answers from the
pretest. These post-tests should then be collected and the number of questions answered
correctly compared to those answered correctly on the pretest. As a group, scores should
improve after the training for each question that was answered correctly by less than 90% of the
pretest participants. The goal is to achieve an 80% or higher correct response rate for each test
question. If, after the pre and post-test results are analyzed, the scores do not meet the 80%
threshold the department should determine the best method of improving results. Some
possible responses include
Conduct a short EBP booster session(s) (e.g., 30 minutes)
Have staff read articles on the topic in need of improved knowledge
Have supervisors conduct coaching sessions individually or in staff meetings
Note: the pre and post-test should be administered as written. However,
under Part I: General, question B it asks the participant to indicate their job
category. This is provided to give the department an understanding of which
job functions might need additional assistance. You will need to fill in the
categories of job functions that match your department’s position titles.
JJSES | STAGE ONE
21
EBP Knowledge Pre/Post Test
This survey is anonymous; please do not include your name on the materials. Please select only one response per question.
Part I: General
A. Please indicate the point in time you are taking this survey:
a) Before the training (pre-training survey) b) After the training (post-training survey)
B. Your Job Category:
a) To be filled in: _______________________ b) To be filled in: _______________________ c) To be filled in: _______________________ d) To be filled in: _______________________
C. How long have you been employed by your current employer?
a) Up to one 1 year b) 1-3 years c) 4-10 years d) Over 10 years
Part II: EvidenceBased Practices For each of the questions below, please select one response only. 1. Which of the following interventions best reduces recidivism in the long term?
a) Boot camps b) Wilderness programs c) Escalating sanctions d) Cognitive behavioral programs
2. Which is not a criminogenic need?
a) Depression b) Antisocial peers c) Low victim empathy d) Unemployment
3. The lack of education is among the top four criminogenic needs.
a) True b) False
JJSES | STAGE ONE
22
4. Responsivity means: a) Being responsive to the needs of juveniles b) Using actuarial risk/need assessment tools c) Focusing on juvenile strengths d) Matching interventions to the individual traits of the juvenile
5. It is generally true that treatment can be effective, and therefore, it is more important to provide a lot of treatment than to be concerned with the type of treatment.
a) True b) False
6. Placing juveniles with low self-esteem in programs that increase their confidence is an effective way to reduce recidivism.
a) True b) False
7. It is generally true that most treatment does not work in reducing recidivism. Therefore, dispositions and case plans should focus on holding the juvenile accountable and restoring the crime victim rather than treatment.
a) True b) False
8. Which of the following programs works in reducing recidivism over the long term?
a) Gardening and horticulture b) Yoga c) Lectures designed to give insight d) AA e) None of the above
9. Which of the following program components contribute the least to reducing recidivism?
a) Juveniles rehearsing skills learned in programs b) Staff role modeling c) Positive reinforcement d) Programs designed to provide insight
10. Which is most important in lowering recidivism?
a) Lower workload for staff b) Providing staff with training and skills c) Getting more resources d) Use of external control
11. Mixed gender treatment groups:
a) Are effective for role modeling and controlled exposure to mixed gender issues b) Tend to produce more harmful than positive results c) Tend to produce neither harmful nor positive results d) None of the above
JJSES | STAGE ONE
23
12. Evidence-based practices are not used in the fields of: a) Medicine b) Nursing c) Corrections d) Plumbing e) None of the above
13. It is important to give juveniles positive reinforcement and feedback when they are prosocial if your goal is risk reduction.
a) True b) False
14. Swift and certain responses to acts of noncompliance are important if your goal is risk reduction.
a) True b) False
15. What is the number of positive reinforcing statements that must be given before an individual will take in a corrective comment?
a) 1-2 b) 2-4 c) 4-6 d) 7 or more
16. The use of a correctional institution for higher risk juveniles is effective in changing future illegal behavior after release if the confinement conditions are severe enough so that the juvenile doesn’t want to return.
a) True b) False
17. Placing a juvenile who has used or been involved with drugs in drug education classes is an effective method of reducing drug use in the future.
a) True b) False
18. Putting lower risk offenders in a cognitive behavioral class will most likely:
a) Help settle the class down by providing positive role models for the higher risk offenders b) Increase the lower risk offenders' recidivism rates
JJSES | STAGE ONE
24
EBP Knowledge Pre/Post Test
Answer Key
1. D
2. A
3. False
4. D
5. False
6. False
7. False
8. E
9. D
10. B
11. B or C
12. E
13. True
14. True
15. C
16. False
17. False
18. B
JJSES | STAGE ONE
25
Organizational Readiness Survey
Instructions: The organizational readiness survey was derived from the research of Rensis Likert. He is one of the founders of business management theory, a psychologist, and best remembered for his research in organizational management and the development of the Likert Survey Scale. Likert divided management styles into four systems which provide the overall structure for the Likert Organizational Climate Survey. The four systems described by Likert are: 1. Exploitive authoritative system In this type of management system, the job of employees/subordinates is to abide by the decisions made by managers and those with a higher status than them in the organization. The non-management staff do not participate in decision-making. The organization is concerned simply about completing the work. The organization will use fear and threats to make sure employees complete the work set. There is no teamwork involved. 2. Benevolent authoritative system Just as in an exploitive authoritative system, decisions are made by those at the top of the organization and management. However employees are motivated through rewards (for their contribution) rather than fear and threats. Information may flow from subordinates to managers but it is restricted to "what management wants to hear". 3. Consultative system In this type of management system, non-management staff are motivated by rewards and a degree of involvement in the decision making process. Management will constructively use employee ideas and opinions. However, involvement is incomplete and major decisions are still made by senior management. There is a greater flow of information (than in a benevolent authoritative system) from staff to management. The information from staff to manager is often incomplete and euphemistic. 4. Participative system Management has complete confidence in their employees. There is significant communication and staff are fully involved in the decision making process. Employees comfortably express opinions and there is lots of teamwork. Teams are linked together by people, who are members of more than one team. Likert calls people in more than one group "linking pins". Employees throughout the organization feel responsible for achieving the organization’s objectives. This responsibility is motivational, especially as staff are offered rewards for achieving organizational goals which they have participated in establishing. An organization needs to be high functioning in a limited number of areas in order for a change effort to be successful. Without attending to these areas, agencies are more likely to experience perfunctory change “on paper” instead of actual modified staff activities. The fourth system, participative, is described as the ideal system for human-concerned organizations. In this system, leadership has confidence in their staff, personnel at all levels feel real responsibility for organizational goals, there is strong communication, and a substantial amount of cooperative teamwork.
JJSES | STAGE ONE
26
The organizational readiness survey, then, is a tool to determine staff perception as to which of the four management systems the department operates. It does this by comparing what the staff member thinks is true of an ideal organizational environment versus what currently exists. Where there are large gaps, the department will want to develop a plan to address them. The survey should be administered anonymously. Carefully designed collection procedures should be put in place to ensure anonymity. Some departments use survey monkey, others use an outside agency, yet others use a “drop box.” The technique employed should be determined based on the level of concern around anonymity and department history around trust and use of surveys. The surveys should be compiled in such a way as to identify “gap scores.” That is, the questions have a range of four answers. The largest gaps (which occur when the respondent indicates that the first choice represents the current situation and the fourth choice represents the idea situation) should be attended to first. To quantify gap scores, the first answer is given a numerical value of 1, followed by 2, 3, and then 4. Gaps that approach two points (or higher) should be attended to.
JJSES | STAGE ONE
27
Organizational Readiness Survey
For the following questions, address both your perception of the current situation in the organization, as well as your perception of what the ideal for this organization would be. Please select one response only for each the current situation and the ideal situation by circling your perspective of the “current” and “ideal” situation. Example: How much confidence and trust is shown in staff members by leaders in the department?
Current Situation
Ideal Situation
1. How much confidence and trust is shown in staff members by leaders in the department?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
Virtually None
2
Some
3
A Substantial Amount
4
A Great Deal
1
Virtually None
2
Some
3
A Substantial Amount
4
A Great Deal
1
Virtually None
2
Some
3
A Substantial Amount
4
A Great Deal
1
Virtually None
2
Some
3
A Substantial Amount
4
A Great Deal
JJSES | STAGE ONE
28
2. How free do staff members feel to talk to leaders about their job?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 3. How often are staff members' ideas sought and used constructively?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 4. How predominant is the use of fear, threats, or punishment?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
Not Very Free
2
Somewhat Free
3
Quite Free
4
Very Free
1
Not Very Free
2
Somewhat Free
3
Quite Free
4
Very Free
1
Seldom
2
Sometimes
3
Often
4
Very Frequently
1
Seldom
2
Sometimes
3
Often
4
Very Frequently
1
Very Frequently
2
Often
3
Sometimes
4
Seldom
1
Very Frequently
2
Often
3
Sometimes
4
Seldom
JJSES | STAGE ONE
29
5. How predominant is the use of rewards and incentives?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 6. Where is the responsibility felt for achieving the department's goals?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 7. How much cooperative teamwork exists in the department?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
Seldom
2
Sometimes
3
Often
4
Very Frequently
1
Seldom
2
Sometimes
3
Often
4
Very Frequently
1
Mostly at the Top
2
Top and Middle Management
3
Line Level
4
At All Levels
1
Mostly at the Top
2
Top and Middle Management
3
Line Level
4
At All Level
1
Very Little Cooperative Teamwork Exists
2
Cooperative Teamwork Occurs on Situational
Basis
3
Cooperative Team work Occurs Among Select
Groups of Staff
4
Ongoing and Consistent Cooperative Teamwork Exists Among All Staff
1
Very Little Cooperative Teamwork Exists
2
Cooperative Teamwork Occurs on Situational Basis
3
Cooperative Teamwork Occurs Among Select
Groups of Staff
4
Ongoing and Consistent Cooperative Teamwork
Exists for All Staff
JJSES | STAGE ONE
30
8. What is the usual direction of information flow?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 9. How is downward communication accepted/received by staff members?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 10. How accurate is upward communication?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
From Top to Bottom
2
From Bottom to Top
3
Sideways (Only Within Same Hierarchal Level)
4
In All Directions
1
From Top to Bottom
2
From Bottom to Top
3
Sideways (Only Within Same Hierarchal Level)
4
In All Directions
1
Negatively
2
With Suspicion, but not necessarily
negative
3
Neutrality
4
With a Receptive Mind
1
Negatively
2
With Suspicion, but not necessarily
negative
3
Neutrality
4
With a Receptive Mind
1
Usually Inaccurate
2
Often Inaccurate
3
Often Accurate
4
Almost Always Accurate
1
Usually Inaccurate
2
Often Inaccurate
3
Often Accurate
4
Almost Always Accurate
JJSES | STAGE ONE
31
11. How well do leaders in the department understand the problems faced by staff
members?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 12. At what level are decisions affecting the entire department made?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 13. Are staff members involved in decisions related to their work?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
Poorly
2
Not Very Well
3
Well
4
Very Well
1
Poorly
2
Not Very Well
3
Well
4
Very Well
1
Always at the Top without Staff Input
2
Always at the Top/Middle without
Staff Input
3
At the Top/Middle with Staff Input
4
Throughout the Organization
1
Always at the Top without Staff Input
2
Always at the Top/Middle without
Staff Input
3
At the Top/Middle with Staff Input
4
Throughout the Organization
1
Almost Never
2
Occasionally Consulted
3
Generally Consulted
4
Fully Involved
1
Almost Never
2
Occasionally Concsulted
3
Generally Consulted
4
Fully Involved
JJSES | STAGE ONE
32
14. How are the department's goals established?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 15. How much covert resistance to the department's goals is present?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 16. Are there informal groups of staff resisting the direction/goals of the department?
Current Situation
Ideal Situation
Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________
1
Orders are Issued Without Asking for Input
2
Orders are Issued With Limited Input Invited
3
Management Decides With Staff Paricipation
4
By Group Action (Except in Crisis Situations)
1
Orders are Issued Without Asking for Input
2
Orders are Issued With Limited Input Invited
3
Management Decides With Staff Participation
4
By Group Action (Except in Crisis
Situations)
1
Strong Resistance
2
Moderate Resistance
3
Some Resistence At Times
4
Little or No Resistance
1
Strong Resistance
2
Moderate Resistance
3
Some Resitance At Times
4
Little or No Resistance
1
Yes, Most of the Time
2
Frequently
3
Sometimes, but Infrequently
4
No, or Almost Never
Yes, Most of the Time
2
Frequently
3
Sometimes, but Infrequently
4
No, or Almost Never
JJSES | STAGE ONE
33
17. How is data most commonly used in the department?
Current Situation
Ideal Situation
Comments:______________________________________________________________________________________________________________________________________________________________________________________________________________
1
To Admonish/Discipline/
Punish Staff
2
To Identify Areas in Need of Improvement
3
To Take Constructive Corrective Action If
Needed
4
To Reward/Incentivize Staff
1
To Admonish/Discipline/
Punish Staff
2
To Identify Areas in Need of Improvement
3
To Take Constructive Corrective Action If
Needed
4
To Reward/Incentivize Staff
JJSES | STAGE ONE
34
Action Planning
Instructions: The department can use any of a number of action planning templates to
develop their EBP action plan. The attached templates are available for those who do not have
an existing, preferred form. It is recommended that the department complete an eighteen
month action plan after the other Stage One activities have been completed. The following are
recommended steps:
One Assemble an action planning team made up of no less than five staff members
and no more than fifteen. Make sure that the team represents a cross section of
staff functions including but not limited to, management, intake, supervision staff,
and service coordinators. Subcommittees may be formed.
Two Assign a chairperson. The chair should be someone that the other members
respect, knows how to facilitate a meeting, has good follow through, is objective,
is inclusive, and does not have an “agenda.”
Three Develop clear expectations of the group including, for example, length of time it is
to meet, objectives, communication protocol, decision making authority, etc. In
some departments, a committee charter is developed.
Four Create a staff input plan. If all staff cannot participate in the action planning
workgroup, determine the best method to gain staff input. Some ways might
include surveys, focus groups, topic specific invitees to meetings,
subcommittees, or asking for input on draft documents.
Five Determine, based on the JJSES Monograph, what goals, objectives and activities
should be accomplished in the next eighteen months that will ensure that the
department is achieving its JJSES objective. The plan should include the
person(s) taking the lead on each activity and timelines.
Six Use JJSES technical assistance to develop the plan, as needed.
Seven Write the plan and disseminate it to the entire department.
JJSES | STAGE ONE
35
Action Plan Templates
Example A: Goal 1: Utilize validated screening and assessment instruments to identify risk and need of juvenile offenders and to develop case plans that match the appropriate level of intervention.
What should be done?
Objectives
How will it be done?
Strategies or Activities
Who will take the lead?
Date objective to be done?
How will we know it’s done?
Outcome Measure A:
B:
C:
JJSES | STAGE ONE
36
Example B:
Goal: To increase juvenile and family and family engagement through strength-based techniques.
Objective 1:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
Objective 2:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
Objective 3:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
Goal:
Objective 1:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
Objective 2:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
Objective 3:
Action Item Responsibility Deadline Completed
a)
b)
c)
d)
JJSES | STAGE ONE
37
Example C:
Action Plan Template
Purpose: To create a “script” for your JJSES efforts and support implementation. Directions: 1. Use this form as a template, develop an action plan for each goal identified.
2. Modify the form as needed to fit your unique context. 3. Distribute copies of each action plan to the members of the department. 4. Keep copies handy to bring to meetings to review and update regularly. You may
decide to develop new action plans for new phases of your JJSES efforts.
Goal: Identify and target proven and effective interventions based on criminogenic risk and needs of juvenile offenders. Results/Accomplishments:
Action Steps
What Will Be Done?
Responsibilities
Who Will Do It?
Timeline
By When?
(Day/Mo)
Resources
A. Resources Available
B. Resources Needed (financial, human, political & other)
Potential Barriers
A. What individuals or organizations might resist?
B. How?
Communications Plan
Who is involved? What methods?
How often?
Step 1:
A.
B.
A. B.
Step 2:
A.
B.
A. B.
Step 3:
A.
B.
A. B.
Step 4:
A.
B.
A. B.
Evidence Of Success (How will you know that you are making progress? What are your benchmarks?) Evaluation Process (How will you determine that your goal has been reached? What are your measures?)