enhancing practice in work with offenders: the role of evaluation jean hine, de montfort university
TRANSCRIPT
Enhancing Practice in Work with Offenders: the Role of Evaluation
Jean Hine, De Montfort University
Workstream 1: Aim
• To learn:– Better understanding of what evaluation is
undertaken, who does it, how, why, with what result
– What facilitates/inhibits– What might help
• To develop– Evaluation guidance/toolkit– Library of examples of evaluation practice
Four Workstreams
Good and Effective Practice
3. Literature Review
1. Evaluation 4, EPR
2. SEED
• Interviews across the EU• Mapping of evaluation• Working with 2 pilot evaluations• Co-ordination with workstreams 2, 3
and 4• Development of evaluation
guidance
Workstream 1: Components
Mapping
• More scoping than mapping.• Not comprehensive but does
include good range countries, organisations, individuals
• Complexity – language and meaning– range organisations involved– structuring of work with offenders in
different countries
Evaluation takes many forms
• Specific independent research studies• MoJ national studies/reviews• Monitoring/management information• Target assessment• Compliance • Project outcomes• Individual casework• Serious case reviews
Who evaluates• Some countries have strong
commitment to evaluation and active programme, some do not see evaluation as important
• Most countries have some governmental level unit undertaking some data work, some basic statistical, some special centres/units
• Most countries have some university involvement but approach varies by discipline
• Some use of independent organisations
What is evaluated?
• Specific programmes and practices– Electronic monitoring, thinking skills,
domestic violence, sex offender programmes, etc
• Much focus on recidivism/outcomes– But different abilities to assess
• Competition with prison evaluation• Generally short term impact
What’s not?
Limited:• Systematic evaluation of general supervision• Service user evaluation• Detailed analysis of routinely collected data
Key learning from Mapping
• Wide variation• Most evaluation is centrally led– Central commitment important to enable– But prioritises central concerns– Tends to be large scale and quantitative
• There are interesting pieces of work which need to be more widely known
• General commitment to a desire for more evaluation, sometimes externally driven
Facilitators for evaluation
• Central commitment• External pressures– EU– Funders– Government threat
• Individual enthusiasm
Inhibitors for evaluation
• Cultural/practitioner resistance• Limited Resources– Time as well as money
• Lack of skills/knowledge• Lack of interested audience• Overestimation of the task• Limited awareness of possibilities• Lack of clarity about objectives of practice
Pilot work• Shared process• Working together• In some ways artificial but beneficial– for pilots– for guidance
• Useful differences and similarities• Two partners
Italy
• NGO, large scale drug rehabilitation community• Offending not a direct focus but desistance an
anticipated benefit• Formative evaluation• How to retain more alternative sentencees at end of
sentence• Mixed approach– Focus groups– Data analysis
• Highlighted importance of shared language
England
• Social enterprise offering peer support for employability
• Offending not direct focus but an anticipated benefit
• Formative evaluation• How to improve success of specific project
with probationers
Common issues
• Identifying evaluation question• Data system limitations• Designing manageable project• Resources• Time frame• Insider role tension• Support • Changing context
Pilots’ Focus Group
• Right time to evaluate• Political context• Availability of respondents• Rigour and confidence• Ethical issues• Written guidance too ‘academic’• Cop or coach?• Critical friend/benevolent challenging• Evaluation is time and labour intensive (but worth it!)
How does this inform guidance?
• Clarity• Flexibility• Range of uses/users• Different starting points• Making the most of existing data• Making evaluation manageable
International comparability difficult
• Differing terms and meanings• Differing criminal justice processes• Different criteria for sanctions• Different legal contexts
Example of Recidivism
• Usually means reconviction• Availability of data:– Routine standard collection or not– Data protection laws
• Definition of conviction– Legal frameworks
• Time frames– Judicial procedures
The Hope
Quality evaluation
Sound evidence
Informed practice
Good outcomes