action for prisoners’ families seminar 25 june 2014 quality services: ticking all the boxes
TRANSCRIPT
Action for Prisoners’ Families Seminar 25 June 2014
Quality services: ticking all the boxes
Evaluation and family relationship measures
Anna Kazimirski
v
EVALUATION AND FAMILY RELATIONSHIP MEASURESANNE KAZIMIRSKI, NPC
Action for Prisoners’ Families – 25th June 2014
TRANSFORMING THE UK CHARITY SECTOR
4
NPC works at the nexus between charities and
funders
Charity
SectorFunder
Increasing the impact of charities
eg, impact-focused theories of change
Strengthening the partnership
Eg, collaboration towards shared
goals
Increasing the impact of funders
eg, effective commissioning
ConsultancyThink tank
WHY MEASURING IMPACT IS IMPORTANT
5
The Brandon Centre provides counselling and psychotherapy to
young people between the ages of 12 and 21
Influences the debate on
“what works”Improved services
Raises profile
Motivates staff
Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater Impact New Philanthropy Capital.
Helps secure funding
COMPONENTS OF NPC’S APPROACH TO MEASURING SOCIAL IMPACT
6
Map your theory of change
Prioritise what you measure
Choose your level of
evidence
Select your sources and
tools
Effective measurement framework developed
Strategic visionLeadership
Case for impact measurement
THEORY OF CHANGE
Links activities intermediate outcomes final outcomes
clarifies what the activities aim to achieve and how
provides a structure for identifying what can be measured
provides the case for why achieving intermediate outcomes is important
7
A conceptual map of how activities lead to outcomes
EXAMPLE THEORY OF CHANGE
8
Family therapy
Levels of conflict within families
reduced Observable
reduction in problem behaviours during
process
Offenders (and families) feel
empowered to take responsibility and
maintain gains
Strengthened family ties & relationships
Contribution to reduced reoffending
Reintegration into (non criminal) social & family
groups
OFFENDERS’ FAMILY RELATIONSHIPS
9
• Commissioned by NOMS to develop toolkit to measure intermediate outcomes of work to improve family & peer relationships of offenders
• Project going through peer review, to be published later this year
• Parallel project: RAND Europe, with ARCS (UK) and University of Glamorgan, focusing on arts and mentoring work with offenders
10
PROJECT PROCESS
Evidence review – identifying outcomes and toolkits
Consultation with providers & commissioners to select outcomes to focus on
Development and piloting of toolkit
Consultation with providers & commissioners to inform guidance
Analysis and reporting
DEVELOPMENT OF RELATIONSHIPS TOOLKIT
• Desistance theory:
• Link between creation/ investment in family relationships / reduction in peer pressure with reduced offending
• Consultation:
• Outcomes differed depending on the type of intervention & beneficiary
• Needed to reach a balance between being prescriptive and allowing for flexibility
• We researched existing scales and evaluations in each outcome area and prioritised:
• good validity and reliability; recently developed; brevity.
12
Toolkit: 25 standardised scales, to be used in paper questionnaires for offenders to complete.
For each outcome area: short general scales & longer specialised scales.
TOOLKIT STRUCTURE
13
Child relationships
Wider family relationships
RAND personal
development outcomes
Peer relationships
Partner
relationships
Reduced reoffending
+
+ +
+
++
Parenting
OUTCOME AREAS
Partner relationships
Increased satisfaction with partner relationship, reduced conflict and improved commitment
Child relationships
Quality of relationships with children, children’s well-being and behaviour
Wider family relationships
Family functioning, resilience, conflict and communication, satisfaction with relationships in the family and quality of relationships
Parenting
Improved satisfaction with the parent-child relationship and parenting skills
Peer relationships
Satisfaction with peer relationships and reduction of negative peer relationships
14
CHARACTERISTICS OF TOOLKIT
• The questions can be used as monitoring or as part of an evaluation with a ‘before and after’ (pre/post) research design, with a comparison group where possible
15
Before measure
After measure
Programme / service
Before measure
After measure
“Intervention” group
“Comparison” group
RESOURCES
http://www.thinknpc.org/
NPC’s Wellbeing measure: http://www.well-beingmeasure.com
NPC’s papers on
How to approach evaluation: http://www.thinknpc.org/publications/npcs-four-pillar-approach/
Theory of Change: http://www.thinknpc.org/publications/theory-of-change/
How to communicate your results: http://www.thinknpc.org/publications/talking-about-results/
Profile of six charities who radically improved their approach to impact measurement: http://www.thinknpc.org/publications/a-journey-to-greater-impact/
http://inspiringimpact.org/
Code of Good Impact Practice
Online impact marketplace
020 7620 4855
Twitter: @npcthinks, @AnneKazimirski
16
Action for Prisoners’ Families Seminar 25 June 2014
Quality services: ticking all the boxes
www.prisonersfamilies.org.uk @Prisonerfamily