what gets measured gets done presented by frances head george elliott
TRANSCRIPT
What Gets Measured Gets Done
Presented by
Frances Head George Elliott
Many organisations fail to optimise the outcomes that they achieve.
This is in part due to the lack of a strategic approach to evaluation and planning.
Not About Policing
Organisations that measure their outcomes are much more likely to improve performance over time.
Whitelion and Social Responsibility Network have worked to together to build a multi-layered evaluation framework.
This presentation discusses the methodology for the development of the framework and the implementation process.
The evaluation framework that was developed for Whitelion is based on Results-Based Accountability™ (RBA).
RBA has been used as an effective planning and evaluation tool for community and organisational development for nearly twenty years.
It had its beginnings in the USA and is now used internationally in over 15 countries.
RBA asks three important questions:• How much did we do• How well did we do it• Is anyone better off
The standard RBA model is two dimensional
Firstly, it looks at information for a given population group; determines some aspirational goals; and establishes a set of measures that reflect progress toward these goals.
Secondly, it looks at how the specific client group is faring in relation to the population and how to improve outcomes.
In order to do this a set of Performance Measures needs to be established.
This is done through a process that encourages prioritisation and collaborative partnerships.
One of the most important aspects of the model is that it is intended to encourage continuous improvement.
This is partially facilitated by the fact that performance measures are monitored and charted over time.
2009 2010 2011 2012 2013 2014
0
10
20
30
40
50
60
70
80
90
100
Work Experience Placements Baseline Prediction
% of work experienceclients placement inemployment
The evaluation framework that was developed for Whitelion is intended to be multi-dimensional and to demonstrate the outcome relationships at a number of levels within the organisation.
A strategy for gathering information at the individual client level was needed in order to complete the framework.
Based on research done by Jenny Crosbieand Goetz Ottman:
What are the most important issues in the lives of people with intellectual disabilities? A comparative Study
Photovoice
Photovoice can be defined as a method of seeing the world through the viewpoint of people who are leading different lives (Wang and Burris, 1994)
Cameras are provided to members of underrepresented groups, and a “voice’ is created through the photographs that are taken.
The viewpoints of these underrepresented groups can be eye-opening since they often contrast with the majority of viewpoints found in society.
There are three main goals
1. Enabling people to record and reflect upon their strengths, goals and concerns
2. Promote critical dialogue and knowledge about important issues
3. Reaches important decision makers
By combining methodologies we were able to achieve a more holistic approach to performance evaluation, planning and continuous improvement.
Population
OUTCOMESStable & secure
housingSustainableemployment
Sustainableconnection to
community of choice
Economicsustainability
Being healthyEducationalattainment
Social impact
INDICATORS
Homelessness rateLength of time inparticular housingsituation
Length of time inemploymentUnemployment ratesParticipation ratesPeople on benefits
Sport & recreationChurchCultureSchool attendance
Payment received -benefitsSavings rates
ATOD ratesDomestic violenceratesLife expectancy
Educational levelRate of early schoolleaversRe-engagementrates
Community Employers Peers Families
Organisation
OUTCOMESStable & secure
housing
Sustainableemployment
Sustainableconnection to
community of choice
Economicsustainability
Being healthyEducationalattainment
Social impact
PERFORMANCEMEASURES
Homelessness rateAccess to servicesLength of time in aparticular housingsituation
Length of time inemploymentUnemploymentrates
Payment received -benefitsEconomicindependence
ATOD ratesDomestic violenceratesLife expectancy
Educational levelRate of earlyschool leaversRe-engagementrates
Community Employers Peers Families
Programs
MentoringPERF MEASURES
% long-termrelationshipsbeyond Whitelion
Youth Programs -Young Women
PERF MEASURES
% able to make acommitment andfollow through
% return to trainingand furthereducation
Youth Programs -Leadership Program
PERF MEAURES
% go on to 2ndyear
% meet goals
EmploymentProgram
PERF MEAURES
% young peoplematched toemployment
% recidivism
IndigenousEmployment
ProgramPERF MEAURES
% gain sustainedemployment
% staff, externalparties, clientsmore culturallyaware andcompetent
% reduce ATODuse
% who gainqualifications
% who re-offend
Open FamilyOutreach - West
PERF MEAURES
# YP housedduring the year
% reunited withfamilies
How much materialaid per type
% with improvedhealth
Open FamilyOutreach - SouthPERF MEAURES
Occupancy rate
IndividualOutcoomes
PERF MEASURES
OUTCOME AND PERFORMANCE MEASURE EVALUATION FRAMEWORK
Framework ImplementationFramework Implementation
National Staff Workshops
Assess current available data and fill holes
Build an evaluation module and relevant reports
Design and implementation of a Well Being Index & Client Surveys
Volunteer and stakeholder feedback
Implementation of the framework through existing structures e.g. staff meetings, K.P.I’s and work plans.
Assess available data and fill gapsAssess available data and fill gaps
https://www.notforprofitsystems.com
Case Study - Employment ProgramCase Study - Employment Program
Data Collection
# of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young peopleReasons young people disengageEmployers with the best outcomes
• Data input by staff.• Reports are run and
provided to all staff.
Performance Measure
% of young people engaged in the program who will be placed into full time employment.
Average length of employment
• Reports are provided to program managers.
• New targets are set and ways to improve process are discussed in team meetings.
• This inturn increases professionalism and encourages continuous improvement.
Client outcomes
WBI show increases in the domains; standard of living, achievements in life and community connection.
Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security.
• Informs program practise.• Provide for reporting and
funding purposes.• Staff engagement and
morale.
Build an evaluation module and develop relevant reportsBuild an evaluation module and develop relevant reports
Case Study- Employment ProgramCase Study- Employment Program
Data Collection
# of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young peopleReasons young people disengageEmployers with the best outcomes
• Data input by staff.• Reports are run and
provided to all staff.
Performance Measure
% of young people engaged in the program who will be placed into full time employment
Average length of employment
• Reports are provided to program managers.
• New targets are set and ways to improve processes are discussed in team meetings.
• This in turn increases professionalism and encourages continuous improvement.
Client outcomes
WBI show increases in the domains; standard of living, achievements in life and community connection.
Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security.
• Informs program practise.• Provide for reporting and
funding purposes.• Staff engagement and
morale.
All clients will complete a Well Being Index at entry point to the program
Well Being Index & Client Survey; Design and implementation
Once per month, SYARD (client management system) will select 20 random young people from the system to re-survey. They will be re-complete the Well Being Index and respond to questions regarding service delivery.
As a direct result of services received:
Strongly agree
Agree Neutral Disagree Strongly Disagree
Not Applicable
I am more confident 1 2 3 4 5 6
I am more independent 1 2 3 4 5 6
I am better at maintaining relationships
1 2 3 4 5 6
Well Being Index & Client Survey; Design and implementation
Case Study- Employment ProgramCase Study- Employment Program
Data Collection
# of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young peopleReasons young people disengageEmployers with the best outcomes
• Data input by staff.• Reports are run and
provided to all staff.
Performance Measure
% of young people engaged in the program who will be placed into full time employment
Average length of employment
• Reports are provided to program managers.
• New targets are set and ways to improve process are discussed in team meetings.
• This inturn increases professionalism and encourages continuous improvement.
Client outcomes
WBI should show increases in key domains, including standard of living, achievements in life and community connection.
Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security.
• Informs program practice.• Provides data for
reporting and funding purposes.
• Increases staff engagement and morale.
Volunteers and other external stakeholders will be provided with opportunities to complete surveys on their experience. This will further enhance our services and strengthen partnerships across the community.
Due to my experiences volunteering with WL OFA:
Strongly agree
Agree Neutral Disagree Strongly Disagree
Not Applicable
I am more aware of issues facing at risk young people
1 2 3 4 5 6
I believe I am making a difference
1 2 3 4 5 6
I would recommend volunteering for WL/OFA
1 2 3 4 5 6
Volunteer and stakeholder feedback
ChallengesChallenges
Internal changesDatabase developmentEngagement of staffCorrect data entryStaff capacitySecuring funding
Anticipated BenefitsAnticipated Benefits
An organisational commitment to continuous improvementIncreased professionalism The framework can be applied to all areas of the organisationOutcomes and data can be provided to potential and current funders, partners and stakeholdersBetter informed planning and program developmentAn open dialogue with clients about service improvement
Better outcomes for young people!
Questions?
Frances Head
George Elliott