results based accountability: a real life...
TRANSCRIPT
Results Based Accountability:A real life story
SkillsFest 3.0September 19, 2018
Collaborative work on a wicked problem…
Why is this not working?
Identified Need
Design a project or program
Outcomes with Logic Model
It’s working!! The need is being met!!
Grow it!!!
Systemic Problem
The only thing between the project or program level
performance and systemic/population change
is HOPE
HOPE
Working on the system, from within the system will NOT change population
health
1. It is SIMPLE, common sense, uses plain language, useful
2. Data-driven, transparent decision-making
3. Is designed to work with programs and populations
4. Start with ends, work backwards to means
Why use RBA as the model?
3 - kinds of performance measuresHow much did we do? How well did we do it? Is anyone better off?
RBA in a Nutshell2 – 3 - 7
2 - kinds of accountabilityPopulation accountability: Results & IndicatorsPerformance accountability: Performance measures
7 - questions from ends to means in less than an hour
Results-based AccountabilityPopulation Accountability
Is about the well being of WHOLE POPULATIONS
(communities, cities, states, nations)
Performance AccountabilityIs about the well being ofCUSTOMER POPULATIONS
(programs, agencies, service systems)
Children in the Fox Valley are
healthy, safe and ready to learn
92% of 4 year olds in Appleton
Area School District passed
the kindergarten entry
assessment
Performance Measures: How much did we do?How well did we do it?Is anyone better off?
RESULT: A condition of well being for children, adults, families, communitiesINDICATOR: A measure which helps quantify the achievement of a result
Turn the Curve Thinking: Population
RESULT
INDICATOR
BASELINE/TREND
STORY BEHIND THE CURVE
PARTNERS
WHAT WORKS?
STRATEGY & ACTION
Choosing Indicators with:Communication Power (Broad audience)
Proxy Power (Central Importance)
Data Power (Quality and Timely)
Now
Ok? or Not OK?
1975
1980
1982
1990
2000
2005
0
5
10
15
20
25
30
Tho
usan
ds
Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.
Alcohol-Related Traffic FatalitiesUnited States 1975 to 2005
45 people per day
75 people per day
28 peopleper day
MADD –Population
Accountability
“All performance measures that have ever existed for any program in the history of the universe
involve answering two sets of interlocking questions.”
How much did we do? How well did we do it?
ANDHow much effort did it require?
Is anyone better off?
Performance Accountability
How much did we do?
Performance AccountabilityTypes of Measures found in each Quadrant
How well did we do it?
Is anyone better off?
# Clients/customersserved
# Activities (by typeof activity)
% Common measurese.g. client staff ratio, workload ratio, staff, turnover rate, staff morale, % staff fully trained, unit cost
% Skills / Knowledge(e.g. parenting skills)
#
% Attitude / Opinion(e.g. toward drugs)
#
% Behavior(e.g.school attendance)
#
% Circumstance (e.g. working, in stable housing)
#
% Activity-specificmeasures
e.g. % timely, % clients completing activity, % correct and complete, % meeting standard
Point in Time vs.2 Point
Comparison
Drug/Alcohol Treatment Program
Is anyone better off?
Number of clientsoff of alcohol & drugs - at exit
- 12 months after exit
Percent of clientsoff of alcohol & drugs
- at exit- 12 months after exit
Quantity Quality
Effe
ct
E
ffort Number of
studentsStudent-teacher
ratio
Turn the Curve Thinking: Performance
Lower Right
Upper Right
BASELINE/TREND
PARTNERS
WHAT WORKS?
IMPLEMENTATION
DIRECT/INDIRECTINTERNAL/EXTERNAL
Contributionrelationship
Alignmentof measures
Appropriateresponsibility
THE LINKAGE between POPULATION and PERFORMANCEPOPULATION ACCOUNTABILITY
Self-sufficient FamiliesPercent of parents earning a living wage
PERFORMANCE ACCOUNTABILITY
CUSTOMERRESULTS
# personsreceivingtraining
Unit costper person
trained
# who getliving wage
jobs
% who getliving wage
jobs
POPULATIONRESULTS
Job Training Program
ONE PAGE Turn the Curve ReportResult: _______________
Indicator(Lay Definition)Indicator
Baseline
Story behind the baseline------------------------------------------------------ (List as many as needed)
Partners------------------------------------------------------ (List as many as needed)
Three Best Ideas – What Works1. ---------------------------2. ---------------------------3. ---------No-cost / low-cost/off the wall
POPULATIONCalumet: 49,491
Outagamie: 182,006 Winnebago: 169,511
Total: 401,008
STUDENT ENROLLMENT7 – 12th grade: 31,000
8 years$1.2 million investment
Result/Indicator• Our result is: improved behavioral health in your community What population
level indicator(s) are a proxy that would tell you that behavioral health has improved in your community? Where is the indicator now, and where do you want the indicator to move to. The indicator should be (a) easy to communicate; (b) a relevant, logical proxy for the result; and (c) feasible to measure.
Indicator: Reduced rates of youth depression and suicide-related behaviors in 12-18 year olds in the tri-county area –
Calumet, Outagamie and Winnebago Counties.
Why is this indicator a relevant, logical proxy?
Wisconsin has a higher rate of youth depression than the national average. Both depression and suicide-related behaviors share many of the same risk factors. Strategies will likely hit both.
Why youth? We are targeting youth because 70% of mental disorders onset prior to the age of 24
Depression is theNo.1
risk factor for suicide
20%of teens will
experience depression
Youth with severe depression:
Mental health disorders start early:
Teens Who Have Attempted Suicide
Wisconsin: 10.5%United States: 7%
Wisconsin ranks #44 out of 50,making it the 6th worst state in the nation for youth depression
22.6% 22.6%24.2%
26.1%28.40%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
2007 2010 2012 2014 2016
What is the measure today and what has been its recent trends? If nothing changes, what might curve look like?
Where would you like to see the curve go by 2024?
0%
5%
10%
15%
20%
25%
30%
35%
2010 2012 2014 2016 2018 2020 2022 2024
Tri-County Rate of “Sad and Hopeless”
Factor Analysis
“Protective Factors” – push indicator down
• Adequate amount of sleep
• Regular physical activity
• Social-emotional and self-regulation skills
• Loving and engaged caregiver
• Knowing where to go for help
• Atmosphere of school environment:
1. Trusted adult at school
2. Supportive teacher-student relationships
3. Sense of belonging at school
How do I know these are factors that push the curve up or down?What data/research will I need to validate these factors and identify others?
“Risk Factors” – push indicator up
• Sexual orientation (LGBTQ)
• Bullying (at school, electronically)
• Not feeling safe or belonging at
school
• Substance use
• Poor self-regulation
• Genetic predisposition
• Generational transmission
SOCIAL CONNECTION
EMOTIONALREGULATION
SKILLS
IDENTIFY UNTREATEDDEPRESSION
Influence of Factor on Indicator (depression)
Our
abi
lity
to in
flue
nce
this
Fa
ctor
LOW
LOW
HIG
H
HIGH
LOVING CAREGIV
ERSUBSTANCE USE
PHYSICAL
ACTIVITY
SLEEPSCHOOL CLIMATE
DrivingFactors:
Strategies:
Indicator: Reduce rates of teen depression
SOCIAL CONNECTION BULLYING EMOTIONAL
REGULATION SLEEPUNTREATEDDEPRESSION
School climate & social
connectedness
Universal mental health prevention &
promotion
Early identification,
screening, and referral
Navigation of mental health system and
connection to services
