urban institute identity implementation · stiefel m, nolan k. a guide to measuring the triple aim:...
TRANSCRIPT
M&E REINVENTED Validate Results, Inform Practice,
Improve Outcomes in Real Time
Global Youth Economic Opportunities Summit October 7, 2014
Roles in society
Contributions to individual and
community well-being
Impact on civic engagement
Economic impacts
Relationships with government
and business
IMPACT – How do you define it?
IMPACT BROADLY
How many people or organizations were served through your programs?
In what ways were they helped?
oGain knowledge
oLearn new skills
oEmpowered
oChanged behaviors
IMPACT NARROWLY In what ways did your
Program or Initiative
Make your … Community or Issue Area Better?
Did your contribution CAUSE the results you see?
Evaluation and
Learning…A Continuum
What can we learn? Can we do better now?
Did we succeed?
What difference did we make?
Did we succeed?
Impact evaluations • Random control trial
• Quasi-experimental designs using statistical techniques
Important, but expensive and long-term • Results take time – usually many years
Can we do better now?
Performance Management
Continuous Quality Improvement
Rapid Cycle Evaluation
YES
Your organization is striving to be high
performing. That means:
Your leadership guides the efforts
You strive to have sufficient analytic
capacity (staff capabilities and data
system capabilities)
You have created a culture supportive
of innovation where staff can safely
share successes and failures
You have a continuous cycle of learning
Combining good management
with evaluation techniques
= Rapid Cycle Evaluation
Rapid Cycle Evaluation
Identify element of operational process
Decide on how to vary the process
Randomly select part of your population
Implement
Evaluate
Adopt or Reject
Learn More About What Works here:
www.PerformWell.org
a partnership between Urban Institute Child Trends and Social Solutions
Impact is Important but
Continuous Learning is Essential
Continuous learning will help you examine impact:
You routinely collect and examine your data
Helps you identify “why” questions
Provides context for how well you program works in many situations
Social Solutions provides web-based
tools that measure the strength
and impact of transformational
relationships.
Confidential - Copyright, 2014 Social Solutions Global All
Rights Reserved
Modeling the Social Determinants of Health Outcomes
Stiefel M, Nolan K. A Guide to Measuring the Triple Aim: Population Health, Experience of Care, and Per Capita Cost.
IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2012. (Available on www.IHI.org)
After Evans and
Stoddart
Well
Being
Case
Management
Care
Management
SHIFT TO HIGH PERFORMING WHERE DO I START?
Evolution of Directional Intelligence
Tools should project our path from need to strength AND reveal trajectory while guiding practitioners along the way and helping us take corrective action when we
lose our way.
IF WE BELIEVE OUR WORK MAKES A
MEASURABLE DIFFERENCE…
HOW CAN WE MOVE FROM INTUITION TO CORRELATION?
Confidential - Copyright, 2014 Social Solutions Global All Rights Reserved
Target
Population
E
E
E
E
E E
Trajectory
Baseline
Effort = Cost = Investment Frequency
Duration
Type
Impac
t =
Benefit
= R
etu
rn
Period of Service
SCIENCE OF SERVICE DELIVERY
REVEALS TRAJECTORY & LIKELIHOOD
Confidential - Copyright, 2014 Social Solutions Global All Rights Reserved
Target
Population
Posi
tive
Im
pac
t
Period of Service
E E E
E
E
E
E
Roca has taken the stages of change and adapted
them for a very high-risk youth development
model. Roca uses the stages of change as:
Stages of Change & Roca
1) an assessment where young people are in relation to their readiness
and/or willingness to change;
2) a guide to inform youth workers’ use of self in their work with young
people; and
3) a foundation for designing engagement and competency-building
programming that meets young people where they are.
In 2013, Roca was awarded a six-year $23 million
Pay for Success Contract by the Commonwealth of
Massachusetts.
Contemplation Precontemplation Action Preparation Maintenance
Stages of Change (TTM) – What Does the Science tell us?
Key concepts in Stages of Change approach:
• Behavior change is a process that unfolds though a series of stages
• Individually tailored messages have positive effects on those who are ready,
getting ready, or not ready to change behavior
• Can have an impact on all participants – and predict future change
Behavior has
changed
What behavior change interactions are most effective for those who need to change
behavior?
Integration of Stages and Processes of Change
Precontemplation Contemplation Preparation Action Maintenance
Self-Reevaluation
Self-Liberation
Helping Relationships
Counter-Conditioning
Reinforcement Management
Stimulus Control
Prochaska, J.O., DiClemente, C.C., & Norcross, J.C. (1992). In search of how people change:
Applications to addictive behaviors. American Psychologist, 47(9), 1102-1114.
Social Liberation
Dramatic Relief
Consciousness Raising
Environmental Reevaluation
Stages and Processes of Change Correlate with ETO
WHAT KEEPS US FROM IMPROVING? Maybe we are TOO BUSY…
Please send questions to:
Teresa Derrick-Mills
Senior Research Associate
The Center on Nonprofits
and Philanthropy
Urban Institute
Adrian Bordone
Founder
Social Solutions
Bojan Cubela
Director, Workforce
Development
Social Solutions