the outcomes toolbox presentation by bob penna
DESCRIPTION
ÂTRANSCRIPT
The Outcomes Toolbox: What You Need to Know to
Use Outcomes Robert M. Penna, Ph.D.
Author: The Nonprofit Outcomes Toolbox MNA Conference 2012 September 19, 2012 Helena, Montana
2
Today’s topics: An introduction to Outcomes:
What is an “outcome”? How is an “outcomes approach” different? What is the language of outcomes? What are the characteristics of a good outcome? Crafting an outcomes statement
An introduction to “outcomes tools”;; the view from 30,000 feet Tools for planning Tools for tracking and management Tools for reporting Tools for learning and assessment
Using some tools Building your “Outcomes Tree” Understanding and Assessing Capacity Reporting
3
Our targeted outcome:
That within one week you will do something different at work because of something you learned today
4
A quick bit of history……
5
So, think of it this way….
Outcomes: the 3rd Stage of Management
• Stage 1—Agriculture and Early Industrial Economy: Management of Workers
• Stage 2—1920–1970: Management ofWork
• Stage 3—The Age of Outcomes: Management of the Product of Work. . . The Results
6
An introduction to “Outcomes”
What is an “outcome”?
7
What makes an “outcomes approach” different?
8
The Problem Approach
9
The Activity Approach
10
The Process Approach…
11
The Vision Approach…
12
The Outcomes Approach
13
Compare this……
14
To this…. What’s different??
15
Even better…..
16
Three Changes in Thinking
1. From Funder to Investor 2. From Activity to Effectiveness 3. From Services to Change
The BACKS Measures
17
The “Language” of Outcomes
نتائج
Processus השפעת
Ingresso
Επιπτώσεις
Väljund
18
The Characteristics of a Good Outcome Are some outcomes “better” than others? What does a “good” outcome look like? Positive Improvement Meaningful Sustainable Bound in Time Bound in Number Narrowly Focused and “Doable” (with a stretch!) Measurable
Clearly Definable Discernable…can be “evinced” Quantifiable…according to an agreed-upon scale!
Verifiable
19
The “Outcomes Statement”
What is it? Why Use it?
20
Practice
Your turn……..
21
Take a break!
22
An Introduction to “Outcomes Tools”: a view from 30,000 feet….. What are “outcome tools”? 4 sets:
Tools for Planning Tools for Tracking & Managing Tools for Reporting Tools for Learning & Assessment
23
Tools for Planning
You Get What You Measure A Common Outcomes Framework
24
Tools for Tracking and Managing
The Logic Model RBA TOP GTO TRI Outcomes Management Results Mapping The Balanced Scorecard
25
Snapshot: The Logic Model
32
26
A Detailed Logic Model
33
27
But it gets tricky when….
28
Snapshot: Results-Based Accountability
42
29
Snapshot: TOP
Targeting Outcomes of Programs (TOP) Model
KASA:K = KnowledgeA = AttitudeS = SkillA = Aspiration
Program Development(Goals/Objectives)
SEEC:S = SocialE = EconomicE = EnvironmentalC = Conditions
SEECPractices
KASAReactions
ParticipationActivities
Resources
Program Performance(Evaluation Measures)
ResourcesActivities
ParticipationReactions
KASAPractices
SEEC
Source: Claude Bennett & Kay Rockwell, 1994
30
Snapshot: GTO The Getting To OutcomesTM
Process
Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1
Identify goals,target population,
and desired outcomes.
Identify goals,target population,
and desired outcomes.
2
Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Modify the program
or best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Assess capacity(staff, financing,
etc.) to implementthe program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5
Make a planfor getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6
Steps 1-6PLANNING
DELIVERING PROGRAMS
Evaluate planning and
implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7
Evaluate program’s success
in achievingdesired results.
Evaluate program’s success
in achievingdesired results.
8Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9
Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Steps 7-10
EVALUATING AND IMPROVING
Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1
Identify goals,target population,
and desired outcomes.
Identify goals,target population,
and desired outcomes.
2Identify goals,
target population,and desired outcomes.
Identify goals,target population,
and desired outcomes.
2
Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Modify the program
or best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Modify the programor best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Assess capacity(staff, financing,
etc.) to implementthe program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5Assess capacity
(staff, financing,etc.) to implement
the program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5
Make a planfor getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6Make a plan
for getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6
Steps 1-6PLANNING
DELIVERING PROGRAMS
Evaluate planning and
implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7Evaluate
planning and implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7
Evaluate program’s success
in achievingdesired results.
Evaluate program’s success
in achievingdesired results.
8Evaluate
program’s successin achieving
desired results.
Evaluate program’s success
in achievingdesired results.
8Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9
Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Steps 7-10
EVALUATING AND IMPROVING
31
Snapshot: TRI/Outcomes Management
32
Snapshot: Results Mapping
MLS7
MLS6
ACT5 > LEV5
MLS4
ACT3 > LEV3
ACT2 > LEV2
ACT1 > LEV1 38
33
Snapshot: A Balanced Scorecard
34
Tools for Reporting
Scales & Ladders The STORY format
35
Snapshot: Scales & Ladders
36
Telling your story
First off….begin by thinking of it as a story….
37
A Quick Test: These
38
As opposed to these……
050
100
1stQtr
3rdQtr
EastWestNorth
HQ
Boston Seattle Dover
A 82 45 54 62
B 37 97 17 12
C 10 83 7 7
39
Tools for Learning and Assessment
1. Lessons Learned 2. Six Sigma 3. SERVQUAL 4. Value Engineering 5. FMEA 6. TRIZ 7. CAT 8. GTO
40
LUNCH!!!
41
Building Your Outcomes Tree
1. Outcome 2. Performance Targets 3. Milestones
42
You try it….
43
Take a break!
44
Understanding and Assessing Capacity
What is “capacity”? Three levels
Structural Functional Implementation
Two Tools CAT GTO
45
CAT Essentially a self-scoring grid/matrix format…can be created in Excel
ASPIRATIONS
Mission1 Clear need for increased capacity
2. Basic level of capacity in place
3. Moderate level of capacity in place
4. high level of capacity in place
Some expression of organization’s reason for existence that reflects its values and purpose, but may lack clarity; held by only a few; lacks broad agreement or rarely referred to.
Clear expression of organization’s reason for existence which reflects its values and purpose; held by many within organization and often referred to.
Clear expression of organization’s reason for existence which describes an enduring reality that reflects its values and purpose; broadly held within organization and frequently referred to.
VPP/SVP Capacity Tool
No written mission, or only a limited expression of the organization’s reason for existence; lacks clarity or specificity; either held by very few in organization or rarely referred to
46
Snapshot: GTO The Getting To OutcomesTM
Process
Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1
Identify goals,target population,
and desired outcomes.
Identify goals,target population,
and desired outcomes.
2
Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Modify the program
or best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Assess capacity(staff, financing,
etc.) to implementthe program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5
Make a planfor getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6
Steps 1-6PLANNING
DELIVERING PROGRAMS
Evaluate planning and
implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7
Evaluate program’s success
in achievingdesired results.
Evaluate program’s success
in achievingdesired results.
8Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9
Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Steps 7-10
EVALUATING AND IMPROVING
Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1Choose which problem(s) to
focus on.
Choose which problem(s) to
focus on.
1
Identify goals,target population,
and desired outcomes.
Identify goals,target population,
and desired outcomes.
2Identify goals,
target population,and desired outcomes.
Identify goals,target population,
and desired outcomes.
2
Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Find existingprograms andbest practices
worth copying.
Find existingprograms andbest practices
worth copying.
3Modify the program
or best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Modify the programor best practicesto fit your needs.
Modify the programor best practicesto fit your needs.
4
Assess capacity(staff, financing,
etc.) to implementthe program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5Assess capacity
(staff, financing,etc.) to implement
the program.
Assess capacity(staff, financing,
etc.) to implementthe program.
5
Make a planfor getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6Make a plan
for getting started: who, what, when, where, and how.
Make a planfor getting started: who, what, when, where, and how.
6
Steps 1-6PLANNING
DELIVERING PROGRAMS
Evaluate planning and
implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7Evaluate
planning and implementation.How did it go?
Evaluate planning and
implementation.How did it go?
7
Evaluate program’s success
in achievingdesired results.
Evaluate program’s success
in achievingdesired results.
8Evaluate
program’s successin achieving
desired results.
Evaluate program’s success
in achievingdesired results.
8Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9Make a plan for
ContinuousQuality
Improvement.
Make a plan forContinuous
QualityImprovement.
9
Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Consider how to keep the program
going if it issuccessful.
Consider how to keep the program
going if it issuccessful.
10Steps 7-10
EVALUATING AND IMPROVING
47
GTO capacity question
CAPACITY TOOL: Program specific staff capacities (full version) Capacity assessment item Requirements Is current capacity
sufficient? (Y/N) Plan to enhance the capacity
Staff Training needed
Staffing level required
Staff qualifications (e.g. minimum degree, certification or training needed; years of experience)
48
Capacity grid for Recruiting potential trainees Milestone Challenge Unready Marginally Ready Fairly Ready Fully Ready
Identifying and contacting 125 unemployed adults in the community who will be potential candidates for job training and placement program
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
1 Communicating with potential candidates; fielding and answering their questions
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
2 Holding orientation session for 100 potential candidates
Not sure where or how we'll do it
Some idea of where and how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of where and how we'll do it; details all worked out
3 Processing enrollment of 75 potential candidates
Not sure where or how we'll do it
Some idea of where and how we'll do it, but few details worked out
Pretty good idea of where & how we'll do it; only a couple of details still need to be resolved
Firm idea of where and how we'll do it; details all worked out
7 Identifying prospective local employers who might be willing to hire program graduates
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
49
Reporting: Using Scales & Ladders
Next Steps: Action Planning Come up with a specific action step to be started next week
63
51
Wrap-up, feedback, and Questions
52
Thank you for your time and
participation…..
Contact: Dr. Robert Penna PO Box 38081 Stuyvesant Station Albany, NY 12203-8081 518-438-7502 [email protected] Website: www.outcomestoolbox.com