performance 2.0: from measurement to management
DESCRIPTION
Performance 2.0: From Measurement to Management. Jon Desenberg Policy Director The Performance Institute 800 15 th Street NW Third Floor Washington DC 20005 Phone: 202-739-9642 www.performanceweb.org [email protected]. Why measure?. To Plan? To Comply? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/1.jpg)
Performance 2.0: From Measurement to Management
![Page 2: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/2.jpg)
Jon DesenbergPolicy Director
The Performance Institute 800 15th Street NW
Third FloorWashington DC 20005Phone: 202-739-9642
www.performanceweb.org [email protected]
![Page 3: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/3.jpg)
![Page 4: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/4.jpg)
![Page 5: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/5.jpg)
Why measure?
To Plan?
To Comply?
To Manage?
To Optimize?
To Innovate?
What gets measured gets done.
–Peter Drucker
![Page 6: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/6.jpg)
Long-Term Move Towards Performance
Performance management is not a new phenomenon—50 years of work in the making to link resources with results
– Budget Accounting Procedures Act (BAPA) of 1950
– Planning-Programming-Budgeting System (PPBS), 1965-1971
– Management by Objectives (MBO) 1973-1974– Zero-Based Budgeting (ZBB), 1977-1981
![Page 7: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/7.jpg)
Finally a Move in the Right Direction
• GPRA—(1993) Government Performance Results Act
• PMA—(2001) President’s Management Agenda
• PART—(2002) Program Assessment Rating Tool
• Mandates focusing on performance and accountability—bottom line results
![Page 8: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/8.jpg)
More Timely Reporting for Better Real-Time Decisions
OMB has eliminated monthly reporting requirements for the foreseeable future...given the continued demand for timely reporting.
The reporting frequency and detail may change slightly over time, reflecting the information needs of the administration and the public, Agencies should begin exploring the accounting and process changes required to update obligations and gross outlays more frequently than a weekly report.
![Page 9: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/9.jpg)
Transparency = Performance Management:
• Recovery Act Funds are awarded and distributed in a prompt, fair, and reasonable manner;
• The recipients and uses of all funds are transparent to the public, and the public benefits of these funds are reported clearly, accurately, and in a timely manner;
• Funds are used for authorized purposes and potential for fraud, waste, error, and abuse are mitigated;
• Projects funded under this Act avoid unnecessary delays and cost overruns; and
• Program goals are achieved, including specific program outcomes and improved results on broader economic indicators.
![Page 10: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/10.jpg)
Federal Performance : New Developments
• A re-focused Process in development
• The Federal Chief Performance Officer
• Weekly Recovery Act Reporting
• Data.gov
![Page 11: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/11.jpg)
Understanding Current Performance Implementation
• Four GAO Studies – 1994-2007
– 1994: 19% Outcome Measures
– 2007: 50% Outcome Measures
• PMA and PART: Checklist or Results?
– “Costly and Burdensome”
Effective
Ineffective
Results NotDemonstrated
![Page 12: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/12.jpg)
![Page 13: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/13.jpg)
Performance Management:Have We Forgotten “Why?”
• Do the same problems still need our attention? And what are their causes?
• Is the strategy effective? – Can it be replicated?– Can we speed adoption?
• Can we improve real communication?– To the White House, Congress and
(even) appropriators– State, Local, Non-profit and other
partners
![Page 14: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/14.jpg)
“We have run out of money, now we must think” – Winston Churchill - 1943
![Page 15: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/15.jpg)
Peak Deficits (% GDP) Predicted Soon
-4-202468
10121416
USA France Germany
2008Peak
UK Japan RussiaThe Economist June 2009
![Page 16: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/16.jpg)
Performance / Productivity Gains Increasingly Seen as Critical
$0
$20
$40
$60
$80
$100
$120
$140
USA Japan UK France
5%15%
OECD.org –May 2009
![Page 17: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/17.jpg)
Understanding Logic Models
![Page 18: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/18.jpg)
What is a logic model?1. Logical chain of events providing blueprint for mission
achievement2. Graphic representation that illustrates the rationale
behind a program or organization3. Depicts causal relationships between activities,
strategies, and end results4. Contains goals and performance measures 5. Integrates various program activities into a cohesive
whole6. Vehicle for dialogue, planning, program management
and evaluation
![Page 19: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/19.jpg)
What does a logic model look like? • Graphic display of boxes and arrows;
vertical or horizontal
– Relationships, linkages
• Any shape
– Circular, dynamic
– Cultural adaptations, storyboards
• Level of detail
– Simple
– Complex
• Multiple models
![Page 20: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/20.jpg)
Logic modeling is based on mapping and defining linkages between what we do and why we do it .
I Work Out for One Hour
Each Day
I Will Burn More
Calories Than I
Consume
Lose Fat and Build
Muscle
Improve My Looks and Health
Have Better Image,
Feel Better & Live Longer
IF THEN IF THEN IF THEN IF THEN
INPUTS OUTPUTS OUTCOMES
Series of If-Then Relationships
Assumptions: improving looks = better self image
Factors: Health History
![Page 21: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/21.jpg)
InputsPeople and resources required to achieve outcomes
Activities/OutputsWhat the inputs produce
Immediate and Intermediate OutcomesChanges required to achieve end outcome
End Outcome
End goal or ultimate benefit
Clarifying the terms
Assumptions: beliefs or evidence that supports your IF-THEN logic
Factors: external influences beyond control that effect IF-THEN relationships
![Page 22: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/22.jpg)
Logic Model “V”
Bottom-Line Investment
Top-Line Return
Alignment
Linkage
Mea
sure
men
t
![Page 23: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/23.jpg)
Logic Model “V” Performance Dimensions
![Page 24: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/24.jpg)
Distribute program grants
Distribute program grants
Child violence and abuse can be prevented and
detected
Child violence and abuse can be prevented and
detected
Child health and development can be
protected and maintained
Child health and development can be
protected and maintained
Children can grow into productive
citizens and attain their intended
impacts on society
Children can grow into productive
citizens and attain their intended
impacts on society
“So That”
“So That”
“So That”
Degree of Influence by Department
Ultimate Program IntentOutput
Intermediate Outcome
Intermediate Outcome
High-Level Outcome
Value Chain Diagram
![Page 25: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/25.jpg)
Early Outcomes Later OutcomesLater ActivitiesEarly Activities
If we do…
Outreach
Screening
ID of elevated kids
And we do…
Case mgmt of EBLL kids
Refer EBLL kids for medical treatment
Train family in in-home detection techniques
Assess environment of EBLL child
Refer environment for clean-up
Then….
EBLL kids get medical treatment
Family performs in-home techniques
Lead source identified
Environment gets cleaned up
Lead source removed
And then…
EBLL reduced
Developmental slide stopped
Quality of life improves
Global Logic Model: Childhood Lead Poisoning Program
Definition:
EBLL – Elevated Blood Lead
LevelsModule Three: Using Logic Models
![Page 26: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/26.jpg)
Inputs Activities OutputsEnd
Outcomes
Most logic models incorporate the following elements.
Intermediate Outcomes
Attitudes Behaviors Conditions
WHY?
HOW
EFFECT
CONTROL
![Page 27: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/27.jpg)
Performance 2.0 Abroad
• Desired productivity gains translated into specific input and output targets for every agency
Finland
New Zealand
• National outcome dashboard for Hospital Waiting Time, Crime, Test Scores, other.
U.K.
• Deliberate Move Away from Short Term Output Measures towards fewer, more “nuanced” outcomes
![Page 28: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/28.jpg)
The British Model
• UK Center for the Measurement of Government Activity– Output=Input Formula eliminated in 1993– Quality adjustment advancements on outputs
• Through academic and peer review
• Health & Education
– Management SWOT Teams– New Calls for True Independent Center
![Page 29: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/29.jpg)
Important Next Steps
• Defining “Value” – What Do People really want?
• What are the relative preferences for outcomes?
• Match with metrics used in regulatory process
![Page 30: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/30.jpg)
Resources and Opportunities
Capability
Service Offerings
Functioning
Utility
Draft Framework: Outcomes for Adult Care
![Page 31: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/31.jpg)
Connecting Congress with Program Results:
1. CompetitionTransparency and Pressure to Perform
Cross-Government Results
Purchase Agreements for Appropriators
2. InnovationOMB Examiners: “Beyond Triage Mode”
3. Dispersion
4. Enhanced Productivity
![Page 32: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/32.jpg)
The Guidestar Model
• Independent Access to Metrics and Budget Information
• GAO / CBO both reluctant to play this role
• OMB not deemed objective
• Meet Growing Demand
![Page 33: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/33.jpg)
Taking Performance to the Next Level
• Incremental Improvements: No Longer Acceptable?
• Senate Bill 3521 – Is the One Year Budget Cycle Preventing Performance?
• Should Front Line Managers keep their Savings?
![Page 34: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/34.jpg)
• Breaking Out of the “Performance Office” to:– Understand the issue’s
“drivers”
– Discover why performance levels vary
– Find different interventions for different circumstances
– Enhance Knowledge Sharing across programs
Discussion & Diagnostics
![Page 35: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/35.jpg)
Connecting Both Ends of Pennsylvania Avenue
• To Monitor and Control the White House?
• Ensuring Each District’s Interests are Secured?
• To Regularly Evaluate & Re-authorize?
• To Sunset Programs?• Naming and Shaming?• To Begin Using More Data?
![Page 36: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/36.jpg)
Key Members of CongressSenator Claire McCaskill
Former State Auditor
New Subcommittee Chair
Senator Tom Coburn “Waste of the Day” “Dr. No” / USA Spending.gov
Cong. Dennis Moore
Blue Dog Policy Chair
PAYGO Emphasis
Budget Committee
Senator John Cornyn
Transparency and Oversight emphasis
![Page 37: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/37.jpg)
![Page 38: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/38.jpg)
Why is Congress Unhappy with Current Reporting?
“This site is not providing data that the public can understand”
![Page 39: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/39.jpg)
H.R.2142 - Government Efficiency, Effectiveness, and Performance Improvement
Act• Each agency head should consult with the congressional
committees with jurisdiction over the agency each fiscal year regarding the agency performance plan.
• Each agency head, in collaboration with OMB, to conduct an assessment of each agency program at least once every five fiscal years:– assessing the program’s purpose, – strategic plan and objectives, – organizational design, – management, – efficiency, and – effectiveness in achieving its performance objectives and identifying
strengths and weaknesses and factors affecting program success.
![Page 40: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/40.jpg)
(1) make available, by May 1 of each year, a list of programs to be assessed and the criteria and methodology that will be used;
(2) submit in a report to Congress, at the same time the President submits the annual federal budget, assessment results, including an identification of program best practices for allocating resources and recommendations for modifications focusing on opportunities for consolidation and integration of programs and authorities; and
(3) develop an improvement plan, which the agency head shall implement, that responds to the assessment report and identifies follow-up actions to improve program performance.
Rep. Henry Cuellar
HR 2412 Would Require OMB To:
![Page 41: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/41.jpg)
Views from The Hill
• “The Information is buried in thousands of p ages that we end up throwing out.”
• “I pick up the phone and call GAO, PART is political.”
• “They act as if we appropriate by goals. We want information by program area.”
• The Bush Administration redesigned the budget process so that we couldn’t find
core information.”
![Page 42: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/42.jpg)
Does Anyone Want This Job?
![Page 43: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/43.jpg)
How Do We Evaluate Performance
Management?• Past decisions are viewed as
rational if the outcome was good.
• If performance management is always ex post and subjective, honest evaluation of intelligence becomes difficult.
![Page 44: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/44.jpg)
The Knowledge – Data Gap
1. Lessons are learned inadequately,
2. Memories are recalled incompletely,
3. The futures is predicted inaccurately,
4. Data rarely overcomes
preconceived notions.
![Page 45: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/45.jpg)
When The Data isn’t enough
Possible Solutions:
• Rich histories
• Multiple observers and multiple interpretations
• Hypothetical histories
![Page 46: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/46.jpg)
A Data Driven Culture Embraces Experimentation
Think of:
• Trial programs, pilot studies, and small experiments, and about the inferences that can be drawn from them.
• Does the Government embrace experimentation?
![Page 47: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/47.jpg)
How to Get Fired at Harrah’s
“Institute a program without first running a data driven experiment.”-Gary Loveman
![Page 48: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/48.jpg)
A Learning Organization
• An average of 300 data “experiments” per day
• Increased Retention by 87%
• Lowered the cost of a new account by 83%
• Stock price outpaced
S & P by a factor of ten
TestingLearning
Acting
![Page 49: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/49.jpg)
Analytical Expertise:How do we get there?
0%10%20%30%40%50%60%70%80%90%
100%
1997 1999 2005 2008
371 Large and Medium Sized CompaniesJeanne G. Harris Institute
![Page 50: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/50.jpg)
![Page 51: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/51.jpg)
![Page 52: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/52.jpg)
![Page 53: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/53.jpg)
Measurement Maturity Index
• In order to have a good idea of your organization’s measurement progress the index will give a score up to a maximum of 300 in 4 key areas.
• Continue to assess your organizations over time to understand their progress and help them understand where to focus.
![Page 54: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/54.jpg)
Four Maturity Themes
1. Interaction Discussion, Feedback, Pilots, Learning, Actionable
2. Integration Cross-Functional Understanding, Casual Relationships Explored
![Page 55: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/55.jpg)
Four Maturity Themes
3. Focus Results-focus, Critical and Strategic The routine measures are eliminated for the high-
leverage
4. Context Widely Used, Valued, Trusted No Fear, Positive and Motivating
![Page 56: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/56.jpg)
Level One
Basic Performance Management
Systemic Negative/
Neutral
Context
Programmatic
Specialized
Ad Hoc
Level Two
Transformational Performance Management
Transforming Positive Context
Collaborative
Level Three
300 Score
Measurement Maturity Index
![Page 57: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/57.jpg)
Re-emphasizing theLogic Model “V”
Bottom-Line Investment
Top-Line Return
Alignment
Linkage
Mea
sure
men
t
![Page 58: Performance 2.0: From Measurement to Management](https://reader035.vdocuments.us/reader035/viewer/2022062518/568144d9550346895db1a660/html5/thumbnails/58.jpg)
Exit Opportunity
Proven Success
ResourcesAvailable
IMPORTANCE
PERFORMANCE
4
3
2
1
1 2 3 4
a
Factors
a = I4, P2
b = I3, P3
c = I2, P1
d = I1, P4
b
c
d
Attention Needed
Selecting Your MeasuresThe Program Performance Assessment Window™