ellig gpra and use of performance info april 2009
DESCRIPTION
TRANSCRIPT
Has the Government Performance and Results Act Improved the Availability and
Use of Performance Information?
Jerry ElligSenior Research Fellow
Government Performance and Results Act of 1993 One motivation: “Federal managers are seriously
disadvantaged in their efforts to improve program efficiency and effectiveness, because of insufficient articulation of program goals and inadequate information on program performance.”
Requirements:
Strategic plans outlining goals (including outcomes) and planned program evaluations
Annual performance plans establishing quantitative goals Annual performance reports beginning fiscal year 1999
Possible research questions
Has GPRA increased the availability and use of performance information in agencies?
Has GPRA increased the use of performance information by policymakers?
Has increased use of performance information affected government’s efficiency, effectiveness, scope, or size?
Have GPRA-induced changes in government affected human welfare?
Why might GPRA improve availability or use of information? Tullock, Downs theories of bureaucracy:
Individuals advance by doing what their superiors want.
How do agency officials know policymakers want performance management?
GPRA is clearly the law Presidents have shown consistent interest Congress has shown less consistent interest
Why might GPRA fail?
GPRA skeptics Analytical and data difficulties Outcomes harder to measure for some kinds of programs
(eg, grants, R&D) US political system hampers agreement on outcome goals
Agency theory Align agency incentives with performance budgeting Align managers’ incentives with performance contracts Give mangers greater discretion over use of resources Reform civil service
The empirical analysis
Dependent variable: Periodic Government Accountability Office (GAO) surveys ask federal managers in 24 agencies if they have various kinds of performance information and use it for various purposes
Independent variable: Annual Mercatus Center Scorecard measures quality of the same 24 agency GPRA performance reports
Are these 2 things correlated?
GAO surveys
Asks federal managers if they have various kinds of performance measures for their programs or projects
Asks federal managers if they use performance information for various specified purposes in their programs or projects
2000 and 2007 had enough responses to calculate valid percentages for each agency
Use % of managers responding “to a great extent” or “to a very great extent”
Average of agency responses
Figure 1: Average % of managers with various types of performance measures
0 10 20 30 40 50 60
Output
Efficiency
Customer satisfaction
Quality
Outcome
2000
2007
Average of agency responses
Figure 2: Average % of managers who use performance information for these purposes
0 10 20 30 40 50 60
Setting priorities
Allocating resources
Adopting new approaches
Coordinating with external orgs.
Refining program measures
Setting performance goals
Setting job expectations
Rewarding employees
Managing contracts
2000
2007
The Mercatus Scorecard 1-5 rating scale
3 Categories
TransparencyPublic BenefitsLeadership
4 criteria in each category
Criteria tightened each year to reflect previous year’s best practices
Total score can range from 12 to 60
5 Sets a standard for best practice
4 Shows innovation and creativity
3 Satisfactory
2 Partially complete
1 Fails to meet expectations
Figure 3: Mercatus Scorecard scores
0
10
20
30
40
50
60
2000
2007
2 econometric specifications
Pooled 2000 and 2007 data 46 observations Control variables
Year, score*year Leadership commitment (survey response) % of budget spent on competitive grants, block grants,
R&D, regulation
2007-2000 differences 22 observations Leadership control variable sometimes significant
GPRA linked to availability of information(results significant at 90% level or higher) A one point change in an agency’s Scorecard score is
associated with between one-third and one-half of a percentage point change in the number of managers stating that they have measures of Outcomes Outputs Efficiency
An agency producing a GPRA report with an average Scorecard score of 34 would have about 10 percent more managers reporting that they have outcome, output, or efficiency measures.
40-60 percent of managers in various agencies said they had outcome, output, or efficiency measures.
GPRA and use of performance information(results significant at 90% level or greater)
Score of agency GPRA reports is positively correlated with five different uses of performance information:
allocating resources setting priorities coordinating with external agencies establishing measures setting goals
How big is Scorecard score’s effect on uses of performance information? Coefficients range between 0.3 and 0.6.
An agency producing a GPRA report with an average Scorecard score of 34 would have 10-20 percent more managers reporting that they use performance information for these purposes.
25-50 percent of managers said they use performance information for various purposes.
Influence on use of information may be wider Scorecard score may affect other uses of
information
Existence of outcome measures is correlated with Scorecard scores
Every use of performance information is correlated with existence of outcome measures
Scorecard scores may have an indirect effect on every use of performance information
All of these regressions only measure effects that are attributable to differences in quality of agencies’ GPRA initiatives
Other interesting results
Leadership control variable often has a big positive influence
More competitive grants => more managers with outcome measures
More block grants => lower percentages of managers claiming they have efficiency measures
More regulation => lower percentages of managers that say they have outcome measures or use performance information to allocate resources
More R&D => lower percentages of managers reporting they have output measures, efficiency measures, or use performance information for many purposes
What can we conclude?
GPRA had an effect: Quality of agencies’ GPRA initiatives has an effect on availability and use of performance information
Tullock was right: Passage of GPRA, executive branch guidance, and commitment of agency leadership motivated improvement without wholesale revision to bureaucratic incentive structure
Skeptics were right about some things: Some kinds of performance measurement and use of info seems less likely in agencies utilizing more block grants, regulation, or R&D programs