Preparing Performance Reports:Lessons from the First Mercatus Scorecard

Jerry Ellig

Senior Research Fellow

Mercatus Center

George Mason University

Arlington, VA

www.mercatus.org

What is the Scorecard?

• Annual ranking, first published in May 2000

• Evaluation of 24 CFO Act agencies’ reports

• Assessed quality of reporting, not results

• Examined 12 criteria related to transparency, public benefits, and forward-looking leadership

Transparency

• Is the report accessible and easy to find?

• Can a layperson read and understand it?

• Are data reliable, credible, and verifiable?

• Are baseline and trend data included?

Public Benefits

• Are goals stated as results?

• Do measures focus on results, or activities?

• Is causation clear?

• Are goals and results linked to costs?

Forward-Looking Leadership

• Does agency explain how it improves our quality of life?

• Does report explain failures to meet targets?

• Does it identify major management challenges?

• Does it describe changes to improve performance next year?

Easy for a layperson to read?

• Plain English

• Avoid acronyms!

• Headings, graphics facilitate quick scan

Role model: USAID

Credible, reliable, verifiable data

• Validation procedures

• Limitations of data

• External review/verification

Role model: Education’s frank discussion

Baseline and trend data

• Are they included?

• Can the reader easily track progress?

DOD: 3 years of actual data, plus goals for next 3 years

Goals/measures stated as results

Result: Desired benefit produced (or harm avoided) due to agency’s programs

Examples:• Cleaner air vs. enforcement actions• Safer roads vs. seat belt use• Reduced disease vs. increased access to care

DOT’s strategic goals

• Safety

• Mobility

• Economic growth

• Environment

• National security

DOT measures focused on results

Safety Fatality and injury rates

Mobility Congestion, condition of pavement and runways

Economic growth Openness of international aviation markets

Environment Aircraft noise exposure

National security Sealift capacity, military readiness

Causation

• Correlation is not causation

NSF assumes funding causes breakthroughs

• Multiple causes: Identify portion of improvement caused by the agency

USAID: multiple causes of economic growth

Goals and results linked to costs?

VA juxtaposes costs and performance measures

• Cost per patient in Veterans Health

• Cost per pension claim

• Admin. Costs per housing loan

• Included unfunded liabilities

Quality of life

• Focus on results makes this easier to explain (e.g., DOT)

• “What the department is doing” vs. “What citizens are getting.”

• How does avoidance of nuclear accidents make the U.S. a better place to live?

Explanation of failure

• First step is admitting and reporting failures!

• Don’t move the goalposts after the game

Role model: USAID

Major management challenges

• Identified by GAO, IG, agency’s own management

• Most reports are silent on this

Role models: USAID, DOT, Veterans

Changes for next year

• GPRA is a management tool, not just a reporting requirement

DOT combined performance report with 2001 strategic plan

What won’t work

Going through the motions

Recitation of activities with no plot

Salesmanship and hype

Dishonesty

What will work

• Honesty, candor, and humility

• Willingness to recognize reality

• Willingness to learn from experience

• Commitment, rather than compliance • Use of GPRA as a management tool

For more information…

The complete Mercatus Performance Report Scorecard can be reached through our home page ---

www.mercatus.org