1

PERFORMANCE MEASURES:So WHY should I care?

Remarks by Sherry Sterling

Senior Advisor, OPP

PREP Course

13 June 2005

2

Critical Components of Program Success

CLEAR GOALSPublic Health & Ecological

Protection Outcomes

GOOD PLANNINGStrategic Objectives

EFFECTIVE IMPLEMENTATIONMultiple Levels

ADEQUATE RESOURCESAcquisition & Allocation

MEANINGFULACCOUNTABILITYMeasures & Tracking

3

Accountability Drivers in EPA

Statutes

EPA 5 Year Strategic Plan

3 Year NPM Commitments

Budget Formulation

Annual Work Plan

Performance Standards

Budget Allocation

Audits

Tracking Systems/Reporting

4

Government Performance and Review Act (GPRA)

GPRA is a statute aimed at increasing accountability for all federal agencies by requiring measurement of program results & linking those results to resource allocation.

Provides a statutory framework for strategic planning and reporting that requires federal agencies to:– Establish accountability measures to assess the outcomes of

each program activity, and – Prepare performance plans that establish annual performance

goals.

5

GPRA: Auditing Function

Office of Management & Budget (OMB) assumed the role of

evaluating progress against GPRA, developing the Program

Assessment Rating Tool (PART)

OMB is committed to audit all federally funded programs against

GPRA requirements on a 5 year cycle.

This is the third year. Several Pesticide Programs have been

audited.

6

Principle focus has been on Outcome Measures.

Short turn-around times for agencies to develop their data.

Average scores have been very low; many classified as “Results

Not Demonstrated”.

“Re-PARTing” is the norm.

Results are published each year with the federal budget.

Programs not demonstrating results for successive years are

subject to budget cuts or elimination.

PART: Outcomes

7

How do we measure?OUTPUTS

An activity, effort, or work product related to a goal

May be quantitative or qualitative Must be measurable during the funding

period Examples: number of policies developed,

number of REDs issued

8

How do we measure?OUTCOMES

The result, effect, or consequence that will occur from carrying out a program or activity related to a goal

Must be quantitative May not be achievable during the funding

period Examples: reduced occurrence of disease,

fewer incidents

9

Levels of Outcomes

Short-Term– Most direct and measurable– 1 to 3 years

Intermediate– Less measurable– 4 to 6 years

Long-Term– Difficult to measure impacts– 7 to 10 years

10

How do we measure?EFFICIENCY

Ratio of outputs to cost Cost can be dollars or time Examples: cost per purchase order

transaction, cost per seminar conducted

11

Framework for Action in OPP

Senior Level Steering CommitteeFOCUS: Set Direction; Manage Process; Resolve conflicts/issues

Task GroupsFOCUS: developing outcome and efficiency in specific areas

ESAWater Quality

PESPSAIFood Safety

WorkerSafety

??? ???

Coordinating CommitteeFOCUS: developing outcome and efficiency measures for MISSION AREAS; coordinating work of task groups

12

Process highlights…

Inclusive – HQ, Regions, States, Tribes, Stakeholders

Performance measures will serve as an important management tool

Very tight timeframe

13

Products expected from process…

Protect Human Health

Protect Environment

Other Benefits from Use

Worker Safety Yes Yes

Food/Agg. Exp. Yes Yes Yes

ESA Yes Yes

Water Quality Yes Yes Yes

SAI + PESP Yes Yes Yes

14

What’s a Logic Model?

A systematic and visual way to develop and present the causal links between the:– resources available to operate a program– activities conducted with those resources– products or services produced by the activities– intended changes or results

15

Why the Logic Model?

Simple to understand and use Links budget to goals Helps to identify what to measure Communication tool Endorsed by National Grants Office as a way

to address EPA Order 5700.1 Endorsed by OMB (conference website)

16

Logic Model Template

Resources Activities Outputs Outcomes/Results

What do we need to operate this program well?

What are we doing to achieve our intended results?

How many, how often, over what duration?

So what?

What difference are we making?

17

Criteria for Good Measures

VALUEMeaningful Outcomes BeansUnderstandable

Clear ComplexUseful

Sliced bread File 13

BURDEN# Measures

Few ManyCollecting Difficulty

Simple HardData System Cost

Low HighExpertise required

User-friendly Geeks

18

Performance Measures Quadrant Analysis

Value

Burden

OUTCOMENIRVANA

OUTPUTHELL

Low

High

High Low

19

Example of a good measure….

From OMB’s website Efficiency measure: Percent reduction in

review time for registration of conventional pesticides

20

Why is this a good measure?

Properly assists the program in improving its administrative efficiency

Important to stakeholders Reduction in time, taking into account various

steps in the process, is a valuable method for improving efficiency and potentially cost effectiveness

21

WHY do we care? Because…

As public servants, we must assume responsibility

We are always searching for the highest quality public service at the lowest cost

We want to use whatever information is available to improve programs

Remembering that we are the trustees for the public’s monies

22

WHY do we care? Because…

Meaningful Accountability =

Good Management =

Good Government