Evaluation in the Government of Canada

Robert Lahey

Senior Director, Centre of Excellence for Evaluation

Treasury Board Secretariat of Canada

16 October 2001

Centre of Excellence for Evaluation 2

1. Canada at a Glance2. Key Players in Evaluation3. Perspectives on the Past4. TBS Study 5. New Agenda: Results for Canadians 6. New Evaluation Policy 7. TBS Centre of Excellence for Evaluation8. Community Renewal: Competency Profile9. Resources

Outline of Presentation

Centre of Excellence for Evaluation 3

Sovereign parliamentary democracy

Population 31 million

Federation of 10 provinces and 3 territories

Areas of federal responsibility include: defence, criminal law, postal service, census, copyrights, trade regulation, external relations, money and banking, transportation, citizenship, and Indian affairs.

Canada at a Glance

Centre of Excellence for Evaluation 4

Government Accountability

Parliament of CanadaHouse of Commons

Auditor General(appointed)

-Independent audits of govt operations

Parliamentary Committees

PM and Cabinet

TB Minister

Treasury Board Secretariat(secretary appointed)-government policy- oversees spending

Public servants

All other ministers

Federal depts/agencies(DMs appointed)

-government operations- approved budgets

Public servants

Centre of Excellence for Evaluation 5

Treasury Board Secretariat (TBS) Government’s management board (financial,

management and human resources) Comptrollership function Centre of Excellence for Evaluation Policies and standards Capacity building Links evaluation and performance

measurement

Key Players in Evaluation

Centre of Excellence for Evaluation 6

Auditor General (AG)

Conducts independent audits of government operations

Produces periodic oversight reports on the conduct of evaluation

Promotes accountability and best practices

Reports directly to Parliament

Key Players in Evaluation

Centre of Excellence for Evaluation 7

Departments/Agencies Deputy Ministers (DMs) accountable for the

application of Evaluation Policy within their departments

Heads of Evaluation implement policy as per TBS standards and guidelines

Internal accountability and reporting to DMs External accountability and reporting to TBS and

Parliament

Key Players in Evaluation

Centre of Excellence for Evaluation 8

Graphical Interpretation of the Extent of Formal Evaluative Activity in the Federal Government

1960s 1970s 1980s 1990s 2000s

Perspectives on the Past: Activity Graph–

Inte

nsi

ty o

f ac

tivi

ty

+

Centre of Excellence for Evaluation 9

Growing demand for information to support increasingly complex and costly programs

New management systems created for financial administration and planning, programming and budgeting

Evaluation as a practice not yet formalized

Lessons Learned Need for formal evaluation increases as resources

become scarcer and the identification of priorities becomes more important

1960s

Centre of Excellence for Evaluation 10

Building of key infrastructure elements in departments

Treasury Board Evaluation Policy (1977) was the first formalized evaluation policy in Canada

Evaluation Policy centre created within the new Office of the Comptroller General (1978)

Lessons Learned Necessary, but not sufficient, conditions for an

effective evaluation system are: government investment and support; formalized policies and standards; and, leadership for capacity building

1970s

Centre of Excellence for Evaluation 11

High expectations for evaluations not always met Government-wide review led by Parliament

Lessons Learned Evaluation quality depends on an approach that

balances: timeliness, usefulness, methodological purity, client requirements and cost

1980s

Centre of Excellence for Evaluation 12

Evaluation capacity affected by general government downsizing

AG report (1993) indicated renewal of evaluation capacity needed

New Review Policy (1994) linked evaluation closer to internal audit

Lessons Learned Critical mass in capacity is required to ensure

evaluation remains credible, relevant and strategic

1990s

Centre of Excellence for Evaluation 13

TBS study pointed to uneven delivery of evaluation across departments

Strong support to reinvest in evaluation Commitment to linking evaluation to broader

accountability and reporting requirements Objectivity, not independence, needed

TBS Study (2000)

Centre of Excellence for Evaluation 14

Agenda to:

Improve government services and respond to demands for better value and transparency

Move from reporting on results to managing for results

Applied through a wide-reaching series of initiatives, including the new Policy on Evaluation (2001)

Perspective on modern management shared by the political level and the Public Service

New Agenda: Results for Canadians (2000)

Centre of Excellence for Evaluation 15

Re-emphasis on the need for evaluation to be strategic, comprehensive and systematic

Scope broadened to include programs, policies and broad initiatives

Provides guidance on standards

Evaluators are encouraged to work directly with managers to build evaluation into the “life cycle” of programs

Highlights links between evaluation and results-based performance measurement

New Evaluation Policy: Highlights

Centre of Excellence for Evaluation 16

Managers are responsible for the active monitoring of their programs

Government is committed to the public reporting of evaluations

Centre of Excellence for Evaluation created in TBS

New funding available for departments to build and meet future evaluation capacity

Highlights (continued)

Centre of Excellence for Evaluation 17

Six standards for evaluation: Evaluation planning and issues Competency Objectivity and Integrity Consultation and Advice Measurement and Analysis Reporting

Guidance based on ‘good practices’

New Evaluation Policy: Standards

Centre of Excellence for Evaluation 18

TBS Centre of Excellence for Evaluation (CEE)

Pro-active leadership role in the renewal of evaluation in Canada through:

Building evaluation capacity Communicating and networking Policy development, implementation and

evaluation Repositions and renews the evaluation function

across government Emphasizes partnerships and knowledge

management

Centre of Excellence for Evaluation 19

Draft report (2001) identifies two key attributes of successful evaluators:

Cognitive capacity Ability to communicate well

The right people: Respect diversity Work collaboratively and openly Recognize and diffuse conflict Demonstrate sensitivity, tact and empathy Care about professional practices and standards

Community Renewal: Competency Profile

Centre of Excellence for Evaluation 20

Resources: Evaluation Infrastructure Checklist

Cultural institutions are prepared to

divulge information managers trust that

assessments will be objective agencies are willing to be

reviewed managers have the courage

to make changes and implement recommendations

evaluation function is prepared to evaluate itself

relevant accountabilities have been clarified

Operational technical, professional and

financial resources are available

time is sufficient evaluation policies and

standards are in place business/strategic plans are

developed need for objectivity can be

met authority exists to oversee

evaluations and act on findings

Concise Version

Centre of Excellence for Evaluation 21

Treasury Board Secretariat of Canada (TBS) www.tbs-sct.gc.ca

TBS Centre of Excellence for Evaluation www.tbs-sct.gc.ca/eval

TBS, Guide for the Development of Results-based Management and Accountability Frameworks

www.tbs-sct.gc.ca/eval/pubs/rmaf-cgrr/rmaf-cgrr-01-e.asp

TBS, Evaluation Policy www.tbs-sct.gc.ca/pubs_pol/dcgpubs/TBM_161/ep-pe_e.html

Resources: Selected GoC Web Sites

Centre of Excellence for Evaluation 22

Robert LaheySenior DirectorCentre of Excellence for EvaluationTreasury Board of Canada Secretariat300 Laurier StreetOttawa, Ontario K1A 0R5CANADA

Telephone: (country code 1) – 613 - 957-2602

Fax: (country code 1) – 613 - 952-1782

Email: Lahey.Robert@tbs-sct.gc.ca

Contact Information