HOW TO WRITE A GOOD TERMS OF REFERENCE FOR FOR EVALUATION

Programme Management Interest Group 19 October 2010

Pinky Mashigo

PRESENTATION FORMAT

• Introduction

• Definitions explored

• Logframe and Evaluation

• Link between Logframe and Evaluation Criteria

• Components of ToR

• Summary – Reminder for emphasis

INTRODUCTION

• Terms of Reference in general provides a guide towards results that can be used for decision making, planning and learning

• The theme today is EVALUATION• There is a tendency to use certain terms

inter-changeably or linked without acknowledging that they all mean different things to different people

• If terms are used in this manner they tend to confuse outcomes, outputs expected from the study, project purpose, etc.

INTRODUCTION CONT’D

• In order to avoid the confusion, it best we define these terms:

• These are Monitoring, Evaluation and Audits

DEFINITIONS

Monitoring Evaluation Audit

Who ? Responsibility is at all levels of management (internal)

Usually incorporates external inputs (objectivity)

Incorporate external inputs

When ? Ongoing Periodic – midtermCompletionEx –post

Ex – Ante(systems reviews)Completion – Compliance

Why ? Check progress , take remedial action, update plan, control

Learn broad lessons, application to other activities, decision making, input to policy review,

Provides assurance and accountability to stakeholders Provide recommendation for improvements

Links to the Results Chain

Inputs, Activities, and Outputs

Depends on focus - Across the results chain

Inputs, Activities and Outputs

DISTINCTIONS BETWEEN THE THREE

• Monitoring: Ongoing analysis of (project) progress towards achieving planned results, with the purpose of improving management decision making

• Evaluation: systematic collection and analysis of data to ascertain efficiency, effectiveness, impact, relevance and sustainability of project, programme, policy, etc.

• Audit: i. analysis of the legality and regularity of project expenditure &

income, ie. Compliance with the laws & regulationii. analysis whether funds have been used efficiently and

economically in accordance with sound financial management iii. whether the project funds have been effectively used for the

purpose intended

LOGFRAME AND EVALUATION

• Having understood the difference – focus is how to write a good Terms of Reference for an Evaluation exercise

• Key to any evaluation is the Results Chain (Logic Model) Or Logframe

• Logframe guides in terms of what kind of an evaluation and what kind of results are expected

IMPACT

OUTCOMES

OUTPUTS

ACTIVITIES

MEANS

PROBLEMATIC SITUATION

IMPACTIMPACT

EFFECTIVENESSEFFECTIVENESS

EFFICIENCYEFFICIENCY

RELEVANCERELEVANCE

SU

ST

AIN

AB

ILIT

YS

US

TA

INA

BIL

ITY

LOGFRAME OBJECTIVELOGFRAME OBJECTIVE EVALUATION CRITERIAEVALUATION CRITERIA

LINK BETWEEN EVALUATION CRITERIA

EVALUATION CONT’D

• Therefore, Terms of Reference must reflect what you want to achieve/intent

• The ToR is stage 1 of the evaluation process, which is very key:

• Who should design the ToR, all stakeholder involved in the project, policy to be evaluated

COMPONENTS OF TOR

1. Background and Rationale (Introduction)2. Purpose/Objective of the study

• What is the focus of the evaluation • How will the results be used/utilised• Target (primary and secondary users)• Timeframe

3. Issues to be studied – Define scope• Mid-term• End • Ex – Post

• Guide the evaluator in terms of focus, expectation, target group, utilisation of results

COMPONENTS OF TOR CONT’D

4. What are the assumptions and risks5. Plan of Work – Evaluation design

• Methodology of the study (provide guide)• Secondary or primary • Survey (what type)• Quasi experimental

• Importance: guide on costs, implications and link back to the purpose. Can it be achieved!

(a). Data elements – sources for collection; define method and tools to collect data

(b). Data analysis - what method will be used, what tools can be used

(c). Stakeholders – who should be consulted

COMPONENTS OF TOR CONT’D

5. Expertise Required

Team members: • Define competencies, experience & skills• If required define qualifications as well

Composition of the team• Number • Language where required

COMPONENTS OF TOR CONT’D

6. Reporting • Language of the report• Date of delivery • Number of copies required (recipients)• Type of reports (recipients) and timelines

• Inception• Data analysis report • Ist draft • 2nd draft • Final report

COMPONENTS OF TOR CONT’D

7. Administration and Logistical Support

- Management of the evaluation, who is responsible for what

- Define all the support to be provided to the evaluator (define boundaries)

8. Time Schedule

- Duration of the study

9. Indicative Budget

SUMMARY

• Of Importance:– Level of evaluation (efficiency, effectiveness,

impact & sustainability) – Purpose – ToR

• Issues to be studied• Team • Indicative budget

– Respondent to the ToR, should be given a chance to ask questions for clarification, correction, etc. Prior to the proposal being submitted