presentation training on result based management (rbm) for m&e staff

Three Days TOT on

Results-Based Management

(RBM)

for M&E Staff – ITA By

Fida Karim Head of Monitoring & Evaluation (M&E)

Dated: 10th to 12th August, 2015

Planning, Monitoring, Evaluation & Reporting together for developmental results: Results-based Management-RBM (RBM)?

Logical Framework Approach (LFA) Planning for results Monitoring for results Evaluating for results Enhancing the use of knowledge from monitoring

and evaluation

Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA

2

Day 1: Major Topics To Be Covered

Tuesday, April 18, 2023

RBM is a management approach and project Planning, Monitoring, Evaluation & Reporting tool aimed at to improve management effectiveness and accountability in achieving developmental results.

RBM is focused on chain results: output, outcomes, and impact.

RBM objectively measure how well results are being achieved and report on measures taken to improve them.

It enable senior management and Project Management team to judge if work is going in the right direction, whether progress and success can be claimed, and how future efforts might be improved on the basis of ground realities and data.

RBM is concerned with learning, risk management and accountability.


3

What is Results-Based Management (RBM)?


Tuesday, April 18, 2023Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA

4

• Without proper planning and clear articulation of intended results, it is not clear what should be monitored and how; hence monitoring cannot be done well.• Without effective planning (clear results frameworks), the basis for evaluation is weak; hence evaluation cannot be done well.• Without careful monitoring, the necessary data is not collected; hence evaluation cannot be done well.• Monitoring is necessary, but not sufficient, for evaluation.• Monitoring facilitates evaluation, but evaluation uses additional new data collection and different frameworks for analysis.• Monitoring and evaluation of a programme will often lead to changes in programme plans. This may mean further changing or modifying data collection for monitoring purposes.

Understanding inter-linkages and dependencies between Planning, Monitoring and Evaluation


5

• Traditional M&E mainly focuses on inputs & Activities and success is measured by expenditure (e.g. profit maximization not social welfare maximization)

• Result Based Management (RMB) focuses on internal results and performance of organization in producing developmental results and positive changes in society. •Management for Developmental Results (MfDR) seeks to keep the focus on development assistance demonstrating real and meaningful results at targeted community level.

• RBM & MfDR focuses on good planning, monitoring, evaluation, learning and feeding back into planning to bring continuous improvement in quality of service delivery.

Traditional M&E Vs RBM vs MfDR

Three Days TOT On Result Based Management for M&E Staff; Conducted by : Mr. Fida Karim – Head of M&E Department ITA 6


Problem analysis to understand causes

Stakeholder analysis

Structuring of programmes around a chain of desired results - addressing causes

Causality in the chain of results (if… then logic)

Use of ‘change language’ (future conditional)

Reliance on indicators to measure performance

Costing of results rather than isolated activity budgeting

Three Days TOT On Result Based Management for M&E Staff 7

Common elements of RBM


A Typology for RBM

HIV incidence reduced

Leadership empowered

Skills of Nat’l Aids Comm. strengthened

Train 250 district AIDS officers

Results Like… Focus @

Timeframe

<1 yr

<3 yrs

5 yrs

5-10 yrs

more

less

Collective

Accountability Institutional/

Behavioural

Operational/ skills, abilities,

products & services

Human!

UNOutcome

Impact

Output

Activity

if

if

if

then

then

then

Assumptio

nsUNOutcome

Impact

Output

Activity

Outcome

Impact

Output

Activity


9

Key Features of RBM

Focus on:Analyzing problems and determining their causes;Identifying measurable changes (results) to be achieved based on problem analysis;Designing strategies and activities that

will lead to these changes (results);Balancing expected results with the resources available;



10

Key Features of RBM

Monitoring progress regularly and adjusting activities to ensure

results are achievedEvaluating, documenting and incorporating lessons learned into next planning phase;Reporting on the results achieved and their contribution to achieving goals;



11

Why Results-Based Management?Why Results-Based Management? Improved focus on results instead of activities

Improved transparency, quality & accountability

Improved measurement of programme achievements (performance rather than utilization)

Enhanced strategic focus Industry standard To get more funds!! It is a global trend: using results-based

management to improve the efficiency of development program


RESULTSare measurable changes in knowledge, skills, motivation, behavior, decision making, practices, policies, social action, social, economic and environmental conditions that derive from a cause-and-effect relationship. There are three types of such changes:1. Intended or unintended, 2. Positive or negative3. Direct or indirect.Change can be set in motion by a development intervention – outputs, outcomes and impacts.

.

What is Result?*C

ON

TR

OL

& R

ES

PO

SIB

ILIT

Y

MA

KIN

G A

DIF

FE

RE

NC

E


13

Key RBM Terms (iii)Key RBM Terms (iii)

Inputs

Activities

Outputs Outcomes Impact

How?

What we want? Why?


Results


14

Definitions Definitions (Implementation) (Implementation)

InputInput: these are human, material, financial : these are human, material, financial and other resources that are required to and other resources that are required to undertake activities. undertake activities.

ActivitiesActivities: Actions taken or work : Actions taken or work performed to produce specific outputs performed to produce specific outputs through mobilizing inputs.through mobilizing inputs.



15

Definitions (Results)Definitions (Results)

Output: Output: these are immediate results as a these are immediate results as a consequences of completed activitiesconsequences of completed activities

OutcomeOutcome: these are likely or achieved : these are likely or achieved short-term and medium-term effects of short-term and medium-term effects of outputs.outputs.

Impact:Impact: long-term effects that are the long-term effects that are the logical consequence of the achievement logical consequence of the achievement of the outcomes.of the outcomes.



16

Input? Activity? Output? Outcome? Impact?

Splash (cause)

Ripple(effect)



17

Key principles of RBMKey principles of RBM

Define expected results first and Define expected results first and activities lateractivities later

Foster the active participation of Foster the active participation of stakeholdersstakeholders

Ensure that all stakeholders work Ensure that all stakeholders work towards achieving expected towards achieving expected resultsresults

Apprise your work critically and Apprise your work critically and learn the lessonslearn the lessons



18

Hierarchy of RBM TermsHierarchy of RBM TermsIMPACTIMPACT

Sustainable improvements in society or well-Sustainable improvements in society or well-being of peoplebeing of people

OUTCOMEOUTCOMEChanges in behavior or improvements in access Changes in behavior or improvements in access

or quality of resourcesor quality of resourcesOUTPUTOUTPUT

Product of project/ program activitiesProduct of project/ program activitiesACTIVITIESACTIVITIES

Activities done by project/programActivities done by project/programINPUTINPUT

Resources needed to undertake activitiesResources needed to undertake activities


The LFA is an RBM tool used for systematic planning, implementing, monitoring, and evaluating projects/ programmes.


19

RBM and Logical Framework Approach (LFA)


The Logical Framework Approach (LFA)


20

Features of LFAFeatures of LFA : : stakeholder involvementstakeholder involvement needs-based approachneeds-based approach logical intervention approachlogical intervention approach framework for assessing framework for assessing

relevance, feasibility and relevance, feasibility and sustainabilitysustainability



21

The Logical Framework The Logical Framework

Approach Approach Features of LFAFeatures of LFA : :

results-oriented – not activity driven

logically sets objectives and their causal relationships

shows whether objectives have been achieved: Indicators (for M&E)

describes external factors that influence the project’s success: assumptions and risks Tuesday, April 18, 2023


22

The Result/Logical frame The Result/Logical frame MatrixMatrix

The Logical Framework Matrix is used to The Logical Framework Matrix is used to

present information about project present information about project

objectives, outputs and activities in a objectives, outputs and activities in a

systematic and logical way. systematic and logical way. The basic Logframe matrix contains 16 The basic Logframe matrix contains 16

cells organized into 4 columns and 4 rows, cells organized into 4 columns and 4 rows,

as indicated in the next slide:as indicated in the next slide:



23

The Logical Framework Matrix (LFM)

Objectives & activities

Purpose/ (Outcome)

Goal (Impact)

Outputs

Activities

IndicatorsMeans of

verificationAssumptions


Ways how to depict the Results chain

Logframe


NarrativeIMPACTA. XXXXXXXXOUTCOMEA.1. XXXXXXXXA.2. XXXXXXX XXXOUTPUTSA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXXACTIVITIESA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXX

Results Tree Results Pyramid

Results Chain


Results not logically linked Results not sufficiently specific Results are composites of several results Results don’t express change (e.g. support

provided to strengthen….) Results statements are too wordy Confusion between levels of results Indicators

Not logically linked to the resultNot measurable

Are new results Three Days TOT On Result Based Management for M&E Staff 26

Results chain problems

They can Clarify the scale and scope of a result in the results framework Demonstrate progress when things go right Provide early warning when things go wrong Assist in identifying changes that need to be made in strategy and practice Inform decision making Facilitate effective evaluation

Indicators describe how the intended results will be measured - accountability

Objectively verifiable, repeatable measures of a particular condition

They force clarification of what is meant by the result …….the fine print!

Must be accompanied by baselines and targets


Indicators ‘indicate’ that change is happening or not happening

1. A key test in ensuring whether an indicator is really good is to define “means of verification” for each indicator;

2. Are data sources available?

3. What does it take to obtain the data? Who needs to be involved? How much will it cost to obtain the data?

4. If means of verification are unclear or unrealistic, the indicator and, possibly, the result need to be revisited;

5. The definition of results, indicators and means of verification is an iterative process.


Indicators and Means of Verification

They should be SMART:

SPECIFIC The indicator needs to be as specific as possible in terms of quantity,

quality, time, location, target groups, baseline and target for the indicator

MEASURABLE Will the indicator show desirable change? Is it a reliable and clear measure of results? Is it sensitive to changes in policies & programmes? Do stakeholders agree on exactly what to measure?ACHIEVABLE Are the result(s) realistic and based on risk assessment, partnership

strategy and other factors contributing to the underlying resultRELEVANT Is it relevant to the intended result? Does it reflect the expectations and success criteria for change in

the target groups?TRACKABLE/Time-boundResults are never open-ended—there is an expected date of

accomplishment. Therefore, indicator should be trackable.


What are good indicators?

Unit of analysis; Existing baseline information; Target for subsequent comparison; Expected perceptions or judgments of

progress by stakeholders; Detailed description of expected conditions

or situations to be observed; 1-4 indicators for each result are adequate

– fewer the better to reduce cost.

Three Days TOT On Result Based Management for M&E Staff30

What to specify for each indicator?

Types of indicatorsFactual indicators

Factual (yes/no)

Existence (yes/no) Classes (x/y/z)

Policy recommendation submittedConstitution passed by ParliamentChamber of Commerce established Existence of free electronic media: free, partly free, not free

Numeric indicators Number

Percentage

Ratio

No. of government officials trainedNo. of regional networks on aid effectiveness createdNo. of regional CSOs attending regional conference on human rights

% of government budget devoted to social sectors% of population with access to basic health care

Ratio of female to male school enrolmentRatio of doctors per 1.000 people


Qualitative indicators

Process change

Behavioral changes

Attitudinal changes

ExamplesInclusive and participatory policy making processes in place in 10 countries in the regionPresence of a functional network of local governance practitioners in the regionPro-poor policies formulated Increased level of awareness on human rights among CSOs and governments in the region Quality of judiciary processes improved Improved perception of public on existence of free electronic mediaExtent of involvement of CSOs representing indigenous groups in national planning processesQuality of public-private partnership in the preparation of national plan of action on SME development


Indirect (Proxy) Indicators

Direct indicators are preferable. However, they do sometimes not exist, are too expensive or inefficient to obtain. In such cases, indirect indicator

have to be used.

Purpose Direct indicator Proxy indicator as substitute

Improvements in the national judicial system

Quality of judicial cases improved -> difficult and expensive to obtain

Number of judicial cases challenged in the higher courtsRatio of cases filed to the cases processed in the judicial courts

Increase in per capita family income in the poorest state

household income -> household income survey needed -> not frequent enough

changes in local retail sales Increase in the total number of cell phone users in the state

To increase per capita income of small farmers

Crop sales and food consumption -> survey needed -> too expensive

purchase of typical consumer items improvement in buildings (e.g. roof) of life style (consumption of meat per week)


“If-Then” causality between levels of results

Common results language to describe changes

Collective accountability increases as you move up the chain of results towards outcomes and impacts (Key message M&E can’t do it alone without proper support and commitment of Senior Management !)

A results matrix is a mean not an end

A results matrix is contextual


Principles of RBM

Start with the results Determine indicators to measure progress

towards achieving each result Define explicit targets for each indicator to judge

achievement Collect information to verify/monitor the

achievement/progress Review, analysis and report actual results


35

Key RBM Techniques


RBM are good and useful techniques But it won’t work itself and not sufficient to achieve

results RBM depends on the organization's ability to create a

management culture that is focused on results. Manage change in your organization.


36

RBM


• Getting started• Stakeholder engagement• The planning exercise: Information flow, timeline and method.• Finalizing the results framework • Preparing to operationalize• Putting it together: Planning for change.

Planning for Results


37

• Preparing to monitor by reinforcing the initial M&E plan• Monitor: Collection of data, analysis and reporting: Information flow, timeline and process.• Use of monitoring data for management action and decision making

Monitoring for results


38

Based on the logical framework Strengthens accountability and transparency Provides information for effective

management Helps determine what works well and what

requires improvement Builds knowledge on the basis of best

practices, case studies and success stories etc. regarding developmental results.


39

Monitoring for results


• What is Evaluation?-It’s a management tool to periodically assess an intervention.• Why evaluate? Uses of evaluationIt is a rigorous process to independently assess an intervention in terms of Relevance, Efficiency, Effectiveness, Impact & Sustainability. • Evaluation policy: Principles, norms and standards for evaluation.• Types of evaluationProgramme units

Evaluating for results


40

• Why use monitoring and evaluation?• Learning and generating knowledge from monitoring and evaluation• Knowledge products and dissemination-Case Study-Success Story-Best Practices-Lesson Learnt

Enhancing the use of knowledge from Monitoring and Evaluation


41

In some situations, the problem may have been previously identified and presented with an analysis and proposal for the government, UNDP or other funding partner to consider. A common problem in these situations is that many project proposals are presented to the funding agency with a fixed solution. Quite often, the solution presented only relates to part of a bigger problem. This is often because the agency presenting the proposal tends to be concerned with obtaining financing for the component(s) for which it has a strong interest. For example, an NGO may submit a proposal for assistance to strengthen its capacity to participate in monitoring national elections. While this may be an important project, it is likely that it would only address part of a bigger problem.

Good results-oriented programming requires that all project-level proposals be subject to a problem analysis to determine whether the stated problem is part of a bigger problem and whether the proposed solution will be adequate to address the challenges. The answers to these questions can sometimes be found, particularly in situations where the projects proposed are within the context of an already designed national programme (such as a Poverty Reduction Strategy Paper, UNDAF or country programme). However, in many cases, there will need to be deeper discussions of what the larger problem is and what other actions are needed by different partners to solve that problem. The aim in asking these questions is not to slow down the process of project review and approval but to ensure that problems are analysed properly and appropriate solutions are found. These solutions may involve actions beyond the scope of the specific project. This is one of the differences between a project approach and a results focused approach to development.

Difference between a ‘project’ and a ‘results-based’ approach to development


42

presentation training on result based management (rbm) for m&e staff

Government & Nonprofit

resultsbased management

result based management

meaningful results

internal results

chain results

management approach

based management rmb

management effectiveness