promoting development effectiveness in multilateral...
TRANSCRIPT
Eustace Uzor,
Consultant, IDEV/AfDB
June 22, 2018
AfDB, Abidjan
IDEV Webinar Series
Promoting Development Effectiveness in Multilateral
Development Banks (MDBs): Striking the right balance
between Evaluation Independence and Operational
Embeddedness
Outline
1. Setting the stage
2. The objective of development evaluation
3. Independence of the evaluation function at MDBs
4. Striking the right balance between independence and
embeddedness.
5. Discussion
The Independent Development Evaluation of the African Development Bank
Subtopic
Setting the Stage
The Independent Development Evaluation of the African Development Bank
Subtopic
Development Evaluation = Enhanced Development Effectiveness
The Independent Development Evaluation of the African Development Bank
Subtopic
What development evaluation is not POLICING!
The Independent Development Evaluation of the African Development Bank
Subtopic
Independence in the Development Evaluation Function
What is evaluation independence?
The evaluation function is independent when it is structurally independent
from operational management and decision-making functions within an
organization so that it is free from undue influence, more objective, and has
full authority to submit reports directly to the appropriate levels of
decision-making (UNDP 2016).
Formal independence – can either enable/hinder evaluations.
Substantial independence – ensures integrity, objectivity, impartiality.
The independence of evaluation departments/offices should not
translate to isolation from operations
The Independent Development Evaluation of the African Development Bank
Subtopic
S
Striking the right balance between independence and embeddedness
Problem Statement:
Although independence is necessary to ensure high-quality and objective (White
2014), it is not a guarantee for a quality evaluation product (ADB, 2008).
Institutional independence does not necessarily safeguard against biases
toward positive evaluation (poor quality evaluation designs, contract
renewal bias, and ‘friends’ bias etc) World Bank (2003).
These biases are best overcome by having a good quality assurance system
What degree of embeddedness with operations is required to improve
evaluation quality and make evaluation recommendations more
actionable?
The Independent Development Evaluation of the African Development Bank
Subtopic
S
Striking the right balance between independence and embeddedness
Rationale for engaging with Operations/Embeddedness:
The Independent Development Evaluation of the African Development Bank
Subtopic
The BUSD-heuristic (betterment, undermining, support, distortion) Model of Influence
Explicitness of Influence
Explicit Implicit, Indirect
Direction of
Influence
Destructive
Distortion
Negative Pressure
Undermining
Negative Persuasion/Counsel
Constructive
Betterment
Positive Pressure
Support
Positive Persuasion/Counsel
Source: Pleger and Sager (2016)
S
Striking the right balance between independence and embeddedness
Engagement with Operations - Current Practice:
The Evaluator General is regularly invited to senior management meetings
such as Senior Management Coordination Committee (SMCC), and the
Operations Committee (OPSCOM), to ensure inclusion of the independent
evaluation function, and support feedback of evaluation knowledge (AfDB,
2016).
Capitalization Workshops, with 1-2 pages of Management Response to
Committee on Operations and Development Effectiveness (CODE).
Involvement from ToR, Reference Group, to Learning Events on
Evaluations.
The Independent Development Evaluation of the African Development Bank
Subtopic
S
Striking the right balance between independence and embeddedness
However, IDEV staff recuse themselves from evaluating:
Any project, program, or activity that they worked on or had line
responsibility for the work on, including preparation, appraisal, supervision,
and completion reporting, or that they had a personal influence over, in a
previous capacity either in the Bank or prior to joining the Bank (AfDB 2016,
Annex 1).
In contrast, the Independent Development Evaluation of the Asian
Development Bank (ADB) comments on and reviews important projects or
programs and TA before approval (ADB 2008).
What degree of embeddedness is sufficient to maintain
independence?
The Independent Development Evaluation of the African Development Bank
Subtopic
Striking the right balance between independence and embeddedness
My view:
Review and Validate the Results Frameworks and Theory of Change of operations/projects, ex ante.
The Office of Evaluation and Oversight at the Inter-American Development Bank reviews the proposed monitoring and evaluation (M&E) systems of projects at the design stage, rather than waiting around to criticize them once the project is over (White 2014).
Rotate Young Professionals and Low-Mid-level Evaluation Staff to Operational Departments for a few months, as part of onboarding.
ADB encourages evaluation staff rotation – in principle.
Is Independence preferred over influence?
The Independent Development Evaluation of the African Development Bank
Subtopic
Thank you for your time!
The Independent Development Evaluation of the African Development Bank
Subtopic
References
1. ADB. 2008. “Review of the Independence and Effectiveness of the Operations EvaluationDepartment.” Mandaluyong, Philippines:. Asian Development Bank (ADB).
2. AfDB. 2016. “Independent Evaluation Policy: The African Development Bank Group.”Abidjan, Côte D’Ivoire:. Independent Development Evaluation (IDEV), AfricanDevelopment Bank Group.
3. Pleger, Lyn, and Fritz Sager. 2016. “Betterment, Undermining, Support and Distortion:A Heuristic Model for the Analysis of Pressure on Evaluators.” Evaluation and ProgramPlanning 69: 166–72.
4. UNDP. 2016. “Evaluation and Independence: Existing Evaluation Policies and NewApproaches.” New York:United Nations Development Programme IndependentEvaluation Office.
5. White, Howard. 2014. “Is Independence Always a Good Thing?” International Initiativefor Impact Evaluation (3ie). http://blogs.3ieimpact.org/is-independence-always-a-good-thing/.
6. World Bank. 2003. “Independent Evaluation: Principles, Guidelines And Good Practice.”The World Bank Development Grant Facility (DGF) Technical Note. Washington D.C.:World Bank.
The Independent Development Evaluation of the African Development Bank
Subtopic