1. BASIC INFORMATION

a. Basic project data

Project title: Gender Responsive Skills and Community Development Project (GRSCDP) Project code: P-GH- IDO 003 Instrument number(s): Loan and Grant Project type: Investment Sector: Social Country: Ghana Environmental categorization (1-3) : 3

Processing Milestones Key Events Disbursement and Closing date

Date approved: 19/12/07 Cancelled amount: Original disbursement deadline: 31/12/12

Date signed: 14/05/08 Supplementary financing: Original closing date: 31/12/12 Date of entry into force : 20/01/09

Restructuring: Revised disbursement deadline: 30/09/15

Date effective for 1st disbursement:20/01/09

Extensions (specify dates): 31/12/13 and 30/09/15

Revised closing date: 30/09/15

Date of actual 1st : 13/10/09

b. Financing sources NOTE: The PCR reports financing in a very confusing manner mixing loan and expenditure in the funding part of the table. The correct figures seem to be as follows

b. Financing sources

Financing source/ instrument (MUA)

Approved amount

(MUA) : Disbursed amount

(MUA) : Percentage disbursed

(%):

Loan: 5.95 2.09 35.13

Grant: 2.36 1.5 63.56 Government: 1.09 0.5 45.87

Other (ex. Co-financiers):

TOTAL : 9.4 4.09 43.5

Co-financiers and other external partners:

Execution and implementation agencies: Ministry of Women and Children’s Affairs later renamed Ministry of Gender ,Children and Social Protection (MoGCSP) c. Responsible Bank staff

Position At approval At completion

Regional Director J. Litse Abdellatif Bernoussi

Sector Director Tom Hurley Sunita Pitamber

Sector Manager Sibry Tapsoba Justin Murara

Task Manager Yeshi Dejene Mulle Chikoko

Alternate Task Manager Efua Amissah-Arthur PCR Team Leader Ms. Mulle Chikoko

PCR Team Members Ms Efua Amissah-Arthur; Mr

PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS

Philip Doghle; Mr Daniel Osei-Boakye; Procurement Officer; Mrs Joana Opare.

d. Report data

PCR Date : SEPTEMBER 2016 PCR Mission Date: From: 5/09/16 To: 16/09/16 PCR-EN Date: 5 July 2018

Evaluator/consultant : Stephen CUNNINGHAM Peer Reviewer: Judith OFORI

2. PROJECT DESCRIPTION Summary from Appraisal Report including addendum/corrigendum or loan agreement, and taking into account any modification

that occurred during the implementation phase.

a. Rationale and expected impacts: Provide a brief and precise description on the project/programme rationale (concerns/questions raised), expected impacts and the

intended beneficiaries (directly or indirectly impacted by the project/programme). Highlight any change that occurred during the

execution phase.

The Project aims to reduce gender-based inequalities as an effective means to reduce poverty within the

context of GPRS II and is based on the need to increase women’s access to and control over productive

assets and improved access to social service and protection. The Project aligns with the

third pillar of GPRS II: ‘vigorous human resource development’ which seeks to promote skills and

entrepreneurial development.

The Project is also aligned with the Bank’s Country Strategic Paper (2005-2009) under the second pillar

“support pro-poor and pro-gender-equity policies”. The Project will contribute to the MDGs related

to reducing extreme poverty and hunger and gender equality.

The beneficiaries are intended to be the Ghanaian population as a whole but women in particular.

The main impact will be to contribute to gender equitable socio-economic development by reducing the

rate of women living in extreme poverty by 7.5% by end of project implementation and by 10 % by

2015 as measured by National statistics and GPRS

Related to this will be an improvement by 2012 in national capacities for enhanced gender

mainstreaming as measured by:

Percentage of MDAs budget allocated for gender related activities increased by 20% from the

current level

Number of gender sensitive projects increased at district level by 30% from current level

All Bank-financed projects are gender-sensitive projects.

And an enhancement of women’s marketable skills through greater access to quality training & an

improvement in women’s productivity and self-sufficiency through greater access to services in micro

and small entrepreneurship, measured by:

80% of skills trainees gainfully employed or self-employed

Percentage of women having access to Micro Finance Institutions (MFIs) increased by 25%

from current rate

Volume of average loans taken by women micro and small entrepreneurs increased by 25%

from current rate

b. Objectives/Expected Outcomes: Provide a clear and concise description of the project objectives, expected outcomes, and intended beneficiaries. In so doing,

highlight any revision/amendment.

The project goal is to promote gender equitable socio-economic development. The objectives are to: (i)

improve national capacities for enhanced gender mainstreaming and (ii) improve access to quality skills

training for gainful employment and entrepreneurial development of women.

The expected OUTCOMES are:

Longer-Term Outcomes: The living conditions of disadvantaged groups, women in particular have

improved

Medium-Term Outcomes:

1.1 Human resources base and professional competency for increasing gender mainstreaming into

development management strengthened

2.1 Women’s marketable skills enhanced through greater access to quality training

2.2 Women’s productivity and self-sufficiency improved through greater access to services in micro and

small entrepreneurship

c. Outputs and intended beneficiaries: Provide a clear and concise description the expected outputs and intended beneficiaries. In so doing, highlight any

revision/amendment.

The expected outputs from the Logical Framework are:

1.1 Capacity built within MOWAC, selected line ministries, agencies, and district level administrative

structures through scholarships, training programs and workshops, revision of gender training

curriculum, gender analysis of key planning frameworks, and improvement of system of gender

disaggregated data collection and analysis

NB There is no number 2 in the results matrix; the numbering here is as per the matrix

3.1 Curriculum improved and more responsive to market needs

3.2 Training centers rehabilitated and equipped

3.3 Teachers trained

3.4 Enrolment increased and girls encouraged to study non-traditionally female trades

4.1 Access to internet and ICT services increased

4.2 Capacity for service delivery to women in micro and small entrepreneurship improved

5.1 Efficient management of project activities

d. Principal activities/Components: Provide a clear and concise description of the principal activities/components. In so doing, highlight any revision/amendment.

Annex III of the Appraisal Report includes a Project Implementation Schedule that lists the main

activities by Component as follows:

Component 1 Institutional Strengthening of Enhancing Gender Mainstreaming

1. Assessment of Training Needs and Existing Curricula and Revision of Curricula and field testing

and training of trainers

2. Training of professionals

3. Training of Decision Makers

4. Staff selection for long term and training

5. Study on Gender and Culture & Validation Workshop

Component II Support To Skills Training and Entrepreneurial Development

1. Study on Vocational Skills conducted, field tested

2. Teachers in new curricula trained

3. Construction and Upgrading of Centers

4. A committee for scholarship selection and award established

5. Scholarship to girls awarded

6. Study on training needs of women entrepreneurs

7. Study on women’s access to micro-credit and the capacity of MFI conducted and validated

8. Training of staff of MFI, BDS, GHAMFIN, NBSSI, DCD extension workers

9. Data base of service providers operational

Component III: Project management and coordination

1. Midterm-Review

2. PCR

3. PROJECT PERFORMANCE ASSESSMENT

IMPORTANT NOTE: It was not possible to undertake an independent evaluation of this

project because the only documents available to the evaluation were the project appraisal and

the PCR. The following sections simply review the PCR. The PCREN has not been able to

provide its own scores, only to suggest corrections to the PCR scores in a couple of places.

RELEVANCE

a. Relevance of the project development objective: Evaluation of the relevance ex-ante and ex-post (including during the implementation phase). The relevance of the project

objective (during the evaluation ex-ante and the post-evaluation) in terms of alignment with country’s development priorities

and strategies, the beneficiary needs (including any changes that may have occurred during the implementation), applicable

Bank sector strategies, the Bank country/ regional strategy, and general strategic priorities of the Bank. This criterion equally

assesses the extent to which the project’s development objective was clearly stated and focused on outcomes and the realism of

the intended outcomes in the project setting.

The discussion presented by the PCR is consistent with a rating of 4.

b. Relevance of project design (from approval to completion): The evaluator should provide an assessment of the relevance of the project design regardless of the one provided in the PCR.

The evaluator will also comment on the PCR conclusion for this section, and will provide an evaluation of the relevance of the

project design. The latter assesses the soundness and the timing of eventual adjustments, or technical solutions to ensure the

achievement of the intended results (outcomes and outputs), the adequacy of the risk assessment, environmental and social

protection measures, as well as the implementation arrangements. For Programme Based Operations (PBO), an assessment will

be made on the relevance of the prior actions, the policy dialogue and the extent to which the operation could have been more

pro-poor in its design.

The points made about the project design are consistent with a rating of 3, although the number of

potential beneficiaries (section 4.2 of the Appraisal report) seems to be very optimistic.

EFFECTIVENESS

c. Effectiveness in delivering outputs: Evaluation of the extent to which the project achieved its stated results (obtained from the logical framework) based on the last

Implementation Progress and Results Report (IPR) and by considering accurate reporting of direct or indirect evidence on

intended and unanticipated outputs. In the absence of sufficient data (as direct evidence), indirect evidence (such as project

outcomes and other pertinent processes/elements of the causal chain) should be used particularly in the evaluation of the extent

to which the project is expected to achieve its stated results/ objectives. The absence of sufficient data to assess the effectiveness

should be indicated (and clearly detailed in the PCR quality evaluation section). The PCR score should equally be indicated in

this section.

The report on the outputs is clearly evidence-based and fully justifies the rating of 2.

d. Effectiveness in delivering outcomes: Evaluation of the extent to which the project achieved its intended set of outcomes (including for Program Based Operations

(PBOs) where complementary measures are necessary for their implementation, namely public awareness, policy dialogue and

institutional arrangements for instance). The evaluator should make an assessment based on the results of the last project

Implementation Progress and Results (IPR). The evaluator shall indicate the degree to which project outcomes (intended and

unanticipated) as well as reasons for any eventual gap were discussed in the PCR.

The assessment presented in the PCR is evidence-based insofar as evidence is available. In many

cases, such evidence is lacking suggesting that the M&E system was ineffective. The rating of 2 is

justified by the discussion in the PCR.

e. Project development outcome:

The ratings derived for outcomes and output are combined to assess the progress the project has made towards realizing its

development objectives, based on the rating methodology recommended in the Staff Guidance Note on project completion

reporting and rating (see IPR Guidance Note for further instruction on development objective rating).

The PCR has a rating of 3 for the PDO. However, given that both outcomes and outputs were rated to

2, the PDO rating should also be 2.

f. Beneficiaries: Using evidence, the evaluator should provide an assessment of the relevance of the total number of beneficiaries by categories

and disaggregated by sex.

The beneficiaries discussed in the PCR represents only a small subset of those mentioned in the

Appraisal Report (section 4.2) although as noted above the estimate of potential beneficiaries in that

Report seems to be very optimistic.

g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the project

logical framework): This includes gender, climate change, as well as social and socio-economic- related issues. Provide an assessment of the extent to

which intended or unanticipated additional and important outcomes have been taken into consideration by the PCR. The

assessment should also look at the manner the PCR accounted for these outcomes.

The PCR identifies only that some numbers trained were higher than expected

EFFICIENCY

h. Timeliness: The timeliness of project implementation is based on a comparison between the planned and actual period of implementation

from the date of effectiveness for first disbursement. For Programme Based Operations (PBOs), the timely release of the

tranche(s) are assessed through this same criterion.

The PCR says here that the end date is Dec 2013 but there was an extension in the disbursement

deadline to 30 Sept 2015 that presumably also affected the closing date. If so, instead of 4 years the

project actually ran for almost 7 years rather than the 5 claimed in this section by the PCR. There is no

rating in this section of the PCR but the summary table includes a 2 rating for timeliness which seems

to be the maximum that could be given.

i. Resource use efficiency: Provide and assessment of physical implementation (based on outputs delivered) against resources used (based on cumulative

commitments) at completion for all contributors to the project (the Bank, Government, and others). This criterion would

normally not apply to PBOs, as there is often no direct link between the outputs and the amount of contribution (in which case

the rater would indicate N/A).

A substantial amount of the special fund was not justified and disbursement rate was extremely low.

The project could only use 43% of financial resources in 7 years of implementing a 4 year program.

With 56.5% of undisbursed funds, special account unjustified at PCR, Fraud investigations, the

Review recommends Highly unsatisfactory (1) for Resource use efficiency.

j. Cost-benefit analysis: Provide an assessment of the timeliness of the development outputs, and the extent to which costs of the costs have been effective

and have been provided in the most efficient manner. The PCR rating should be discussed. The evaluator should verify whether

the benefits of the project (achieved or expected) exceed its actual costs. To achieve this, evidences will mainly be based on a

comparison between Economic Rates of Return (ERR) calculated at appraisal, the mid-term review and completion. When

commenting PCR ratings, the degree of utilization of valid sources for evidence justifying the rating assigned should be taken

into consideration. The evaluator should ensure of the validity of assumptions and that the same model was used for the

calculation of others ERRs. For PBOs for which this calculation model does not apply, an assessment could be done with regards

to the contribution of policy reforms to economic growth. In the absence of sufficient evidence, an appropriate rating should be

assigned.

N/A

k. Implementation progress: The assessment of the Implementation Progress (IP) on the PCR is derived from the updated IPR and takes into account the all

applicable IP criteria assessed under the three categories : i) Compliance with covenants (project covenants, environmental and

social safeguards and audit compliance), ii) project systems and procedures (procurement, financial management and

monitoring and evaluation), and iii) project execution and financing (disbursement, budget commitments, counterpart funding

and co-financing).

Progress is reported to be good on component 1 only. The PCR raises some very serious issues

concerning Project Management and the 2 rating seems appropriate

SUSTAINABILITY

l. Financial sustainability: Provide an assessment of the extent to which funding mechanisms and modalities (eg. Tariffs, user fees, maintenance fees,

budgetary allocations, other stakeholder contributions, aid flows, etc.) have been put in place to ensure the continued flow of

benefits after completion, with particular emphasis on financial sustainability. For PBOs, the assessment should focus on

financial sustainability of reforms, as well as the Bank’s policy dialogue to promote financial sustainability of the reforms.

This aspect seems to be a disaster – not only are there no apparent plans for post-project financial

sustainability, it seems that there is not even a strategy to deal with scholarships that overhang the

project closure date. This is completely unacceptable and the Evaluation strongly agrees with the 1

rating given by the PCR (and hopes that a successful solution was later found).

m. Institutional sustainability and strengthening of capacities: Provide an assessment of the extent to which the project has contributed to the strengthening of institutional capacities –

including for instance through the use of country systems – that will continue to facilitate the continued flow of benefits

associated with the project. An appreciation should be made with regards to whether or not improved governance practices or

improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For

PBOs, this should include an assessment on the contributions made to building the capacity to lead and manage the policy

reform process; the extent to which the political economy of decision making was conducive to reform; the Government’s

commitment to reform; and how the design reinforced national ownership.

Given what is said in the PCR the rating of 2 may seem a little on the harsh side. However, it is true

that there is no evidence to demonstrate the impact of the activities and outputs so the 2 rating does

seem to be appropriate.

n. Ownership and sustainability of partnerships: Provide an assessment of whether the project has effectively involved relevant stakeholders, promoted a sense of ownership

amongst the beneficiaries (both men and women) and put in place effective partnerships with relevant stakeholders (eg. local

authorities, civil society organizations, private sector, donors) as required for the continued maintenance of the project outputs.

For PBOs, the assessment should measure the extent to which the Government’s capacity to conduct consultations during policy

dialogue and the extent to which the Bank supported the Government in deepening the consultation processes.

From the PCR it appears that ownership began to be created but was lost with changes in personnel.

The 2 rating seems justified

o. Environmental and social sustainability: Provide an assessment of the objectivity of the PCR rating on the project’s implementation of environmental and social

mitigation/enhancement measures with regard to the Environmental and Social Management Plan (ESMP), the capacity of

country institutions and systems, as well as the availability of funding to ensure the environmental and social sustainability of

the operation. This criterion would normally only apply to Environmental Category I and II projects.

The PCR reports a 4 rating for this category. However, the project was ranked a Category III and this

criterion would not normally apply. Even if it does, this Review recommends a satisfactory (3) rating.

The Environmental aspects with ESMP may have been well covered, but social sustainability is not

guaranteed. The participation, ownership, absence of sustainability plans to ensure continuous social

engagement is not guaranteed.

4. PERFORMANCE OF STAKEHOLDERS

a. Bank performance: (Preparation/approval, ensure of Quality at Entry (QAE) : quality of the supervision, completion) : Provide observations on the

objectivity of the PCR ratings and feedback provided by the Borrower, and if necessary, re-assess the Bank’s performance

throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to 7 criteria

defined in the PCR Guidance Note.

The PCR is critical of Bank performance because (a) a fraud investigation took too long to complete

and (b) there were frequent changes in task manager. Both factors negatively affected project

performance and the 2 rating for this section seems justified.

b. Borrower performance: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the Borrower’s performance throughout

the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to questions defined in

the PCR Guidance Note.

Project management seems to have been very poor with political interference regarding the project

manager. The 1 rating seems justified. The list of other failings include, Lack of counterpart funding,

M&E system developed but not fully utilised. Outstanding audit reports among others.

c. Performance of other stakeholders: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the other shareholders’ performance

throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to relevant

questions specific to each stakeholder (co-financiers, NGO, contractors and service providers).

The explanations put forward by the PCR are consistent with a 2 rating

5. SUMMARY OF OVERALL PROJECT PERFORMANCE

a. Overall assessment: Provide a summary of the project/programme’s overall performance based on the PCR 4 key components (Relevance,

Effectiveness, Efficiency and Sustainability). Any difference with the PCR and the reasons that have resulted in them should

be mentioned. For cases with insufficient evidence (from the PCR and other documents) available, the evaluator should

assign a partly satisfactory rating (to be revised) until a PPAR is complete.

From the PCR it seems clear that the project’s performance was poor. The PCR has an average score

of 2.5 but this should be lower because (a) the PDO should be 2 not 3 and (b) environmental

sustainability should be removed. Making these two changes would reduce the average score to 2.2

which seems to be a clearer reflection of the project quality. The Review recommends an

unsatisfactory (2) rating

b. Design, implementation and utilization of the M&E (appreciation of the evaluator): Provide an assessment of planned and actual cost of the design, implementation and utilization of the M&E system. Design : To

which extent the project M&E system was explicit, adequate and realistic to generate and analyse relevant data ;

Implementation : To which extent relevant data was collected – Elements of M&E implementation and effectiveness in the

PCR ; Utilization : degree of utilization of data generated for decision-making and resource allocation – elements of M&E

utilization in the PCR.

No information on which to judge

6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS

a. Lessons learned: Provide a brief description of any agreement/disagreement with all or part of the lessons learned from the PCR after analysis of the

project performance with regards to each of the key components of the evaluation (Relevance, Effectiveness, Efficiency, and

Sustainability). List the PCR main new and/or reformulated pertinent (and generic) lessons learned for each of these components here. It

is recommended that no more than five lessons learned are discussed. Key questions and targeted audience must also be specified for

each lesson learned.

The PCR has the following lessons. The Evaluation’s opinion of each lesson is given.

1. Project/political leadership NOT VALIDATED – MOVED TO RECOMMENDATIONS

2. Effective Stakeholder collaboration VAIDATED

3. Transition, especially from one political/technical leadership to another NOT VALIDATED –

REPEAT OF 1

4. Capacity enhancement of Project teams in Bank’s procurement and administrative processes –

VALIDATED

5. Sustainability plan/exiting strategy NOT VALIDATED – MOVED TO RECOMMENDATIONS

6. Handing overs -NOT VALIDATED – TOO PROJECT SPECIFIC

7. Commitment to long-term scholarships - NOT VALIDATED – COMBINED WITH 5 AND

MOVED TO RECOMMENDATIONS

8. Ownership and Capacity building – VALIDATED

9. Communication with stakeholders – NOT VALIDATED, TOO CLOSE TO 2

10. Quality at Entry – NOT VALIDATED, IT IS A RECOMMENDATION BUT IS TOO PROJECT

SPECIFIC IN THIS CASE AND SHOULD ALREADY BE PART OF GOOD PRACTICE

OTHERWISE

11. Investigations into fraud allegations – NOT VALIDATED, REFORMULATED AS A

RECOMMENDATION

12. Government Counterpart Funds – NOT VALIDATED, REFORMULATED AS A

RECOMMENDATION

b. Recommendations: Provide a brief description of any agreement/ disagreement with all or part of the recommendations from the PCR. List the PCR main

new and/or reformulated recommendations (requiring more actions by the Borrower and/or the Bank) here.

1. Scholarships – NOT VALIDATED, TOO PROJECT SPECIFIC

2. Women access to MFIs - NOT VALIDATED, TOO PROJECT SPECIFIC

3. Follow-up - NOT VALIDATED, TOO PROJECT SPECIFIC

The Evaluation suggests instead the following 3 generic recommendations that emerge from the PCR:

1. The role of political leaders, Executing Ministry’s administration, Steering committee and

Political actors in Project Management should be clearly defined in order to ensure effective

Project management. It is necessary to orient new leadership on the Project including the

policies and procedures.

2. Where a project commits to activities such as scholarships that go beyond its closure date, a

clear exit strategy must be put in place at appraisal or developed during project implementation

so that after project closure, beneficiaries are not adversely affected.

3. Where a Project is subject to serious fraud allegations, the Bank’s investigation service

(IACD) should attempt to ensure that project activities can continue with minimum disruption

whilst investigations are conducted.

7. COMMENTS ON PCR QUALITY AND TIMELINESS The overall PCR rating is based on all or part of the criteria presented in the annexe and other: The quality of the PCR is rated

as highly satisfactory (4), satisfactory (3), unsatisfactory (2), and highly unsatisfactory (1). Le timeliness of the PCR is rated as

on time (4) or late (1). The participation of the Borrower, co-financier, and the bank’s external office(s) are rated as follows:

Very Good (4), Good (3), Fair (2), Poor (1).

The PCR team seems to have been objective and to have based its scores on the evidence but the

Evaluation has no information on how complete this evidence is. The score for the Project

Development Objective is incorrectly calculated and a score for environmental sustainability is

included when not required given project category. Correcting for these two things lowers the average

score for the Project from 2.5 to 2.2 which seems to be a more accurate representation of the Project’s

performance, given what is presented in the PCR.

No accessible evidence was provided either from the PCR including annexure or other data making it

impossible to undertake an independent evaluation. The PCR is clearly written and presented but in

the absence of any other evidence one cannot tell how complete it is.

The lessons and recommendations tend to be mixed up with many of the former in fact belonging to

the latter category. They are often far too project-specific to be very useful for informing future

projects

Given the above, the PCR is rated unsatisfactory (2.33).

It is not entirely clear but the PCR seems to have been completed one year after the end of the project.

8. SUMMARY OF THE EVALUATION This is a summary of both the PCR and IDEV ratings with justification for deviations/comments. Appropriate section of the

PCR Evaluation should be indicated in the last column in order to avoid detailed comments. The evaluator must provide a

reasonable explanation for each criterion the PCR rating is not validated by IDEV. Consequently, the overall rating of the

project could be “equally satisfactory”.

Criteria PCR PCREN Reason for disagreement/ Comments

RELEVANCE

Relevance of project development objective 4 4

Relevance of project design 3 3

EFFECTIVENESS

Development objective (DO) 3 2 The PCR calculation is incorrect given the

ratings for outcomes and outputs

EFFICIENCY

Timeliness 2 2

Resource use efficiency 2 1 The project could only use 43% of financial

resources in 7 years of implementing a 4

year program.

With 56.5% of undisbursed funds, special

account unjustified at PCR, Fraud

investigations

Cost-benefit analysis na na

Implementation progress (IP) 2 2

SUSTAINABILITY

Financial sustainability 1 1

Institutional sustainability and strengthening of

capacities

2 2

Environmental and social sustainability 4 3 Category should probably not be included.

Social factors not covered

OVERALL PROJECT COMPLETION

RATING

2

Bank performance: 2 2

Borrower performance: 1 1

Performance of other shareholders: 2 2

Overall PCR quality:

9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR

PERFORMANCE EVALUTION REPORT, IMPACT EVALUTION,

COUNTRY/SECTOR REVIEWS OR THEMATIC EVALUATION STUDIES:

- Project is part of a series and suitable for cluster evaluation

- Project is a success story

- High priority for impact evaluation

- Performance evaluation is required to sector/country review

- High priority for thematic or special evaluation studies (Country)

- PPER is required because of incomplete validation rating

Major areas of focus for future evaluation work:

a) Performance evaluation is required for sector/ country review

b) Cluster evaluation (institutional support)

c) Sector evaluation (budgetary support or public finance management reforms)

Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to

facilitate a), b) and/or c).

Division Manager clearance Director signing off

Data source for validation:

Task Manager/ Responsible bank staff interviewed/contacted (in person, by telephone or

email)

Documents/ Database reports

Attachment:

PCR evaluation note validation sheet of performance ratings

List of references

Appendice 1

PROJECT COMPLETION REPORT EVALUATION NOTE Validation of PCR performance ratings

PCR rating scale:

Score Description 4 Very Good – Fully achieved with no shortcomings 3 Good – Mostly achieved despite a few shortcomings 2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced 1 Poor – very limited achievement with extensive shortcomings

UTS Unable to score/rate NA Non Applicable

Criteria Sub-criteria PCR

work

score

IDEV

review Reasons for deviation/comments

RELEVANCE Relevance of the project

development objective

(DO) during

implementation

4 4

Relevance of project

design (from approval to

completion)

3 3

OVERALL RELEVANCE SCORE

EFFECTIVENESS* Effectiveness in delivering outcomes

Outcome1

Outcome2

Effectiveness in delivering output

Output1

Output2

Development objective (DO)

Development objective

rating

3 2 The PCR calculation is incorrect given the ratings for

outcomes and outputs

Beneficiaries

Beneficiary1

Criteria Sub-criteria PCR

work

score

IDEV

review Reasons for deviation/comments

Beneficiary2

Unanticipated outcomes (positive or negative not considered in the project logical

framework) and their level of impact on the project (high, moderate, low)

Institutional

development

Gender

Environment & climate

change

Poverty reduction

Private sector

development

Regional integration

Other (specify)

EFFECTIVENESS OVERALL SCORE

EFFICIENCY Timeliness (based on the

initial closing date) 2 2

Resource used

efficiency

2 1 The project could only use 43% of financial

resources in 7 years of implementing a 4 year

program.

With 56.5% of undisbursed funds, special account

unjustified at PCR, Fraud investigations

Cost-benefit analysis na Na

Implementation progress

(from the IPR) 2 2

Other (specify)

OVERALL EFFICIENCY SCORE

SUSTAINABILITY Financial sustainability

1 1

Institutional

sustainability and

strengthening of

capacities

2 2

Ownership and

sustainability of

partnerships

2 2

Environmental and

social sustainability 4 3 Category should probably not be included. Social

factors not covered

*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest IPR of the

project (see Guidance Note on the IPR).

The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets, and the

overall development objective of the project (DO) is obtained by combining the ratings obtained for outputs and outcomes

following the method defined in the IPR Guidance Note. The following method is applied: Highly satisfactory (4),

Criteria Sub-criteria PCR

work

score

IDEV

review Reasons for deviation/comments

Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).

Criteria Sub-criteria PCR

Work

score

IDEV

review Reasons for deviation/comments

BANK

PERFORMANCE Proactive identification and

resolution of problems at different

stage of the project cycle

Use of previous lessons learned

from previous operations during

design and implementation

Promotion of stakeholder

participation to strengthen

ownership

Enforcement of safeguard and

fiduciary requirements

Design and implementation of

Monitoring & Evaluation system

Quality of Bank supervision (mix

of skills in supervisory teams, etc)

Timeliness of responses to

requests

OVERALL BANK PERFORMANCE SCORE 2 2

BORROWER

PERFORMANCE

Quality of preparation and

implementation

Compliance with covenants,

agreements and safeguards

Provision of timely counterpart

funding

Responsiveness to supervision

recommendations

Measures taken to establish basis

for project sustainability

Timeliness of preparing requests

OVERALL BORROWER PERFORMANCE SCORE 1 1

PERFORMANCE

OF OTHER

STAKEHOLDERS

Timeliness of disbursements by

co-financiers

Functioning of collaborative

agreements

Quality of policy dialogue with

co-financiers (for PBOs only)

Quality of work by service

providers

Responsiveness to client demands

OVERALL PERFORMANCE OF OTHER

STAKEHOLDERS

2 2

The overall rating is given: Very Good, Good, Fair and Poor.

(i) Very Good (HS) : 4

(ii) Good (H) : 3

(iii) Fair (US) : 2

(iv) Poor (HUS): 1

DESIGN, IMPLEMENTAION AND UTILIZATION OF MONITIRING AND

EVALUATION (M&E)

Criteria Sub-criteria IDEV

Score Comments

M&E DESIGN M&E system is in place, clear,

appropriate and realistic UTS

Monitoring indicators and

monitoring plan were duly

approved

UTS

Existence of disaggregated gender

indicator UTS

Baseline data were available or

collected during the design UTS

Other, specify

OVERALL M&E DESIGN SCORE

M&E

IMPLEMENTA-

TION

The M&E function is adequately

equipped and staffed UTS

OVERALL M&E IMPLEMENTATION SCORE

M&E

UTILIZATION

The borrower used the tracking

information for decision UTS

OVERALL M&E UTILIZATION SCORE

OVERALL M&E PERFORMANCE SCORE UTS

PCR QUALITY EVALUATION

Criteria PCR-EVN

(1-4) Comments

QUALITY OF PCR

1. Extent of quality and completeness of the PCR

evidence and analysis to substantiate the ratings of

the various sections

2 The PCR team seems to have based its scores on

the evidence but the Evaluation has no information

on how complete this is

2. Extent of objectivity of PCR assessment score 3 Seems objective

3. Extent of internal consistency of PCR assessment

ratings; inaccuracies; inconsistencies; (in various

sections; between text and ratings; consistency of

overall rating with individual component ratings)

3 Seems consistent except for calculation of PDO

and question of whether to score environmental

sustainability

4. Extent of identification and assessment of key

factors (internal and exogenous) and unintended

effects (positive or negative) affecting design and

implementation

- No information

5. Adequacy of treatment of safeguards, fiduciary

issues, and alignment and harmonization

- No information

6. Extent of soundness of data generating and

analysis process (including rates of returns) in

support of PCR assessment

- N/A

7. Overall adequacy of the accessible evidence (from

PCR including annexure and other data provided)

1 No documents provided

8. Extent to which lessons learned (and

recommendations) are clear and based on the PCR

assessment (evidence & analysis)

2 The Review recommends 2 (unsatisfactory). The

PCR, used findings, lesson, recommendations

interchangeably.

9. Extent of overall clarity and completeness of the

PCR

3 The Review recommends 3 (satisfactory); the PCR

was complete, the numerous outputs and outcomes

tables were fully completed. All the required

narratives were given and also most if not all the

ratings completed with analysis.

Other (specify)

PCR QUALITY SCORE 2.33

PCR compliance with guidelines (PCR/OM ; IDEV)

1. PCR Timeliness (On time = 4; Late= 1) 1 Seems to have been done one year after the

completion date

2. Extent of participation of borrower, Co-financiers

& field offices in PCR preparation

UTS

3. Other aspect(s) (specify)

PCR COMPLIANCE SCORE

*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)

References