LAO PDR RETROSPECTIVE

A Review of the role of Capacity Enhancement in Recent Bank Projects

in Lao PDR

1

TABLE OF CONTENTS

Executive Summary……………………………………..3

Project Review………………………………………….4

Quantitative Observations…………………………..……8

APPENDIX 1 - Analysis of Recent Projects…………………..13

APPENDIX 2 – OED Project Analysis………………………19

APPENDIX 3 - TTL and TM Interviews…………………….21

2

Lao PDR: Project Review

Summary

This report examines the role and impact of capacity enhancement on recent Bank projects implemented in Lao PDR. A sample of ten projects served as the basis for the analysis, which was carried out by reviewing various project documents and interviewing the Project TTLs and TMs. This report divides the projects into three phases: 1. design, 2. implementation, and 3. follow-up. Each phase is explored separately in order to clearly identify the presence and/or effectiveness of CE at every stage of the project’s life. understanding was reached to define CE as the increase in the ability of organizations and individuals to perform functions effectively, efficiently, and sustainably. The project background, analyses, and interviews are provided in the attachments at the end of the report. Key findings are outlined below. The factors highlighted are not exclusive to only one project phase, however they are listed below the phase in which they were viewed to have the most impact.

From the planning and design stages, projects require a clear assessment of how CE framework at various levels can be integrated into the design and how to help clients recognize the capacity gap and invest the resources required to help bridge this gap. Key lessons include the importance of setting realistic targets, timelines and measurable indicators of success as determined largely by the clients themselves, the need to coordinate projects with all relevant key players from the start - including the donor community, ministries, and agencies who will be involved in capacity building activities. Analysis of the implementation stage revealed the challenges of defining the right incentive formula for consultant-client training, the vital role of champions in obtaining client cooperation, the effectiveness of learning-by-doing versus lecture-style training, and the importance of effective monitoring and evaluation of CE activities to measure performance gains.

The final phase focuses on sustainability. To accomplish this, the project requires a stronger buy-in and commitment from the client side. Strengthening the capacity of local institutions to backstop the CE process might be the most effective way to deliver continued support to clients and to help them assess and address their capacity enhancement needs.

3

Lao PDR: Project Review

________________________________________________________________________

Introduction:

The objectives of this assessment, broadly stated, are to review the factors which have affected, either positively or otherwise, the capacity enhancement components of recent Bank projects in Lao PDR.

Approach

The idea was to identify at what points in the project cycle supply factors, such as design, and/or demand factors, such as institutions and incentives, might affect Capacity Enhancement/Development (CE/D) absorption and outcomes. Project management experience was notionally segmented into three phases:

Design: includes reference to consideration of CE/D in components such as project planning, feasibility studies, and ESWImplementation: includes components like modalities, champions, priority by client and TTL, relationships, incentives, pedagogy, vehicles for monitoring, etc.Sustainability & Ownership: looks at components like feedback mechanisms and sustained involvement.

FindingsBased on several TM and TTL interviews as well as analyses of selected projects, the findings below are separated into the three project phases:

Design

1. Objective Setting: While Training and TA activities can be found in almost every project, capacity building as an integrated component of the project is usually not articulated. Although project objectives were clearly spelled out (for example, improving service delivery, improving quality of basic education, increasing efficiency of land titling, improving the livelihoods of local communities through intensified agricultural practices, creating a road maintenance system and culture, etc.), specific and realistic CE objectives for achieving these goals were less evident. CE performance indicators were often not given adequate importance or effectively communicated to clients as critical to the successful achievement of operational outcomes. CE remained a priority when it was clearly integrated into the project objectives, however difficulties were encountered in creating incentives and in motivating stakeholders at various levels to invest the resources required to help bridge the capacity gap. Lessons learned from the Forest Management and Conservation Project emphasize the importance of illustrating to Government officials and other decision makers that local community-focused CE will not only

4

benefit these communities, but that these benefits will also extend to the higher levels. For example, building capacity for village-based forest management in rural communities will result in a more efficient collection of royalties and taxes within the community, thereby enhancing regional and central budgets. To better illustrate this kind of positive impact, TTLs would be much more effective if they had access to tools for assessing capacity needs and for helping the client recognize existing capacity gaps and the importance of investing their time and resources to achieve more sustainable outcomes.

2. Performance Indicators: Many of the projects lacked adequate definitions of how effectiveness or progress of CE may be assessed. Performance indicators for CE were not always clearly outlined (if mentioned at all) in the reviewed documentation. Those which were outlined lacked readily measurable and relevant yardsticks. A few of the projects that did outline measurable indicators provided timelines that were over-ambitious. The main lessons from the selected projects accentuate the importance of setting realistic targets and timelines; identifying resources required to achieve the project goals; integrating institutional capacity analysis into the project preparation stage; and encouraging effective communication with clients to ensure that performance targets and timelines remain clear and can be met. Of the lessons learned in the Highway Improvement project, the most notable was the need for sustained involvement throughout the life of the project, which in turn allows for better monitoring of performance. The key to attaining this involvement is convincing the stakeholders that maintenance is as essential as construction and is best achieved through capacity enhancement. The incremental transport sector approach, which was based on a series of rehabilitation projects with increased focus on important capacity building components proved to be effective in Lao PDR. Although this approach was not directly identified as a CE objective, the task manager played an instrumental role in encouraging CE.

3. Coordination and Management: Project TTLs heavily emphasized the importance of maintaining harmony within the country through inclusion of the donor community in the Bank’s activities. Capacity building in the country will require considerable coordination among various donors who will be working with the local communities to carry out CE activities. The Bank, along with the donor community, individual ministries and agencies needs to collaborate in designing and implementing targeted capacity enhancement initiatives that are consistent with national priorities.

4. Targeting: Both the project documents and the Task Managers seemed very clear on what performance the CE initiatives were supposed to improve, and who the beneficiaries would be. Managers acknowledged that stakeholder engagement was a key driver of success of CE initiatives. It follows that stakeholders should take the lead in the planning, management and monitoring of CE. However, an important component often missing from the project design was identifying strategies for effectively involving these stakeholders and for ensuring a high degree of ownership at various levels through, for instance, defining accountability or setting up incentive and reward structures in order to better meet project goals and enhance project sustainability.

Implementation

5

1. Defining Consultant/Counterpart relationship: The most challenging element in CE is sustainability. The results-focused nature of traditional consultant/counterpart relationship tends to detract from achievement of sustainable results. In most cases, consultants generally focused on achieving a certain monthly output often at the expense of the “training and knowledge transfer” aspects of their mandate. Past projects pointed to the importance of setting clear objectives, measuring and monitoring the performance of the client at various stages of the training process. While finding the right incentive formula for consultant/counterpart relationships has been a significant challenge, some TTLs have already begun to change the consultant/client remuneration in the second phases of their projects. For instance, in the first phase of the Land Titling Project, financing of technical assistants was based on input per month, which then subsequently changed to output per month. However, the latter was still ineffective in achieving the desired knowledge transfer as TAs focused more on their output than on client training. Therefore, in an effort to achieve more effective and sustainable results in the second phase of the project, TAs will be compensated based on a combination of 30% output, 30% input, 30% for successful client training and results, and 10% on service standard results. Focusing on more innovative methods to remunerate TAs and consultants will significantly impact the transfer of knowledge and consequently the sustainability of the project.

2. Training: Present training structure generally puts emphasis on lecture-style teaching. More practical, hands-on, learning-by-doing approaches have been found by all Task Managers to be more effective in transferring desired knowledge/skills and in enhancing the prospects of achieving sustainability. Specifically, study tours and on-the-job training were the two most powerful. Of course, this does not suggest a complete replacement of lecture-style knowledge sharing especially since study tours and on-the-job training can be costly and time consuming. A wide range of alternate pedagogies are now available and worth reviewing in conjunction with local partners. Beneficiaries prefer to see problem solving in action, and to learn from interacting with same-level officers from similar but more effective/efficient institutions. This seems considerably more effective than observing a consultant or other more knowledgeable/skilled person perform tasks.

3. Champions: Client cooperation was almost always acquired through champions, who were thus, very instrumental to success of CE. Champions were identified by their progressiveness; they also carried considerable clout within the recipient organizations/institutions. They were usually around the upper middle management cadre. Three task managers report that this cadre would also be the best audience to target with increased CE investments. The involvement of local institutions was rarely if ever mentioned. On the Bank end, facilitating a shift in focus from outcomes to strategy and process, and an across the board enhanced appreciation for, and confidence in, the validity of CE remains the challenge.

4. Monitoring: Effective monitoring for performance gains and sustainability was not evident in any of the documentation and TTLs acknowledged the negative impact of their

6

absence. Where CE objectives were articulated in project design, the strategies for achieving and monitoring them were not.

Sustainability and Ownership

Selected efforts made to ensure client ownership through jointly designed approaches to CE and the measurement of its achievement were met with relatively more success by clients. The traditional results-focused, consultant-managed reward system tended to undermine ownership and accountability and as a result, the sustainability of the project. Sustainability also requires the cooperation and commitment of local training or research institutions, such as the National Organization for Studies of Politics and Administration (NOSPA) with which to plan, test and demonstrate CE framework under preparation and also create long-term partnerships that will facilitate the country’s capacity enhancement activities.

Observations

Bank Evaluation: Most of the OED and ICR Documents generated for these projects have overlooked the risks implied by capacity shortfalls. A more systematic ex-ante review of CE issues like implementation procedures, specific outcomes and performance indicators would have supported stronger outcomes

Existing Capacity: Proper and thorough analysis of existing capacity needs is an important foundation for the overall design of projects and programs

Effective Capacity building efforts appear to enhance feeling of confidence and ownership by clients and as such may help to build constituencies for more fundamental change.

7

Quantitative Observations

This section offers a brief analysis of the correlation between the expenditure on Capacity Enhancement and the CE outcome of each project. This analysis does not attempt to make assumptions regarding the sustainable outcome of a project based solely on the level of a project’s CE expenditure, however, observations of general trends or patterns on the correlation of CE activity expenditures with either positive or negative CE project outcomes can be made. Finally, OED Reviews were available for only three out of the five projects. The analysis for the final two projects was arrived at through interviews previously conducted with the task managers of the respective projects.

1. The classification of CE activities varied substantially for each project. While activities that would fall under CE were evident cost components of each project, CE was not identified as a separate cost category. CE activities were listed differently in the five projects below as: HR Development and Technical Assistance, Consulting Services and Training, Institution Building, and Institutional Strengthening and Monitoring and Evaluation. Given the differences in classification, a consistent project-to-project comparison was not possible.

2. In two of the three projects where spending on CE was below 20% of the total project costs, the OED reviews indicated that sustainability was “likely” and institutional development was “substantial”, whereas in the project where CE accounted for 36% of total project costs (of which only 4% was HR Development and 32% was TA), the OED review indicated that the project’s sustainability was unlikely and institutional development, negligible.

3. All projects divided activity costs into foreign and local costs. Foreign costs usually outweighed the local costs in the capacity building component of a project. This discrepancy was highly notable in all except one of the projects were local costs were slightly above those of the foreign costs. The number of local versus foreign consultants was not always specified and therefore difficult to break down to cost per foreign unit/consultant and cost per local unit/consultant.

4. For projects with a limited Technical Assistance component, the total expenditure on CE was generally less than twenty percent of the total project costs. However, in the case where CE costs were substantially greater (36%, 30% and 50%), Technical Assistance was usually the largest component or was combined with other activities, such as Monitoring and Evaluation mechanisms, which together made up the largest cost components.

8

1. Forest Management and Conservation Project

Technical Assistance was notably the largest CE component of this project. Of the 32% allocated for TA, foreign costs made up 80%. HR Development made up 4% of total project costs. Villagers and ministry staff involved in village forestry were successfully trained, and the village forestry model was successfully piloted, however the substantially lower cost allocation for HR development may have made it difficult for the project to achieve one of its objectives of establishing a national training program. This is the only project and only activity were local costs surpassed foreign costs, however there is no clear indication of the effect this spending approach had on the project outcome.

Forest Management and Conservation ProjectP004196

US$(M)Local

US$(M)Foreign

US$ ‘000Total

CE as % of total

cost

CE Outcome ICR & OED Review

HR Development 410(52%)

379(48%) 789 4%

The project developed, tested and implemented a very successful, village-based sustainable forest management model and demonstrated that it is replicable at a reasonable cost.

The emphasis on a step-by-step, modular training program, reinforced by immediate field activities proved highly effective.

However, HR development was only partially achieved as the original objective of establishing a national training program was not accomplished.

In the absence of tangible programs and in-depth policy analysis, the optimism about government's commitment to reform was misplaced

Lesson: In trying to integrate development and conservation goals, less attention should be paid to conceptual models and more to winning the confidence of the community and local government.

Technical Assistance

1,285(20%)

5,139 (80%) 6,424 32% Institutional

Development:Negligible

Sustainability:UnlikelyTotal Project

Costs 5,482 14,872 20,354 36%

All costs are approximate and include a 16% mark-up for price and physical contingencies

SAR : Forest Management and Conservation Project http://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1999/04/28/000009265_3961002055124/Rendered/PDF/multi0page.pdf

OED Review: http://wbln1023.worldbank.org/oed/OEDDocLib.nsf/View+to+Link+WebPages/8525682E0068603785256A7600471675?OpenDocument

9

2. Southern Provinces Rural Electrification Project

Spending on CE was less than 20%of this project’s total costs. TA and training fell under “Consulting Services”, which together made up less than 15% of total costs. Foreign costs accounted for 100% of the CE activity costs and the project's contribution to institutional development is rated as substantial and its sustainability is rated as likely. The project substantially achieved all of its objectives.

Southern Provinces Rural Electrification ProjectP044973

Institutional Building

US$(M)Local

US$(M)Foreign

US$(M)Total

CE as % of total cost

CE Outcome

Consulting Service to EDL

- 2.62(100%) 2.62 7%

The project developed an Institutional Development program that included a study on the organization, management and finance of EDL; equipment for communications, metering and loss reduction; and billing and office equipment.

Lesson: Projects can succeed even if implemented by agencies with very limited technical capacity, provided that their own capacity is adequately supplemented by high quality technical assistance and continuous training.

Consulting Service to MIH - 2.28

(100%) 2.28 5.8%

Equipment for EDL 0.91

(100%) 0.91 2.3%

Total Project Costs

7.6 31.7 39.3 15.1% Institutional Development:

Substantial

Sustainability: Likely

All costs are approximate and include a 14% mark-up for price and physical contingencies Consulting Services includes hands-on assistance and training, procurement, construction supervision and monitoring and

technical assistance.

PAD: Southern Provinces Rural Electrification Projecthttp://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1998/02/24/000009265_3980429111603/Rendered/PDF/multi_page.pdf

OED Review:http://wbln1023.worldbank.org/oed/OEDDocLib.nsf/View+to+Link+WebPages/12816C03AC6D6F3E852567F5005DAB26?OpenDocument

3. Highway Improvement Project (2)

This project achieved a sustainability rating of ‘likely’ and a high likelihood of institutional development. Similar to the Southern Provinces Rural Electrification Project, CE spending accounted for less than 20 percent. Foreign costs are still substantially higher than local, especially in Consulting activities. The project successfully achieved its principal objectives of reducing vehicle operating costs, providing access to social services, and facilitating export/import and transit traffic, and mostly achieved the institutional strengthening component. Furthermore, performance of TA in project coordination and in road maintenance management was satisfactory while TA effectiveness in transport planning was partly successful. The financially minor training component, comprising fellowships and short-term courses, was partly successful.

10

Highway Improvement Project (2)P004201

US$ ‘000Local

US$ ‘000Foreign

US$ ‘000Total

CE as % of total cost

CE Outcome

Consulting Services and Training

360(29%)

880(71%) 1,240 3.7%

The capacity of Transport Planning Unit (TPU) has been significantly strengthened. TPU provides data, advice and training to MCTPC staff to develop its transport plan.

TPU also provides training and assistance to the provincial departments and conducts annual traffic counts

The capacity of local contracting industry has also been enhanced.

TA to Enterprises 329(35%)

611(65%) 940 2.8%

TA to RMMD 850(35%)

1580(65%) 2,430 7.2%

Construction Supervision

230(35%)

428(65%) 658 2%

Total Project Costs 7,447 26,235 33,682 15.7%

Institutional Development:

Substantial

Sustainability: Likely

All costs are approximate and include an 18% mark up for contingencies Consulting Services and Training includes: Transport planning Unit, coordinating consultant, Accounting/auditing, fellowships, and short courses, seminars and training In this case, TA fell under separate headings of Savannakhet-Pakse Road and National Road Maintenance and not under Consulting Services and Training.

Sources: SAR: Highway Improvement Projecthttp://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1994/03/16/000009265_3961005142746/Rendered/PDF/multi0page.pdf

OED Reviewhttp://wbln1023.worldbank.org/oed/oeddoclib.nsf/View+to+Link+WebPages/BB213C3DA68FCC0A852567F5005D5CFF?OpenDocument

4. Land Titling Project

CE accounted for approximately 30% of this project’s total costs, with 15.5% allocated to TA. Foreign costs in this project far exceeded the local costs. While heavy technical assistance was a contributing factor for the project’s progress, TA was not always very effective as technical experts, though experts in their field, were not always skillful in training and disseminating their knowledge effectively. Most effective were the study tours, which gave the clients an opportunity to see what other countries are doing. Conferences often worked, however the decision makers at the senior level rarely attend these events, therefore lessening their effectiveness. Also effective was one-on-one leadership training at high and influential levels; in Lao PDR the highest accessible levels are the vice ministers. An impediment to sustainability was the civil service salary; once the civil servants are trained and better educated they are recruited by the private sector, thereby draining the government of educated staff. Without addressing this public sector reform issue, it will continue to be a serious constraint that limits capacity enhancement.

Lao Land Titling Project P004208

Institutional Building Local US$ (M)

ForeignUS$ (M)

TotalUS$ (M)

CE as % of total cost

CE OutcomeForeign Local

93% 7%

Project Administrator - 2.3(100%)

2.3 8.1%

Training N/A

11

Local 0.15(50%)

0.15(50%)

.3 1%

Overseas - 1.53(100%)

1.53 5.4%

Technical Assistance 0.44(11%)

3.87(89%)

4.3 15.5%

Total Project Costs 28.38 30%

All costs are approximate and include a 15% markup for price contingency only

Source: SAR: Land Titling Projecthttp://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1996/02/21/000009265_3961019135223/Rendered/PDF/multi0page.pdf

5. District Upland Development and Conservation Project

Although CE was not an articulated objective of this project, it was reflected in the “Institutional Strengthening and Monitoring and Evaluation Component”, which made up 50% of total project costs, and helped achieve the above objective. Institutional Strengthening and Monitoring and Evaluation Component helped improve the management and technical skills of staff to implement activities at the provincial, district, and village levels. It also tested whether the current administrative implementation mechanism and decentralization process were enhancing delivery of services to beneficiaries and the extent to which village organizations have the capacity to absorb these services. The training component was not as useful as it should have been. In most cases throughout this project the government may not have sent the right people to these sessions, which significantly reduced the effectiveness of the training.

District Upland Development and Conservation ProjectP059305

US$(M)Local

US$(M)Foreign

US$ (M)Total

CE as % of total cost

CE Outcome

Institutional Strengthening, Monitoring & Evaluation*

0.24(22%)

0.87(78%) 1.11 50%

N/ATotal Project Costs 0.73 1.52 2.25 50%

All costs are approximate and include a 18% markup for physical and price contingencies* Institutional Strengthening, Monitoring & Evaluation includes upgrading of management and technical skills, technical assistance to enhance overall local government and institutions' capacity to implement, monitor and evaluate the project activities, travel and subsistence for the TAs.and participatory M&E to provide feedback on implementation and lessons learned.

PAD: District Upland Development Projecthttp://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/1999/09/17/000094946_9903250555387/Rendered/PDF/multi_page.pdf

12

APPENDIX 1: Capacity Enhancement for Lao PDR- Analysis of Recent Projects

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedLAO PDR Education

Sector Development

Project P004203

Robert Varley

Major Sector:Education

Document:http://

imagebank.worldbank.org/

servlet/WDS_IBank_S

ervlet?pcont=details&eid=000094946_01072104010

091

This project sought to address the immediate needs for quality improvement in basic education, and to establish the policy and institutional framework for longer-term sector development through:

(1) developing a new curricula(2) initiating a sustainable construction program for primary and lower secondary schools(3) establishing systems of data collection and analysis to support sector mgmt and strategic planning(4) strengthening institutional capacities in educational administration and management at the central and local levels

CE Objectives School Infrastructure: Develop Institutional Arrangements that preserve the vital role of local communities in school provision, and to provide TA to communities continuing to build provisional and semi-permanent structures. Educational Admin. and Mgmt: To strengthen institutional capacities in educational admin and management while assisting the sector to respond to the overall administration and financial reforms of the government. Curriculum/Pedagogical Support: Introduce teachers to the new curriculum and materials through orientation program. Outcome The overall performance of the Project including financial management is considered to have improved by the Ministry of Education. The MofE as a core of HR development has put great effort in the development of capacity building through a wide range sub-sectors. Editorial capacity (editing and publishing

of new materials) was enhanced through staff training.

Cascading training system provided to teachers for new curriculum was developed and implemented, however it was not cost-effective since basic academic knowledge of teachers is still poor and learning environment not appropriate (overcrowded classes, poor infrastructure).

Capacity of PUCDA strengthened through decentralization and staff training.

The Ministry of Education, teachers, students (Incl. minorities), contractors (for building school infrastructure)

Developing and distributing teaching materials on schedule,

Number of pedagogical advisors recruited and trained,

Number of school clusters successfully established under project pilot, progress of studies and pilot activities in Lao language and language of instruction.

Indicators not included (but should have been):The benefits and impact of HRD on the institutional capacity building on central, provincial and district levels.

The need to include institutional capacity analysis during project preparation when attempting project implementation arrangements for the first time

The need to build local capacity for publishing, printing and distributing pedagogic and other reading materials.

13

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedLAO PDR

Forest Management

and Conservation

Project

ProjectP004196

Major Sector:Agriculture, fishing, and

forestry

Task ManagerSusan S. Shen

Document:http://

imagebank.worldbank.org/

servlet/WDS_IBank_S

ervlet?pcont=details&eid=000094946_01072404025

99

The project attempted to resolve the fundamental problems of the forestry sector initially aiming at assisting the Government of Laos to implement a new resource management system. This broad and ambitious objective was later narrowed down to developing a sustainable forest management and bio-diversity conservation system and a community-based forest management systems for economic and conservation purposes through:

(1) piloting a village-based forest management system;(2) developing appropriate institutional capacity and operational procedures for the expansion of the systems when they prove effective;(3) exploring the feasibility of integrating socio-economic development of the forest-based communities with

CE Objectives Village Forestry DevelopmentIntensive training of both government forestry staff and villagers, establishment of a wide-scale national training programThe Village Forestry ModelDevelop a replicable community-based management systems for economic and conservation purposesOutcome Human Resources DevelopmentA Village Forestry Handbook and Village Forestry Training Manual that could be used for training elsewhere was prepared. Although the Village Forestry training program was extremely effective in elevating the professional capabilities of the staff and skills of villagers, the initial intent to establish a national training program was not achieved. Technical AssistanceThe Village Forestry model was implemented with full participation of the villages. The model has been tested and replicated to a second province using primarily local staff as trainers; Both the village organizations and the department staff gained experience and confidence to sustain and expand the system. A comprehensive training needs assessment provided a firm foundation for training carried out for the National Biodiversity Conservation Area sub-program staff. The

The Lao Government; the Forest Department; local communities (including villagers, village organizations, forestry staff)

Number of villages where a sustainable forest management model was applied as a pilot;

Development of a national level training program;

Person-days of training;

Overall, the project lacked a clear logical framework and performance indicators

Need to set realistic targets, timeframe and required resources;

Adequate analysis of Government policy framework during project design;

Village-based forest management benefits not only rural communities, but also the Government, through more efficient collection of royalties and taxes, improved forest protection and sustainable management, and enhanced rural development;

Much greater awareness of the needs of people in addition to the need for biodiversity conservation is required.

14

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons Learnedthe bio-diversity conservation programs.

Forest Department in two provinces had established offices with well-trained staff able to sustain and expand the village-based forest management program.

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedLAO PDR Provincial

Grid Integration

Project

Project P004197 Enrique

Crousillat

Major Sector:Electric

Power and Other Energy

Document:http://

imagebank/servlet/

WDS_IBank_Servlet?

pcont=details&eid=000094946_0003010

This project sought to:

(1) Extend electricity supply to more consumers in the Southern and Central Regions of the Lao PDR;(2) Enhance the quality of supply for existing consumers in those provinces(3) Improve Electricité du Laos (EDL) system efficiency(4) Continue EdL's institutional development (5) Develop additional power export prospects for the Lao PDR.

The project provided for a comprehensive technical assistance program and consulting services for EdL to support its institutional development.

Institutional Development

15

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons Learned5341252

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedLao: Second

Highway Improvement

Project

Project P004201

Robitaille, Denis

Major Sector:

Transportation

Document:http://

imagebank.worldbank.org/

servlet/WDS_IBank_

Servlet?stype=AllWords&all=p004201&ptype=sSrch&pcont=results&sortby=D&sortcat=D

Project aimed to increase the efficiency of the country’s road network through: a) financing a road maintenance program including the execution of spot improvements and periodic maintenance on about 2,000 km of the Borrower ' s network; (b) upgrading about 135 km of the Savannakhet-Pakse road to bituminous standard, including rehabilitation of 36 bridges; and, (c) strengthening institutional planning capabilities

CE Objectives: 1) To strengthen institutional capacities to plan, manage, and maintain the national highway network; and, 2) to support the development of local contracting capacity through the commercialization of relevant State Owned Enterprises (SOE). Components: 1) Provision of technical assistance to develop capacity for project design and implementation, the establishment of RMMD and TPU; cost accounting; estimating and project management; project audits; feasibility studies and design; and transport planning and policy development; and, 2) provision of training through degree courses and short-term training for Ministry of Communications, Transport, Post and Construction staff; seminars; lectures on-the-job-training.Outcome: Institutional and capacity development impact was positive. Capacity of government and its agencies in negotiations, road maintenance planning, budgetary, and control has improved. Progress was made in the practices of contracting out, project management, and SOEs privatization. In particular: 1) RMMD and TPU were

Ministry of Communications, Transport, Post, and Construction

Successful development of annual program and contract documents; successful establishment and of RMMD and TPU; successful implementation of RMMD and TPU objectives (which include design and establishment of road numbering and reference system; development and implementation of road maintenance management system etc); number of staff trained.

-Capacity building requires sustained involvement over time. This sustained engagement was most responsible for the huge successes that manifested as great improvement of the skills of counterpart staff and capacity of local sector-institutions in Laos.-To increase the effectiveness TA assignments and the transfer of skills to local nationals, consultants should prepare and implement a Skill Transfer Action Plan, and the Government should assign appropriate counterpart staff at the beginning of the project and avoid turnover. -Considering the local institutional constraints, the development of a modem and efficient maintenance organization will take

16

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons Learnedestablished and was able to prepare a five-year maintenance program; develop a bridge maintenance database and program, standard bid documents, annual maintenance programs and documents for management of works; 2) Trained local staff are now able to undertake some of the assignments previously undertaken by foreign experts, such as, provision of advice, support, and training for formulation of transport plans etc

much longer than normally anticipated, therefore expectations under any individual project must be realistic. - An incremental transport sector approach--based on a series of rehabilitation projects with increased focus on important capacity building components--has proved to be effective in Lao.

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedLao:Third Structural

Adjustment Credit Project

Project P004209 Graham,

Hugh Trentham

Major Sector:Multisector

Document:http://

imagebank.worldbank.org/

servlet/

Project aimed to assist the Government of Lao in implementing economic reforms in its transition to a market-based economy. Specifically, it sought to assist the government in providing a strong environment for private-sector development and for improved public resource management. It focused on deepening institutional reforms by building on the achievements in public enterprise, budgetary and financial sector

CE Objective: Project did not have a clearly defined CE objectiveComponents: Provision of technical assistance for: 1) development of BoL’s capacity for bank supervision; 2) assisting the State Property Department to design an information system to inventory state assets, 3) monitoring the performance of SOE’s and controlling lease contracts; and, 4) supporting legal reformOutcome:Project outcome was unsatisfactory, and sustainability is uncertain. Some improvements were however made setting up prudential regulations for the banking sector

Government of Lao PDR and its relevant agencies

None outlined The design of such support needs to, but did not, take adequate account of weak client management capacity. Operations need to seek substantive reforms that are likely to have high impact. (Most of the measures supported by SAC were unlikely to have a positive lasting impact without additional and deeper measures.) Furthermore, project implementation

17

Project Name Description CE Component (T, TA, KS) Target Performance Indicators Lessons LearnedWDS_IBank_

Servlet?all=p004209&ptype=sSrch&pcont=results

management reform, strengthening the legal framework and developing a financially viable social security system.

suffered because most decision makers interacted little with the bank and thus did not understand much of the credits details and objectives. More effort need to be put into building such broad support and understanding reform.

18

APPENDIX 2: OED PROJECT ANALYSIS

P004195Upland Agriculture Development Project Institutional Development: Negligible

The ICR praises the project’s success “in gaining acceptance of the bottom-up planningprocess” and the strengthening of water user organizations and women’s groups (p. 6). Apart from the Nam Mone water users association, the OED mission found little evidence of this. The key point is that the government is now providing little or no funding to activities sponsored by the project. The project has not strengthened the capacity of the district and provincial offices responsible for extension; indeed, there was little explicit coordination with these agencies.

P004196Forest Management and Conservation Project

Institutional Development: Negligible

Although the village forestry development pilot strengthened local management, there isno evidence so far that the model will be replicated. The original intention of setting up a national training program for village forestry was not achieved, although a training manual was prepared and may be taken up by the follow-on project. The team working on the protected areas component was unable to develop a replicable model of integrated conservation and development, owing to delays in funding and disputes involving villagers in the conservation areas. The lack of progress on policy reform means that there was little scope for increasing the efficiency with which human and natural resources were deployed.

P004197Provincial Grid Integration Project

Institutional Development: Modest

The discussion in the ICR is input-centered, describing the training and studies designed to serve the objective of continuing EdL’s institutional development. The previously cited evidence on system-wide efficiency does not back up the ICR’s claim that billing and collection systems were improved. The main institutional contribution made by the project was to restructure the company so that Generating Stations and Distribution Branches are operated as separate profit centers, each with their own balance sheets. This make s it possible to identify sources of loss more easily.

19

P004201Second Highway Improvement Project

Significant Outcomes/Impacts: The capacity of Transport Planning Unit (TPU) has been significantly strengthened. TPU provides data, advice and training to MCTPC staff to develop its transport plan. TPU also provides training and assistance to the provincial departments and conducts annual traffic counts.

ICR Rating OED Review Rating

Institutional Dev.: Substantial Substantial

Lessons of Broad Applicability:

Strengthening the capacity of institutions to properly maintain and preserve infrastructure assets requires sustained commitment by the Bank to develop a long-term sector policy framework. A sustained transport sector approach–based on a series of rehabilitation projects with increased focus on maintenance and important capacity building components–has proved to be effective in Laos. This framework has facilitated timely response to implementation issues and proved to be conducive to building Borrower's commitment towards a long-term sector development goal.

20

APPENDIX 3: INTERVIEWS

LAND TITLING PROJECT (1)

Task Manager: Wael ZakoutLand Titling Project (2) will be presented to the Board in a week. Emphasis on Capacity Enhancement is much greater in the second phase.

General The Department of Land went from an organization with 12 employees to a

much bigger and more established institution of about 1000 employees (After significant investment in training)

Laos capacity weakness is more similar to that of Africa than to East Asia After Vietnamese war, Laos experienced serious exodus of well-educated

citizens. Approx 1/3 of the educated population left. Unlike other communist countries, Laos did not pay high attention to the quality

of education, seriously weakening the education system as a result. (Eg. 12th grade in Laos, almost equivalent to 8th grade in Australia)

Therefore, there is great challenge involved in training people with a very a low level of education.

1. What factors, in your view, contributed towards the achievements of the project vis-à-vis its objectives? The objectives of the project were to improve the security of land tenure, to create

laws, institutions and issue land titles. Giving the project a realistic time frame: given that this is a long term project,

there has been good progresso In about five years, one fifth of the people have been titled

Heavy technical assistance financed by Australians was a contributing factor to for progress- otherwise, the Lao people could not have done this independently.

Another positive factor was the political system. (usually viewed negatively with the exception of this case) Since it is communist country with a one-party system, regulations were issued very quickly. (Eg. Land law regulations reform too about 2 months as opposed to 5 years in the Philippines- a democratic state)

2. Was Capacity Development (CD)/Capacity Enhancement (CE) an articulated objective of the project? If so, how was it defined? Yes it was - to establish a sustainable land management system. It was defined

more clearly as Building sustainable institutions at both the centralized and local level.

It is measured through “Minimum Service Standards”, which is the mechanism for establishing an efficient land management system involving internal and external monitoring.

A high degree of training and multi tasking is required to achieve such an efficient system. (This includes: How long it takes to process registration, how much taxes are paid per transaction)

21

3. Which/What performance was supposed to be improved? Improvement of Service standard delivery

4. Who were the beneficiaries/target audience for enhanced capacity? Government staff at both central and local levels For the education/training program, the beneficiaries included both government

and private sector staff. Trying to increase enrollment of women and ethnic groups by implementing a

minimum 30% enrollment rate.

5. Were there any established measures for assessing CE? Performance measures: Service standards, land offices (how many titles were

achieved in one day) This was done through technical quality review. Review of both product and quality feeds into assessment of capacity

enhancement.

6. Was the client sensitized to the importance of CE? Do they see it as linked to project outcomes? Yes, very much so. They recognized their own weaknesses and always requested

training Very eager to receive support on capacity side CE was number one priority for government officials.

7. How did you solicit client cooperation? Was there a champion? How did you identify them? They were approached very easily, naturally came about as part of the design of

the project. Champion: Vice Minister of Development Cooperation –VERY keen on

education and training for government officials. (Madame Khempeng) Oversight agencies also very open and cooperative

8. Recommendations: the dos and don’ts (Guidance for future task managers)Don’ts

Do not assume that TA always builds capacity of organizations, and this is very true for Laos.

The effectiveness of TA has long been debated in the Bank. In the case of Laos, TA was not always very effective.

Technical experts are very knowledgeable but not always skillful in training and disseminating their knowledge effectively.

There is a high risk of a vacuum being created when they leave the country. In the beginning: TAs were financed against input per month – which was not a

good idea It was then decided to finance them against output per month, but did not work in

the favor of the Lao people because TAs did not focus on the training as much as the output.

22

Currently trying to be more innovative by balancing about 30% output, 30% input, 30% based specifically on successful client training and results, 10% service standard results.

Dos Realize that CE is a long term process and therefore needs a long time frame. Need training in the short term and Education in the long term to achieve more

sustainable results.

Current Problem Civil service salary: once the civil servants are trained and better educated they

are recruited by the private sector, thereby draining the government of educated staff.

Without addressing this public sector reform issue, it will continue to be a serious constraint that limits capacity enhancement

9. What types of knowledge or CE support would you recommend going forwarda. For policy makers/leadership

i. Study tours to give them the opportunity to see what other countries are doing. Give them a chance to think outside the box

ii. Conferences often work, however the decision makers at the senior level rarely attend these events, therefore lessening their effectiveness.

iii. One on one leadership training at high levels very effective- something the Bank does not focus enough on. However access to higher levels differs by country. In Laos, vice ministers are the main contacts.

b. Implementersi. Study tours somewhat effective for this group

ii. Very focused technical trainingiii. Need to link training to performance standards

10. If you had to prioritize a particular audience and a specific outcome in your program that would be affected by higher investments in Capacity Development, what would it be?Education Investment in Education: this is an top priority and then benefits of such an

investment could spill over into other elements. Training

o Technical training is essential. It is also important to provide incentive to `the trainers and trainees.

Community Awareness Increase community awareness on legal rights, what to do with land title once

they receive it.

23

District Upland Development and Conservation Project

Task Manager: Susan S. Shen

General facts on the project

The overall development objective of the project is the adoption of more intensified agricultural practices by farmers in the pilot areas as a means to improve the livelihoods of local communities while at the same time conserve the biodiversity of a high priority protected area. The experiences gained and lessons learned from the LIL would provide timely inputs into the implementation of the Government's new Rural Development Program, IDA's proposed longer-term engagement in rural poverty alleviation and naturalresources management, and other donor-funded programs addressing natural resources development and conservation in Lao PDR.

1. What factors, in your view, contributed towards the achievements of the project vis-à-vis its objectives?

Institutional Strengthening and Monitoring and Evaluation Component. This component would improve the management and technical skills of staff to implement activities at the provincial-, district-, and village-levels. It would also test whether the current administrative implementation mechanism and decentralization process are enhancing delivery of services to beneficiaries and the extent to which village organizations have the capacity to absorb these services.

A simple and practical monitoring and evaluation (M&E) system would be established to ensure timely qualitative and quantitative feedback on project implementation to make needed corrections.

2. Was Capacity Development (CD)/Capacity Enhancement (CE) an articulated objective of the project? If so, how was it defined?

Not exactly. The main project objective is the adoption of more intensified agricultural practices by farmers in the pilot areas as a means to improve the livelihoods of local communities while at the same time conserve the biodiversity of a high priority protected area.

However, capacity building/capacity enhancement, mainly reflected in the “Institutional Strengthening and Monitoring and Evaluation Component” did help achieve the above objective.

3. Which/What performance was supposed to be improved?

Overall institutional strengthening. Mainly at local level (district/village levels).

24

The achievement of the performance relies on country government to a large extent.

Experiences learned from the project are useful for both LAOS PDR and other similar countries.

4. Who were the beneficiaries/target audience for enhanced capacity? Local level (district and village), especially the village level.

5. Were there any established measures for assessing CE?

At least 75% of field arrangements to teams confident of their effectively deliver upland ability to implement poverty alleviation project activities.

At the district-level, a District Rural Development Committee (DRDC) would be established to serve as the District Project Steering Committee (DPSC) with the Nakai District Vice-Governor as the chairman. A small administrative office, the Project Office (PO), would be established and attached to the District Vice Governor's office consisting of a District Project Director (DPD), an Accountant/Bookkeeper, Procurement/Monitoring Officer, and Clerk/Secretary to assist the DPSC in management and monitoring.

At the field and village-level, a site management team consisting of an NBCA staff with support from one or two local consultants would be established and located at each of the three pilot villages to coordinate activities in these villages as well as in the three sub-watersheds of the Project Area.

Villagers (Committee on Economic Affairs) satisfied with field team and supporting management's provision of relevant project services of acceptable quality.

6. Was the client sensitized to the importance of CE? Do they see it as linked to project outcomes? Yes, very much so. Human Resource development is the objective of the

government. They recognized training, a kind of outcome from the project, is necessary to

build human resource capacity.

7. How did you solicit client cooperation? Was there a champion? How did you identify them? The development of human resources is in line with the strategic objective of the

country. The country is working hard to build its staff stock via different projects. They were approached very easily, naturally came about as part of the design of

the project.

8. Recommendations: the dos and don’ts (Guidance for future task managers)

Training is useful. However in most cases, especially in this project, the government may not send the right people for training. Since the salary is too

25

low, the government officials view training as a channel for benefit distribution. Therefore, there are various people for training at different sessions. This significantly reduces the effectiveness of training.

The effectiveness of training, in the case of Laos, was not always very effective. CE is a long term process and therefore needs a long time frame. Civil service salary: once the civil servants are trained and better educated they

are recruited by the private sector, thereby draining the government of educated staff.

9. What types of knowledge or CE support would you recommend going forward

For both policy makers/leadership and implementers, it is not a training issue indeed. It is a strategic issue.

We need better incentive structure. People behave quite differently in the project. There is a sectoral difference in

Laos PDR. People from health and education sectors are more likely to work at local level and get things done. On the contrary, people from agricultural sector are reluctant to work at local levels. They are very difficult to be motivated. One potential reason is that the outcomes for education and health sector are much more obvious compared to that from agricultural sector.

10. If you had to prioritize a particular audience and a specific outcome in your program that would be affected by higher investments in Capacity Development, what would it be?

Yes. The staff at the district level is the target audience. They are the people to implement the project. The objective is to get things done and we would like them to view the implementation of the project in a professional way.

The people at village level are given sufficient training. The current village structure at Laos PDR makes sure that village people have much better performance.

26

Financial Management Capacity Building Credit (FMCBC)

Interview with Abha Joshi-Ghani Project recently approved by the Board in February 2003This is not a specific project, but rather an adjustment credit. Reason for Adjustment Credit: Lao PDR banking sector is burdened by loans which were financing non-performing State-Owned Enterprises

There are close ties between the proposed FMCBC and the Nam Theun 2, which is a Bank project aimed at building large hydro infrastructure and selling electricity to Thailand in the future.

Milestones set for the Adjustment Credit were: Restructuring of 5 State Owned Enterprises

o Measurement of SOE Restructuring is based on output where concrete and tangible quantitative indicators can be used.

Legal and Regulatory Changeso Intangible and difficult to measure: change will occur quickly on paper,

however the defining factor will be whether the framework will be implemented and will function (eg. entry into the sector, competition, etc)

Aviation Sector reform- need tariff structure reform – currently not good for robust entrepreneurship

Entry of private sector- currently no regulatory framework for the few PS enterprises that do exist.

Auditing of banks, non-performing loans/assets Change in the state-owned mentality to private sector mentality through change in

Management

It is very difficult to meet milestones with donor money as the government is allowed very limited input and has little control on spending. World Bank suggested to Lao Government to borrow for Technical Assistance to

improve these areas and meet milestones. WB suggested to bring in local consultants as much as possible, to provide technical

assistance, training and study tours. In addition, borrowing for CE allows them to control the project.

Lao gov. still wanted to bring in donor money, however compromise was made that if donor money was not specifically meeting the goals that were set out, then Lao would borrow for specific projects involving CE.

Annual Lao donor meeting to monitor spending and keep track of grants for different projects as well as achievements.

A Project Implementation Unit (PIU) in the Ministry of Finance was set up for this multi-sectoral project. PIU is led by senior MoF official with assistant director. PIU

27

is fairy autonomous. The budget comes from the adjustment credit. They keep track of the activities of different ministries in meeting deadlines and CE goals (hiring of consultants, etc). So far they have achieved positive results with the PIU.

Study Tours STs are the crux of CE. They include hands-on training and TA. Difficulty in convincing gov. to borrow for these activities but finally agreed on USD

2.5 million for Study Tours and training.

Consultants Proposal to hire consultants who could identify training needs (as well as requests)

and put in place a systematic plan with regional educational institutions who could offer this training (either in neighboring countries or by bringing in WBI consultants)

CE alone is difficult to sell. Idea finally sold based not only on CE but with the Adjustment credit.

Important factors to consider: Changing the mindset of civil servants Training to do their jobs better through exposure to different methods in different, but

similar, countries Grassroots, inclusive bottom up approach. (Eg. WB sponsored selected officials to

attend an APEC meeting; this resulted in openness and exposure of the Lao officials to the activities of other countries their size which are performing much better economically- thereby having a positive impact on them and slightly opening up their thinking)

Involvement of other donors in CE is crucial in maintaining harmony with the country and with the donors as well (Ausaid, KFW, etc)

28

Lao: Summary of five project analyses and three Task Manager interviews

Task Managers interviewed1. Denis Robitaille – Road Maintenance Project2. William D. O Paterson - Road Maintenance Project3. Wael Zakout – Land Titling Project

Capacity enhancement objectives were clearly defined in all of the projects except for one—Third Structural Adjustment Credit Project.

In some cases however, CE objectives may have been better defined for more effective alignment with project goals. An example: In the Second Highway Improvement Project, Task Manager Denis Robitaille was unequivocal in stating that the ultimate purpose of the entire project was to create a road maintenance system and culture in Laos. Now, the biggest hurdle to attaining this was convincing the stakeholders that maintenance is as essential as construction, and this would be achieved through capacity enhancement. In spite of its central importance, this was not specifically identified and defined as a CE objective.

So, even though CE objectives were, in most cases, clearly defined, those defined objectives were not necessarily the optimal objectives.

Both the project documents and the Task Managers seemed very clear on what performance the CE initiatives were supposed to improve, and who the beneficiaries would be.

Many of the projects lacked an adequate definition of how effectiveness or progress of CE may be assessed. Performance indicators for CE were not always clearly outlined. (In some projects they are not even mentioned at all!) And even when they are outlined, some of the outlines exhibited a lack of readily measurable and relevant yardsticks. Indicators that could and should have been included are missed out, and language is vague—creating room for multiple interpretations.

All Task Managers report that there was an appreciation of the significance and importance of CE to project objectives amongst the clients. This aspect is almost ignored in the ICRs however.

Client cooperation was almost always acquired through champions. Champions were usually identified by their progressiveness, and they were usually around the upper middle management cadre. Two task managers report that this cadre would also be the best audience to target with increased CE investments.

29

TM recommendations and Lessons Learned: Capacity building requires a lot of time investments. To ensure success,

engagement should be sustained perhaps for a longer duration than the project Proper and thorough analysis of existing capacity needs to, but was not always,

carried out from onset of project design. Capacity building efforts should increase in such areas as helping build feeling of

ownership in clients and changing general thinking patterns. Set up consultant monitoring and evaluation systems with clear and definite

yardsticks for measuring knowledge transfer to counterparts. Training should move more towards the model of acting as a primer to the

beneficiaries, and away from the model of delivering lectures. TMs agree that on-the-job training is the most effective vehicle for knowledge

transfer. But it is hardly comprehensive—since in most cases, it deals with issues as they arise. It should be supported with “formal” classroom training which will cover concepts etc. Recommended split of efforts between the two—20% formal training; 80% on-the-job.

Beneficiaries prefer to see, and consequently learn a lot, from interacting with same-level officers from similar but more effective/efficient institutions either within or without their countries. This can be achieved through Study tours, facilitating flow of information between the two parties, sharing other countries experiences etc.

Much more time and effort should be spent at the onset of project designing very specific, measurable, and project-objective-relevant Performance Indicators.

Sundry Observation: Most of the paperwork generated for these projects are highly insufficient in

addressing CE activities. They fail to capture some of the more important CE issues like implementation procedures, specific outcomes and performance indicators.

30

31