cara mid-term evaluation final report · cara mid-term evaluation final report ii ... evidence of...

Central American Water Resource Management Network

CIDA Project No 01888/S55601

Final Evaluation Report

Prepared for CIDA by

Neil ThomasTrevorrow Ltd

Mallorytown ON

June 2002

Table of Contents

GlossaryExecutive Summary

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Evaluation Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.1 Evaluation Approach and Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Project Goal & Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3. Project Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.1 Summary of Project activities to date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43.2 Results by Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2.1 Institutional Strengthening (IS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2.1.1 Gender Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2.2 Network Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2.3 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.4 Applied Thesis Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2.5 Development Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.6 Private Sector Involvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.7 Internationalization of Canadian Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4. Project Management & Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.1 The Project Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 Project Committees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.3 Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5. Observations and Synthesis of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.1 Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.2 Contribution of Results to Meeting Project Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.3 Achievements of CARA in relation to UPCD Goal & Objectives . . . . . . . . . . . . . . . . . . . . . . 195.4 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.4.1 Course-only M.Sc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.4.2 Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6. List of Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

7. Conclusions & Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Appendices:

1. Terms of Reference for the Evaluation2. Itinerary & Persons Met3. Proposed TOR for the CA CARA Coordinator4. MSc Students by Institution5. Documents Consulted

CARA Mid-Term Evaluation Final Report

Glossary

CA Central AmericaCARA Central American Water Resource Management NetworkCCAD Comite Centroamericano del Ambiente y DesarrolloCI Canadian InstitutionCIDA Canadian International Development AgencyCIRA Centro de Investigaciones sobre Recursos AcuaticosECAG Escuela Centroamericana de GeologiaIDRC International Development Research CentreIS Institutional StrengtheningMSc Master of ScienceNGO Non-Governmental OrganizationPI Partner Institution (in Central America)RBM Results-Based ManagementUC University of CalgaryUCR Universidad de Costa RicaUNAN Universidad Nacional Autonoma de NicaraguaUPCD University Partnerships in Cooperation and DevelopmentUSAC Universidad de San Carlos (Guatemala)UW University of Waterloo

CARA Mid-Term Evaluation Final Report i

Central American Water Resource Management NetworkCIDA Project No 01888/S55601

Mid Term Evaluation Executive Summary

Introduction

In 1999, under the auspices of theInstitutional Cooperation Division’s “UniversityPartnerships in Cooperation and Development(UPCD)”, CIDA signed a Tier 1 Grant Agreementwith the University of Calgary for theimplementation of the Central American WaterResource Management Network (CARA). CARAaims to increase capacity of Central Americancountries to educate and train human resources inthe area of hydrogeology and water resourcemanagement. The Canadian partners are theUniversities of Calgary (UC) and Waterloo (UW),and the Central American partners are theUniversidad de Costa Rica (UCR), the UniversidadNacional Autonoma de Nicaragua (UNAN), and theUniversidad de San Carlos de Guatemala (USAC).The CIDA grant is for $3,995,000, with scheduledtermination in March 2005. There had been previousCanadian involvement in this same field at UCRwith IDRC funding. The evaluator visited all theparticipating institutions.

Project Goal & Objectives

The project has three objectives which areexpected to lead to the achievement of the goal ofimproved management of Central American waterresources:

6. To create high quality hydrogeology andintegrated water resource managementtraining programs at three PartnerInstitutions (PIs) in Central America (CA).

7. To create a sustainable partnership amongstuniversities and other stakeholders in theCA water sector through the creation ofCARA.

8. To positively influence management of CAwater resources through training of watersector professionals working in the publicand private (business and NGO) sectors.

The project comprises seven technicalcomponents:

• Institutional Strengthening:• Network Building• Training• Applied Thesis Projects• Development Education• Private Sector Involvement • Internationalization of Cdn Universities

The first four components principallyaddress the Capacity Building Strategies of each ofthe three Central American partner institutions. Thelatter three speak more to benefits which will accrueto Canadian institutions and businesses. The projecthas a specific gender objective which appears at anactivity level: the hiring of women professors at thePIs. Through both this, and ensuring a highrepresentation of women as students, the projecthoped to increase the representation of women asprofessional hydrogeologists in Central America.Specific gender targets are not defined.

Project Implementation

In overall terms, since project initiation inDecember 1999, the first calendar year was occupiedin PI institutional assessments and planning of thenew M.Sc. Courses at UNAN and USAC, and thesubsequent two calendar years in implementing thenew programs at the latter institutions, andsupporting the fourth cycle at UCR.

Results by Component

Since, and including, the Tier 1 Proposal,this project has been extremely well-thought-out andcarefully crafted. The proposal shows clear attentionto outputs, indicators and assumptions, and an RBMtraining and workshop session held with the PIs inthe first year introduced them to the managementapproach and style UC has followed since (andresulted in a redefinition of some expected resultsand indicators).

CARA Mid-Term Evaluation Final Report ii

Institutional Strengthening (IS)

Institutional strengthening must be seenwithin the context of institutional development at thetime of project inception. UCR already had an MScprogram in place; UNAN and USAC did not (thoughUSAC already had some plans to develop such aprogram). IS is almost completely founded in theexistence and quality of the MSc program, and theability of the host institution to deliver it. The factthat the project was able to initiate two new MScprograms within the first full calendar year ofimplementation (the programs physically admittedstudents in the last quarter of Project Yr 2) can beseen as a significant accomplishment, and strongevidence of early PI commitment.

A second major element of IS is that offaculty-in-training. A total of 4 persons is currentlyin MSc training in Canada, at either UC or UW, andat UCR. UCR has been an important resource forsuch training - staff at both UNAN and USAC havecompleted their MSc there.

PI MSc curricula show some specializationat each of the three institutions, depending on theirhydrogeology, water quality or agronomy focus.Programs include a combination of full credit andshort courses. Each institution has responded to lackof academic capacity in different ways: USAC inparticular has tapped Mexican and Czechinstitutions for visiting professors; UNAN has beenthe most dependent on Canadian teachers. At UCRtwo staff members provide the core of programteaching.

Network Building

At present, the CARA ‘network’ fallssignificantly short of a functioning mechanism. Theprinciple reason for this is that each of the three PIsis almost fully preoccupied with its ownprogram.The network has, to date, been largelydriven by UC. The Project Manager assumed the roleof Coordinator in Yr 1. In Yr 2, a staff member atUSAC was elected CA Coordinator at the SteeringCommittee Meeting; the role has now just passed toa UNAN staff member.

One of the difficulties in justifying thepresent network (to the extent that it exists) is thedegree to which the PIs consider that benefits accrue

from internal (i.e. regional) linkages. At least one PIindicated: ‘we need Canada more than we need eachother’ - a reference to UC’s and UW’s renownedexpertise in hydrogeology. To a degree, PIs do notseparate the network concept from thestraightforward technical assistance role provided bythe Canadian institutions.

It is suggested that the Project might fosterthe network within CCAD, the Central AmericanCommission for Environment and Development.

Training

Training is the core of the project, as theother PI-focused initiatives are intended to ensurethe existence of high-quality resources for training tothe MSc level. Short-course events are intended toreach a wider audience, though the MSc studentsremain the overriding target of short courses also.

At the time of the evaluation, there weresix, 10 and five students in the 2001/02 MSc cyclesat UCR, UNAN and USAC, respectively. Of thesestudents, one, nine and one are women, respectively.Three of UCR’s six students are Guatemalan, andthe remaining three Costa Rican. UNAN and USACcurrently only admit their own nationals. Non-nationals have been a significant percentage ofUCR’s students since the first IDRC-funded cycle. Itis the evaluator’s position that at this point UNANshould not be contemplating hosting students fromother countries, though in the case of USAC thereare fewer doubts. UNAN needs to sort outcoordination and staff issues which are unlikely to befully resolved in the short-term, and which reallyshould be tested in a situation of stability for a fullcycle before contemplating the added complexity offoreign students.

The students at all PIs indicated that thelack of practical content in many courses is aconstraint to understanding and applying the theory.The programs at each PI still depend on shortcourses to deliver critical material. UC must ensurethat consultants contracted be credible and wellprepared. This may result in a higher consultancycosts.

Gender Representation

There is a high representation of women in

CARA Mid-Term Evaluation Final Report iii

the student body of the programs at UCR(historically) and UNAN (presently). Sixty percent ofthe current faculty-in-training are women; womenare involved in the teaching activities at both UCRand UNAN; CARA coordinators at both UCR andUNAN are women. The only institution wherewomen are under-represented is USAC. The latterfact has been noted by both the USAC Coordinatorand UC. USAC will only be able to redress theimbalance through the student intake. This may haveto be addressed by assigning two of USAC’s fourscholarships exclusively to women.

Applied Thesis Projects

The project is in the first cycle of thesisprojects, though with field studies little advanced.An analysis of the 22 research proposals suggestedthat intended engagement of stakeholder is evidentin less than 50% of the proposals. Proposals showlittle intent of feeding results back to interestedparties.

The evaluation notes concern forachievement of student graduations in 2002 inNicaragua, and to a lesser extent in Guatemala. Itdoes not appear to be an issue for full-time studentsin Costa Rica.

While conceptually it is perhaps an elementof both the training program and developmenteducation, CI student activity in CA is groupedunder applied research projects. Three CI studentshave completed MSc research in CA (at UCR), andfour more will be undertaking similar research in Yr4 (two of them in Mexico).

Development Education

This component has yet to receive majorattention. Apart from minor activities, such aswebsite development and poster displays, the mainactivity was participation in a CIDA sponsoreddocumentary on various CA projects prepared byCanadian Learning Television. This video isreported to have been aired several times.The Projectintends to focus in Yr 4 on the development of asecondary school geography curriculum on waterissues, to compare Canadian and CA water issues.

Private Sector Involvement

Private sector involvement has been limitedso far to three companies either providing short-courses or internships. The outcome of these contactsis not directly tangible. While Canadianhydrogeology expertise could undoubtedly enhanceCA water-resource management, it does not yetappear to be a field of significant businessopportunity.

Internationalization of Canadian Institutions

There is no doubt that both faculty andstudents have a broader understanding of CA waterissues as a result of exchanges; to what extent this issignificant in the overall context of course contentcould not be determined. It appears most likely thatgraduate students are the main contributors, as theyevolve, investigate and present, research findings. Atleast one CI professor indicated the use of CA casematerial in his teaching.

The activity focus of the workplan results inthe majority of this component being conceptualizedunder IS, Training and Applied Thesis Projects.There is a need to revisit the purpose of thiscomponent, and redefine the expected results.Internationalization is a strategic priority for UC, butthe linkages are not necessarily in place to ensurethis strengthens UC’s teaching and advancementprocesses.

Project Management

Project Management shows close attentionto detail - the framework of outputs, outcomes andindicators is evidence of the thought and care whichhas gone into design. The indicator-based approachhas been extended to the PIs, where the AssistantProject Manager has undertaken participatorymonitoring against the Capacity Building Strategies.Reports are easy to follow and progress towardsoutcomes understood. The comprehensiveness of theframework allows a clear picture of progress towardsProject goal.

Inter-institutional relationships between CIsand PIs have been excellent, except for some strainbetween UC and UCR. Certain changes occurred atUCR at Project Inception which were adisappointment to the Project Manager. Mutual

CARA Mid-Term Evaluation Final Report iv

tensions have made it difficult to make progress onsome issues. It is necessary for the Project Managerto rebuild bridges at UCR. Part of the latter willrequire that he see that UCR has made a significantcontribution to IS of the other PIs, and that it is stillan institution with a regional commitment. It is alsoimportant for the Project Manager to step back fromsome of the internal issues, and allow the PIs tomanage them themselves. He should spend Yr 4fostering the strategic mechanisms important to thebroader outcome of the Project.

Budget

At 30 September 2001, the Project haddisbursed approximately 20% of its budget.Assuming a disbursement rate in the secondsemester equivalent to the first semester of 2001/02,the Yr 3 budget will still be underspent by about35%, and will take overall disbursement up to about28%. Overall variances are generally uniformbetween Project components. UC notes thatunderexpenditures are a combination of delayedactivities, lower activity levels or unreportedexpenses.

Observations and Synthesis of Findings

Rationale

The water resource management focus ofthe Project speaks quite clearly to an increasinglypressing need in Central America: knowledge andeffective management of water resources in a regionof markedly variable physiography and land use.Water is one of the four strategic areas of the CCAD,under which this body has defined three objectives:

• guarantee the protection of water sources;• ensure the long-term supply of water (in

quantities and of a quality adequate formultiple uses);

• encourage a complete economic valuation ofregional water resources.

Contribution of Results to Meeting Project Goal

The Project is at a critical point. It hassuccessfully established two MSc programs at twoCA universities, modelled on the one established atUCR (which itself depended on the UW pattern)with the support of the same group of Canadian

scientists. However, what may be termed second-generation issues, which have surfaced at UCR, maysurface similarly at UNAN and USAC, and it will bethe successful resolution of these issues which willresult in meeting the Project goal. Very largely, theseissues surround a successful institutionalization ofeach program beyond the vision or efforts ofindividual champions.

A ranking has been developed to respond tothe question ‘Where should be the main emphasis inthe remaining years to increase the likelihood ofachieving the Project goal?’Order of ranking wasdeveloped as the relative importance of componentsfor continued investment of Project resources interms of contribution of probable outcome to Projectgoal. The ranking indicates that the balance ofinvestment should favour network building andtraining as the two most important, ongoingcomponents; investment in the remainingcomponents will, for a variety of reasons, have lesseffect on goal achievement.

Achievements of CARA in relation to UPCD Goal& Objectives

At this mid-point in the Project, theprogress towards achievement of UPCD objectives issignificant. To give a general indication of progressby UPCD objective, another ranking has beendeveloped. The Program still has sufficient time andresources to make further progress towards itsoverall goal.

UPCD Goals & Objectives Ranking

Goal: To increase, in a sustainable manner, thecapacity of developing countries to educateand train the human resources required to meettheir priority development needs

4

Obj 1: To strengthen developing countryinstitutions responsible for the training ofhuman resources

4

Obj 2: To contribute to the establishment oflinkages fostering sustainable partnershipsbetween Canadian and developing countryinstitutions

3

Obj 3: To contribute to theinternationalization of Canadian Universitiesin all aspects of their mission and to thepromotion of Canadian university expertise inhigher education

3

CARA Mid-Term Evaluation Final Report v

Obj 4: To contribute to the developmenteducation of the Canadian University and thecommunity it serves

2

ODA Obj: Support for the full participation ofwomen as equal partners in the sustainabledevelopment of their societies

3

ODA Obj: A commitment to help developingcountries protect their environment andcontribute to addressing global and regionalenvironmental issues

3

ODA Obj: Promote sustained and equitableeconomic growth by supporting private sectordevelopment in developing countries

1

Ranking

0 - no perceptible progress1 - minor change or progress, but which cannot specifically beattributed to the Program2 - minor change or progress, some of which is attributable to theProgram3 - significant change, most of which is attributable to the Program4 - high probability of achieving outcomes

Concepts

Many of the recommendations made in thereport relate to minor process or effectiveness issues.However, there are two specific areas of the Projectwhere a conceptual review might contribute toimproved realization of certain outcomes. Both ofthese would imply a significant shift in the wayProject resources are applied.

Course-only M.Sc.

From the historic UCR data, the risk forwomen students of not graduating appears a high-level risk (>50%; 20% in males). While an effort hasbeen made to accommodate female students throughpart-time studies, there is no indication that this hasinfluenced graduation rates. UCR points out thatwomen do not require a Master’s degree in order towork as hydrogeologists.

One of the current underlying assumptionsof the program is that an individual research projectis essential to turning out a well-roundedhydrogeologist, and that the degree should not begranted without it. However, if a student has gonethrough a frustrating process whereby the thesisbecame just a means to graduation rather than thecore of the experience-building (especially in acommunity), then it may be suggested that insisting

on a thesis project adds no value to developingtechnical knowledge. If the thesis trimester were tobe converted into a solid, practical, community-basedfield school it is quite possible for the experiencegained to exceed what even an effective thesisexperience could impart.While recognizing that anynatural-resource focussed program should containelements of practical work, it is suggested here thatthe project goal does not automatically require thatstudents undertake an individual research project tobe effective water-resource managers. More intensivetraining could result in greater impact than half-hearted research.

Research

Intellectual advancement is at the core ofprofessorial quality. It is suggested that the projectcould improve the joint commitment of professor andstudent to the research program, i.e. that thestudent’s research better contribute to the professor’sown research interests and intellectual gain. Bycommitting him/herself to the generation of researchoutputs, and incorporating the student(s) in thatprocess, the professor places a higher expectation ofoutcome on the student than currently exists inCARA. This can be achieved by the allocation ofresearch grants, which include student stipends.

Because the institutional strengtheningcomponent is so focussed on MSc program deliveryit may be said that current expected outputs areadequate for training outcomes, but that they fallshort of the model of a Centre of Excellence. ISshould not stop at training future faculty to a higherpostgraduate level. A Centre of Excellence requiresactive research agendas which position academicstaff at the forefront.

It is noted that the idea of a research fundrecently surfaced, and that an amount of $20,000 hasbeen assigned to each institution for the remainder ofthe project. It is recommended that this researchfunding be further considered, and some significantalterations be made to the way in which perhapssome student support is allocated. The opportunityexists here to establish a creative (and possiblycompetitive) mechanism which could also fosternetwork linkages.

CARA Mid-Term Evaluation Final Report vi

Lessons Learned

Two higher-level lessons learned from theProject to date are especially relevant tosustainability:

1. The sustainability of an MSc programrequires at least three core staff memberswith similar disciplinary background; in athesis-based program, each core staffmember can probably manage no more thantwo student projects effectively

2. Champions serve program developmentwell; whether they serve theinstitutionalization of that same programequally well depends on their collegialvision.

Conclusions and Recommendations

The evaluation has found that the CARAProject has made significant progress towardsachieving its goal. The high commitment of theCanadian institutions has been noted, as has theoverall sense of partnership between the Canadianand Central American Institutions. The evaluationhas noted that there are several areas which could bestrengthened, include conceptual approaches toInstitutional Strengthening, Network Building andTraining. From the full set of recommendations inthe report, the following particularly will enhancethe probability of achieving the Project goal.

Network Building

That the Project foster the network withinCCAD. With motivation provided byCCAD’s strategic focus on water as one offour areas, the Coordinator could serve amuch more fundamental role in regionalinstitution building.

Training

That, by at least the end of calendar year2003, the Project Steering Committeereview Project expenditures to determinewhether sufficient funds remain to supportfurther scholarships in Cycle III, andwhether a recommendation to this effectshould be made to CIDA.

That the partners review present programstructure to determine whether greaterallowance need to be made for research timeduring current ‘teaching’ trimesters.

Management

That the Project Manager rebuild bridges atUCR. UCR has made a significantcontribution to IS of the other PIs, and isstill an institution with a regionalcommitment.

That the Project Manager begin to considerUC’s disengagement strategy. He shouldstep back from some of the internal issues,and allow the PIs to manage themselves. Heshould spend Yr 4 fostering the strategicmechanisms important to the broaderoutcome of the Project.

Concepts

That increased, professorial-level researchfunding be reconsidered, and somesignificant alterations be made to the way inwhich student support is allocated, i.e. thatacademic staff in the three PIs develop andput forward research projects for CARAfunding which could be undertaken in thesame two-year cycle of the MSc program.

CARA Mid-Term Evaluation Final Report vii

1 Central American. UC was not involved in the IDRC projects, so to this extent CARA is not a direct extension.Neither is it inferred intentionally that CARA, as a project, should be seen as responsible for previous management decisions.The expected continuity at UCR, however, across granting mechanisms and agencies, cannot be ignored.

2 Project Nos 93-1550 and 96-8758. There was external support from other sources including the Pan AmericanHealth Organization, the International Atomic Energy Authority and the United Nations Environment Program.

3 Two UW Ph I personnel who were resident in Costa Rica and taught course material continue to be stronglyassociated with the CIDA phase: Mr David Bethune, the Project Manager, and Dr Cathy Ryan, Associate Project Director(Technical), though they now operate from UC. Other UW personnel also continue to provide teaching and research input.

CARA Mid-Term Evaluation Final Report 1

Central American Water Resource Management NetworkCIDA Project No 01888/S55601

Mid-Term Evaluation Final Report

1. Introduction

In 1999, under the auspices of the Institutional Cooperation Division’s “University Partnerships inCooperation and Development (UPCD)”, CIDA signed a Tier 1 Grant Agreement with the University of Calgaryfor the implementation of the Central American Water Resource Management Network (CARA). CARA aims toincrease capacity of Central American countries to educate and train human resources in the area of hydrogeologyand water resource management. The creation of CARA is expected to lead to improved management andprotection of Central American water resources through the partnerships and collaboration of highly-qualifiedwater-resource professionals from universities, public water institutions, NGOs, the private sector, andinternational funding. The Canadian partners are the Universities of Calgary (UC) and Waterloo (UW), and theCentral American partners are the Universidad de Costa Rica (UCR), the Universidad Nacional Autonoma deNicaragua (UNAN), and the Universidad de San Carlos de Guatemala (USAC).

CARA is largely a regional1 extension of earlier Canadian involvement in two IDRC-supported projects -the Latin American Urban Water Management Network, Phases I and II2. These ran from 1993-99, with UWproviding technical support3. Within an overall goal of improving integrated and multidisciplinary approaches towater problems in Latin American cities, Ph I had several specific objectives:

• establishment of a water management/policies training program at the University of Costa Rica:• assisting in establishing other water management training programs (specifically at the University of Sao

Paolo);• supporting activities that promoted coordination and integration in water issues among and within Latin

American countries, both at the institutional or NGO level;• supporting highly-focused research efforts in the water field conceived to address specific problems with

practical applications;• providing a channel for grassroots input into the urban water decision-making process.

Most of the Ph I effort was focused on the first objective, with the first cycle of six full-time and 3 part-time M.Sc. students being admitted in 1995. The UCR M.Sc. program has operated continuously since then on atwo-year cycle (the Cycle 1 part-time students were expected to graduate at the same time as the Cycle 2 full-timestudents). CIDA funding began halfway through Cycle 3; the UCR M.Sc. Program is now nearing the end of Cycle4.

The CIDA grant is for $3,995,000, with scheduled termination in March 2005.

4 Appendix 1. Terms of Reference for Mid-Term Evaluation.

5 A Logical Framework Analysis was appended to UC’s Clarifications Document of August 1999. This evolvedthrough a couple of iterations into the current performance measurement framework.

6 CPB Program-Project Planning Sheet, and Performance Measurement Framework, CARA April 2000.


2. Evaluation Purpose

This Tier 1 Project is approaching its mid-term. CIDA required that the Project be evaluated, and theevaluation results used by stakeholders to make any necessary adjustments to Project approach and methodology inorder to increase the likelihood of achieving outcomes and impact. The evaluation is expected to address thevarious aspects of Project rationale, effectiveness, efficiency, and results and achievements to date. It is expected toproduce a picture of the Project with the context of a set of detailed questions4.

2.1 Evaluation Approach and Methodology

In keeping with CIDA’s emphasis on Results-Based Management (RBM), the evaluation addressedprogram implementation from an RBM perspective. The Project has a comprehensive performance measurementframework, finalized at the time of the Inception Mission (February 2000)5; the output indicators were the ones ofprincipal interest to this mid-term evaluation. Within the framework of Project objectives, the evaluation addressedoutputs expected from each of the Project components, and the likelihood that these outputs would generate theintended outcomes. Where there appeared to be the possibility that an outcome would not be achieved, the initialassumptions were reviewed to determine whether appropriate risks were recognized. The evaluation also examinedProject management for its effectiveness in delivery of outputs.

2.2 Project Goal & Objectives

The Project has three objectives which are expected to lead to the achievement of the goal of improvedmanagement of Central American water resources:

1. To create high quality hydrogeology and integrated water resource management training programs atthree Partner Institutions (PIs) in Central America (CA).

2. To create a sustainable partnership amongst universities and other stakeholders in the CA water sectorthrough the creation of CARA.

3. To positively influence management of CA water resources through training of water sector professionalsworking in the public and private (business and NGO) sectors.

The Project comprises seven technical components. At this component level, the Project defines a series ofoutputs6 expected to bring the objective-level outcomes:

• Institutional Strengthening: PIs appropriately mandated and prepared to offer high qualitycurricula and training programs in hydrogeology and interdisciplinarywater management;

• Network Building PIs champions of a growing, increasingly integrated and self-reliantregional water management network;PIs complement and enhance each other’s teaching and researchcapabilities;

• Training UCR, UNAN and USAC each deliver programs within CARA“Centres of Excellence” model;


PI M.Sc program graduates placed in positions of high responsibilitywithin the public and private sectors of their respective countries;Professional short course, distance education and undergraduatecourse graduates incorporate new knowledge in professional oracademic careers;

• Applied Thesis Projects (Originally termed “Community-Based Field Research”) PI andCanadian Institution (CI) M.Sc students gain valuable CA field datacollection experience and engage relevant stakeholders (e.g. rural andurban communities, local authorities, NGOs, private sector,government agencies, etc.) in projects;

• Development Education CIs and communities they serve (Calgary and Waterloo) are moreinformed about Canada’s role in improving the management of CAwater resources;

• Private Sector Involvement Exposure of Canadian private sector expertise and technology in CAleads to increased business contacts and opportunities for Cdn firms;

• Internationalization of Cdn Universities CI faculty, students, curricula and programs areinternationally enriched leading to enhanced internationalreputations, additional international projects, foreignuniversity agreements and foreign student applicants.

The first four components principally address the Capacity Building Strategies of each of the three CentralAmerican partner institutions (PIs). The latter three speak more to benefits which will accrue to Canadianinstitutions and businesses. Strictly speaking, these outputs are more truly outcomes - at higher planning levels theaggregation of lower-level outputs complicates the separation of output from outcome, though it may be suggestedthat there is also further confusion here - e.g. the output identified for Institutional Strengthening is much closer toa Project assumption. The three separate specific objective-level outcomes are:

• Partner Institution water management training programs have achieved high academic standardswithin their respective institutions, have excellent reputations nationally and internationally, areattracting increasing numbers of students and collaborators;

• The three PIs lead a growing regional university network (CARA) that derives its strength froma) internal resource sharing and collaboration and b) growing demand for CARA from regionalstakeholders such as communities, the public and private sector, NGOs, international agenciesand other networks;

• Graduates of PI M.Sc. programs and short-course attendees influence water sector policies, plansand practices of their respective countries through solid technical knowledge, participatorydevelopment training and experience at a community level, and the incorporation of a“partnership” ethos in their work.

The Project has a specific gender objective which appears at an activity level: the hiring of womenprofessors at the PIs. Through both this, and ensuring a high representation of women as students, the Projecthoped to increase the representation of women as professional hydrogeologists in Central America. Specific gendertargets are not defined.

2.3 Methodology

The CARA Project is composed of a network of Canadian and Central American institutions. Theevaluator met representatives from all collaborating institutions. Prior to institutional visits, the evaluatorconducted file reviews at CIDA, including:

• reviews of minutes of the Project Steering Committee;• reviews of annual workplans, semi-annual and annual technical reports;

7 Summarized from CARA Annual Narrative Reports, and amended from findings.

8 It should be noted that UCR and UNAN disagree over this interpretation.


Pre-mission interviews were then conducted at Canadian partner institutions (Calgary and Waterloo),followed by interviews at the Central American partner institutions (UCR, UNAN and USAC);

Interviews were partially open-ended, using the performance indicator framework as the guide. First levelquestions by component can be found in the Evaluation Workplam. The itinerary and persons met are indicated inAppendix 2.

3. Project Implementation

3.1 Summary of Project activities to date

In overall terms, since Project initiation in December 1999, the first calendar year was occupied in PIinstitutional assessments and planning of the new M.Sc. Courses at UNAN and USAC, and the subsequent twocalendar years in implementing the new programs at the latter institutions, and supporting the fourth cycle at UCR.A brief yearly summary of main events and/or milestones follows7.

Year 1

• An Inception Mission involving all five universities produced detailed Project planning and agreement. • A Results-Based Management (RBM) training and workshop session was held with resultant strengtheningof results statements, indicators and the Performance Measurement Framework;• PI Appraisals were initiated and resultant Capacity Building Strategies nearly completed.• Significant progress was made in 2000 Network Building with agreement on the CARA “Centres ofExcellence” model;• The CARA website (www.caragua.org) was designed and initiated; and• Development education activities at UC were initiated.

Year 2

• The UCR, UNAN and USAC Capacity Building Strategies were completed, reviewed and approved by the CARASteering Committee;

• The new UNAN and USAC MSc. Programs were designed and approved internally at each respective institution;• UCR began Cycle IV of its MSc program (Masters in Water Resource Management and Hydrogeology) with 8

students from Guatemala, Nicaragua and Costa Rica;• UNAN began Cycle I of its MSc program (Masters in Water Science) with 10 students from Nicaragua;• USAC began Cycle I of its MSc program (Masters in Planning and Management of Water Resources in

Agronomy) with 5 students from Guatemala;• The first Central American CARA Coordinator was approved by the CARA Steering Committee (Maxdelio

Herrera from USAC);• Three UC masters students conducted field research in CA;• A UW student received external funding to conduct a feasibility study for offering a distance education course in

hydrogeology at the UCR;Year 3

$ Two UNAN faculty-in-training continued their MSc programs at UC; two UNAN MSc faculty-in-trainingattending UCR are sent home for poor academic performance8;

9 Valeria Delgado and Heydyy Calderon from UNAN; Antonio Padill and Eugenio Orozco from USAC. Ingrid Vargasand Marco Barahona from UCR will begin in 2002.


• The second Central American CARA Coordinator was approved by the CARA Steering Committee. • Three UC masters students completed thesis research in CA.

3.2 Results by Component

Since, and including, the Tier 1 Proposal, this Project has been extremely well-thought-out and carefullycrafted. The proposal shows clear attention to outputs, indicators and assumptions, and an RBM training andworkshop session held with the PIs in the first year introduced them to the management approach and style UC hasfollowed since (and resulted in a redefinition of some expected results and indicators). The continuity in Canadianstaffing through the IDRC phases and into the present CIDA Project has brought with it understanding of andcommitment to regional institutional development in improved water resource management. It has also, however,brought certain expectations that were not necessarily warranted.

3.2.1 Institutional Strengthening (IS)

Institutional strengthening must be seen withinthe context of institutional development at the time ofProject inception. Very simply, UCR already had anMSc program in place; UNAN and USAC did not(though USAC already had some plans to develop sucha program). IS, from the identified output indicators (seebox), is almost completely founded in the existence andquality of the MSc program, and the ability of the hostinstitution to deliver it. The fact that the Project was ableto initiate two new MSc programs within the first fullcalendar year of implementation (the programsphysically admitted students in the last quarter of ProjectYr 2) can be seen as a significant accomplishment, andstrong evidence of early PI commitment. At UNAN, theUniversity provided a building to house the program,which can be seen as a remarkable early indicator ofupper-level support. At UCR and USAC, the programsare housed in existing academic schools.

A second major element of IS is that of faculty-in-training. A total of 4 persons9 is currently in MSc trainingin Canada, at either UC or UW, and at UCR. UCR has been an important resource for such training in previous phases- staff at both UNAN and USAC have completed their MSc there. The only setbacks in this sub-component have allaffected UNAN - a staff member sent to Canada had fewer English-language skills than had been believed, andwithdrew, and two MSc scholarship students sent to UCR did not have the necessary background to sustain theiracademic records.

This phase is still on-going. The first trainees will return to USAC from UCR during Yr 4. The Project wiselyconverted Ph.D traineeships into MSc traineeships, allowing graduate training programs to be completed in shorterperiods, and staff re-integrated into their institutions within the Project timeframe.

Since the first cycle at UCR, this program has had a strong regional (and extra-regional) representation inits student intake. In comparison, the UNAN and USAC first cycles are wholly nationally focussed. UCR is an

Output Indicators

• Congruence of the scope and quality of curriculawith that delivered by analogous institutions

• Ratio of number of courses taught by PI facultyversus Canadian faculty at each PI

• Ratio of women to men on faculty at each PI• Number of new courses; in particular: content,

type, means of delivery and length• Congruency between course offerings and a) the

number of faculty, b) the skill sets resident in thefaculty to teach the material, and c) availability ofnecessary resources (e.g. offices, classrooms, fieldequipment, laboratories, etc.)

• The number and type (formal, informal) ofopportunities being taken by CARA to influencePI planning and budgetary decisions that arebeing pursued by MSc program faculty.

10 In fact, one of these two staff members, in particular, has shouldered the main teaching load. This is perhaps anoutcome of his coordinating the MSc program, though is not the best indicator of developing capacity, as he will have to shedsome courses when other staff return from training.

11 There have been some changes between every cycle at UCR, but with a constant core reflected in the Cycle IV list.


important trainer of faculty - four of the eight students admitted to the Cycle 4 program were faculty-in-training. Toa degree, UCR has been selflessly non-partisan in its student admissions, and in overall terms, UCR will have beena net contributor to achieving Project objectives. There is some question at UCR over the allocation of funding forscholarships: this is dealt with under section 3.2.3.

PI MSc curricula have closely followed the pattern established in the IDRC-funded phases, with somespecialization at each of the three institutions, depending on their hydrogeology, water quality or agronomy focus.Programs include a combination of full credit and short courses. Each institution has responded to lack of academiccapacity in different ways: USAC in particular has tapped Mexican and Czech institutions for visiting professors;UNAN has been the most dependent on Canadian teachers. Students at UCR note that two staff members provide thecore of program teaching, though from the course brochure they expected a much wider range of EGAC staffinvolvement10.

The full curriculum at each institution is presented in Table 1. Elements added to Cycle 4 at UCR areindicated in bold type11. Comparison with the model Canadian (UW) curriculum is not directly possible, as curriculain a Canadian program tend to be tailored to student needs - the Earth Science course descriptions at UW, for example,show a much wider range of possible courses, with hydrogeology being only one of many possible specializations.However, the broad similarity of the curricula between the three PIs is evident, with the agronomy focus at USACstanding out against the hydrogeology focus at UCR. The second trimester courses in particular at UNAN show thewater quality focus at this institution. The consistency between and focus of each curriculum indicates that each PIoffers a well-founded, technical program, relevant to the issues of regional water management.

Table 1. MSc Program Curriculum at each of the Three Participating Institutions

Trimester/Semester

UCR* Cycle IV UNAN Cycle I USAC Cycle I

First Flow in Porous & FracturedMediaHydrologyAqueous GeochemistryIsotope Hydrology**

Water in Science & SocietyHydrologyHydrogeologyLimnology

HydraulicsHydrologyHydrogeologyApplied Statistics

Second Contaminant HydrogeologyGeophysical ProspectingField Methods**Research Project I

Aqueous GeochemistryEcotoxicologyContamination of AquaticResources & BioremediationParticipatory Development andRural Water ProjectsGeographic Information Systemsand Remote SensingResearch Project IField Methods**

Soil-Plant-Water RelationsWater Quality (Hydrochemistry)Subsoil Water EvaluationEconomics of Water ResourcesResearch Seminar I


Third Research Project II Research Project IIContaminant Hydrogeology**

Agricultural DrainageIntegrated Planning andDevelopment of Water ResourcesSoil and Water Management &ConservationFormulating and EvaluatingProjectsResearch Seminar II

Fourth Management & Development ofSubterranean WaterEnvironmental Legislation &Socioeconomic and InstitutionalAspects**Subterranean Water ModellingResearch Project III

Water EngineeringSocio-Economics Applied toWater ResourcesEnvironmental Impact StudiesEnvironmental LegislationResearch Project III: DataAnalysis

Design & Operation of GravityIrrigation SystemsDesign & Operation of SprinklerIrrigation SystemsThesis Research

Fifth Environmental Impacts of WaterProjectsWatershed ManagementResearch Project IV

Watershed ManagementThesis CompletionThesis Defence

Elective Courses:> Fertirrigation> GISThesis Research

Sixth Thesis CompletionSources: Capacity Building Strategies.* Courses in bold represent new courses in Cycle 4 compared to Cycle 3.** Intensive short courses

3.2.1.1 Gender Representation

The high representation of women in the student body of the programs at UCR (historically) and UNAN(presently), will be noted below. Sixty percent of the current faculty-in-training are women; women are involved inthe teaching activities at both UCR and UNAN; CARA coordinators at both UCR and UNAN are women. The onlyinstitution where women are under-represented is USAC - one women student (but an USAC faculty member), andone woman listed (Ms Carias) out of 24 professors on the MSc program staff listing (however, she does not appear asa thesis supervisor and is not a member of the CARA Commission). This under-representation at USAC has been notedby both the USAC Coordinator and UC. However, no significant change appears likely within Project lifetime, giventhat the only male faculty-in-training are both from USAC. USAC will only be able to redress the imbalance throughthe student intake - the difficulties the single woman student at USAC has had in full-time studies (as she is now part-time) may induce resistance at USAC to increasing female representation. This may have to be addressed by assigningtwo of USAC’s four scholarships exclusively to women.

3.2.2 Network Building

A network is an implied (formal or informal) association of institutions and/or individuals to whommembership confers some advantage of synergy, information or other benefit. A network only exists when its membersactively pursue (including some of the financing of) that association for the benefits they perceive to result. At present,the nature of the CARA ‘network’ falls significantly short of this definition. The principle reason for this is that eachof the three PIs is almost fully preoccupied with its own program; what interaction occurs tends to do so at the urgingor behest of UC.

The Tier 1 Proposal notes:

“[It] does NOT involve the creation of yet another water management network that duplicates the efforts ofexisting networks (e.g. Inter-American Water Resources Network.........). [It] is focussed on the strengthening

12 Tier 1 Proposal, p4.


of Central American academic institutions in the area of hydrogeology and water resource management. Thebuilding of a network is essential to connect these academic institutions to the public and private sector waterprofessionals and communities they serve and to coordinate their training activities with related activities ofother networks and international agencies.”

In other words, the vision of the network is oneof “academic training programs not operating inisolation of the broader communities they serve........”12.While some of the output indicators chosen for thiscomponent do speak to external linkages, the majoritystill focus on academic issues and outputs.

The network has, to date, been largely drivenby UC. The Project Manager assumed the role ofCoordinator in Yr 1. In Yr 2, a staff member at USACwas elected CA Coordinator at the Steering CommitteeMeeting; the role has now just passed to a UNAN staffmember. The USAC ex-CA Coordinator notes that forthe PIs, the network had no firm purpose, as each wasstill immersed in starting up MSc programs. It was hisimpression that there were no resources assigned to theCA Coordinator (UC notes $288,000 assigned to theNetwork), and no work plan. Lack of an efficientinternet connection at USAC was a further barrier.

A functional network requires a critical massof people with time and resources to look outwards. Abird’s-eye view of the three PIs at this moment suggeststhat the core consists of two people at UCR, a nowunknown number (but one appointed) at UNAN, andperhaps four at USAC. The Project Manager believesthat the future of the network lies in the current crop ofstudents (both regional and in Canada), but thissuggests that the original core has not the dynamic tolay the foundation. To the extent that UCR has not met UC’s expectations in leading a regional vision, this may be thecase, but many of the current PI staff are young graduates themselves, so are evidently stakeholders in whateverregional body develops.

The interest expressed by institutions in other countries in joining the network cannot be separated at thismoment from an expectation that this will then result in access to CARA funding. The presence of Canada as thefinancial umbrella complicates the development of a truly democratic, self-sustaining network addressing needs asperceived by the members, though this may be more the consequence of UC trying to kick-start a network.

At issue here is the empowerment of the members. Currently there is correspondence between the ProjectManager and the new CA Coordinator on a set of terms of reference for her position (see Appendix 3). As may beexpected, this is a very Project-oriented function. However, if the Coordinator is to respond to the needs of theinstitutions, her/his role and function should be defined by them. But then, more broadly, this role and function wouldrespond to the roles of those institutions in the broader dynamic of regional water-resource management.

Output Indicators

• Extent to which CARA publicity conveys theintegrated nature of each PI’s course offerings

• Ratio of host country to external students (m,f)attending PI short courses

• Number of universities that a) express interest injoining CARA and b) become members

• Number of other CA public/private sectororganizations sending students to MSc programsor short courses

• Number of CA public/private sector organizationsor international funding agencies collaboratingwith PI short courses or field projects

• Number of non-CIDA scholarships awarded tostudents in PI MSc programs and short courses

• Estimated number of person days spent by one PIproviding technical assistance to another

• Degree of coordination in the scheduling of MScprograms across PIs

• Recognition of academic credits between PIs• Extent to which PIs disseminate information

about CARA activities, training programs andresults of research to a wider audience throughsuch channels as the CARA website, journalpublications, conferences, public presentations,etc.

• Ratio of thesis projects that are published injournals and/or presented at conferences

13 The themes are: protected areas, forests, biodiversity, environmental impact, legislation and climate change. PlanAmbiental de la Region Centroamericana, CCAD-SICA. Undated.

14 Instead of CESUCA, as has been proposed for academic reasons.


One of the difficulties in justifying the present network (to the extent that it exists) is the degree to which thePIs consider that benefits accrue from internal (i.e. regional) linkages. At least one PI indicated: ‘we need Canada morethan we need each other’ - a reference to UC’s and UW’s renowned expertise in hydrogeology, and not specifically toCIDA funding. To a degree, PIs do not separate the network concept from the straightforward technical assistance roleprovided by the Canadian institutions.

It is suggested here that the Tier 1 note is closer to the truth than is comfortable. The Project is in danger ofcreating ‘another’ network, at a time when it is more important to work towards integration (in many senses, e.g. thatof academic institutions in regional problem-solving).

In the Year 4 Draft Workplan, the Project Manager notes: Canadians must be careful not to take too stronga lead role in the Network. However, the network component of same Workplan is largely a set of activities that willhave to be driven by UC for them to be achieved. In the same Workplan it is noted: ‘The Network should link stronglyto other regional water management programs such as Comite Centroamericana de Ambiente y Desarrollo (CCAD)’.This is an important statement.

It is suggested here that as the network exists principally in name alone, a better statement in the workplanwould be that ‘the PIs link strongly to.......... such as CCAD.’ Exploring CCAD further, water is one of the strategicareas of CCAD, and CCAD has its own Central American Program for Integrated Management of Water Resources(possibly the one referred to in the Tier 1 Proposal). Surprisingly, given this strategic area, the CCAD does not yetappear to have a Comite Tecnico with direct focus on water resources13. Would it not be better, through the Project,to foster the network within CCAD14, rather than link it to that body? In such a case, the motivation is provided by thestrategic area or focus, and the Coordinator could serve a much more fundamental role in regional institution building.While this may not have the academic separation that was important to UC, it is suggested that in terms of the Projectgoal it would be a far more effective approach.

Networking is also important in the context ofCIDA regional programming. All three Posts reportedthat they had heard nothing of the Project betweeninception and the February 2002 meetings, when CIDAstaff accompanied the Project Manager. It isrecommended that the Project Manager visit the Post forupdates and interchanges on each substantive countryvisit. CIDA has specific and broader interest in CCAD,so any initiatives with this organization should beundertaken in the context of CIDA’s otherprogramming.

3.2.3 Training

Training is the core of the Project, as the otherPI-focused initiatives are intended to ensure theexistence of high-quality resources for training to theMSc level. Short-course events are intended to reach awider audience, though the MSc students remain the

Output Indicators

• Number of applicants to PI MSc programs andshort courses

• Ratio of candidates to the total number of spacesavailable

• Ratio of the total number of candidates (m,f) thatmeet or exceed entry criteria, broken out forreason for applying

• Ratio of completing students (m,f) to thoseregistered

• Level of student (m,f) satisfaction with thecontent and instruction in MSc program and inshort courses

• Responsiveness of PIs to feedback obtained instudent course evaluations

• Proportion of MSc students (m,f) who graduatewith..........

• Proportion of short-course graduates (m,f) whocan describe how they will apply new knowledgeand skills in their work

15 Underlining is from the workplan.

16 In the case of USAC there are fewer doubts.

17 This was not an issue that the evaluator could substantiate. However, the interviewee noted very specifically thelack of maths preparation in the UNAN student body. This same person noted that a high maths level is required at UCR, andthat USAC has a rigorous exam-based selection process.


overriding target of short courses also.

Project documentation notes that eight, 10 and five students were admitted to the 2001/02 MSc cycle at UCR,UNAN and USAC, respectively. At the time of the evaluation, there were six, 10 and five students in these sameprograms, respectively. The reduction in numbers at UCR was a result of the poor academic performance of two UNANfaculty-in-training; the reduction at UNAN appears to have been the result of dropping out. Of the students that remain,one, eight and one students are women, respectively. Three of UCR’s six students are Guatemalan, and the remainingthree Costa Rican. UNAN and USAC currently only admit their own nationals. Non-nationals have been a significantpercentage of UCR’s students since the first IDRC-funded cycle.

Each PI was allocated scholarships for the 2001/02 cycle. There is some confusion, however, over theconceptual allocation of these scholarships within the Project budgetary framework. UCR was “allocated” sixscholarships, and UNAN and USAC three each. However, four of the UCR scholarships were actually faculty-in-training expenses (deductible under Institutional Strengthening), two each from UNAN and USAC. From thisperspective UNAN and USAC received five scholarships each, and UCR only two.

The Yr 3 workplan notes that ‘to ensure the UCR MSc program continues to benefit the region15, insubsequent phases (i.e. Phase V, Phase VI, etc.) four out of six scholarships for the UCR MSc program will continueto be dedicated to non-Costa Rican students.” The workplan goes on to indicate that UNAN and USAC will continueto receive three scholarships each in subsequent cycles. This has been modified in the draft Yr 4 workplan, where eachPI will receive an equal number of scholarships (four, since UCR is no longer required to provide faculty training toother member institutions), and two of each four will be dedicated to non-citizens of the host institution. This latteris intended to encourage the programs to develop international reputations, relevance and standards.

It is quite clear that at this point UNAN should not be contemplating hosting students from other countries16.UNAN needs to sort out coordination and staff issues which are unlikely to be fully resolved in the short-term, andwhich really should be tested in a situation of stability for a full cycle before contemplating the added complexity offoreign students. Another note of concern relates to a reported reduction in content of the more difficult subjects inorder to cope with the lack of background preparation in some of the UNAN students17. Institutional integrity isimportant here - UNAN can not afford to be in a position where it is seen to favour poorly prepared local students overmore rigorously selected or highly qualified international students. In the next cycle at least, all four scholarshipsshould go to local students, but a rigorous selection process should be applied to all applicants.

If there continues to be the emphasis on ensuring that the UCR MSc program continue to benefit the region,it is suggested that there is no reason why UCR should be limited to the same number of scholarships as USAC andUNAN. In the first instance, it may be suggested that it would be more equitable to restore the imbalance incurredthrough the faculty-in-training contribution of UCR (i.e. three more scholarships), and then allocate at least one morescholarship per year to UCR than to UNAN and USAC precisely in order to support this important service function.This regional emphasis castigates Costa Rican students, as they will have to compete at an international level in orderto gain access to their national Centre of Excellence.

The PIs note that they subdivide the scholarships for certain reasons: in Costa Rica, UCR is unable to pay thefull maintenance amount (which exceeds a assistant professor’s salary) because of local tax regulations (CR students

18 This is unlikely to change in the next cycle. However, short courses are a useful way for students to interact withother experts, and the practice should not necessarily disappear. It has been suggested that short courses should be absorbedinto the main courses; this may not resolve the issue of expertise. Students note that short-course consultants often leaveinsufficient time for interaction.


report that they receive about 50% of the amount paid to USAC/UNAN faculty-in-training); USAC supports fourstudents on three scholarships; UNAN has passed through several steps of a) offering the maintenance amount as agrant, then b) as a loan, and then c) a reduced amount as a loan. While it is not immediately suggested here that theProject should have different funding levels by country, the dissatisfaction in the student body over an apparentinequitable management of stipends, especially at UNAN, suggests that this is an issue that should be examined indepth by the Steering Committee at its next meeting. Part of the issue at UNAN relates to UNAN’s underestimatingresearch costs and a need to recover as much as it can for 10 students out of the scholarship funding for three. UC needsto be clear on an issue of policy: to whom is it providing the scholarship - the student or the institution?

The students were quite specific in indicating that the lack of practical content in many courses is a constraintto understanding and applying the theory. This is not a criticism specific to any single PI. UCR is attempting toimprove practical content through the establishment of a permanent field station linked to its Faculty of Agronomy;the Dean of the Faculty of Agronomy at USAC indicated his intent to establish a similar facility. Whether or not thesetwo initiatives go ahead, neither one is likely to benefit even the next round of students. Some of the deficiency willbe met at UCR through the Hydrogeology Field Methods course (equally as important at the other two Pis - it wastaught at UNAN in Yr 1, for a combined UNAN/USAC student audience), though each discipline will have itsparticular field practices, and each Program should ensure that lecturers include field practice for each course.

UC has, on occasion, contracted short-term consultants to fill in for or support PI staff who were unavailableto give a course. It has also been a UC principle that PI staff should attend these courses in order to update their ownlearning. UCR notes that the latter principle at least requires that the visiting consultant be an expert in his or her ownfield, but also assumes that ECAG staff have the time to spare. In the case of Jenny Collins, who filled in for the UCRCARA Coordinator who was on maternity leave, the general impression among staff and students was of a very juniorperson with limited language skills. She received a poor course evaluation from students, as she was insufficientlyprepared. Another consultant requested UCR’s course notes prior to arrival.

The programs at each PI still depend on short courses to deliver critical material18. UC must ensure thatconsultants contracted be credible and well prepared. This may result in a higher consultancy costs, but neither IS nortraining goals are served if the person selected either delivers an inadequate product, or is seen to lean on the veryinstitution he or she is supposed to be strengthening. Whether or not PI staff attend a guest lecturer’s course deliveryshould be the decision of the PI. The profile of the person selected as guest lecturer will have a large role to play in thisprocess; UC should recognize that if it wishes trainers to reach staff as well as students, it may do this better throughdialogue on research, and stimulating personal interests, rather than forcing a classroom approach.

If the current two-year immediate follow-on cycle continues, the calendar sequence of future MSc programcycles will be as follows:

CIDA FY / 00-01 / 01-02 / 02-03 / 03-04 / 04-05 / 05-06 /

Cycle: / I / II / III?

This implies that current end-of-project will fall in the first semester of Cycle III (or VI at UCR). By at leastthe end of calendar year 2003, the Project Steering Committee will have to review Project expenditures to determinewhether sufficient funds will remain to support further scholarships in Cycle III, and whether a recommendation tothis effect should be made to CIDA. This does not necessarily imply a Project extension - these are funds that couldbe transferred to the PIs for administration.

19 See Bibliography. The two Nicaraguan faculty-in-training who left the UCR had submitted proposals; these wereincluded in the analysis. UC and UCR could usefully review the extent to which theses from previous cycles at UCR have ledto policy development, engagement of stakeholders, etc.

20 Clearly there is also a role here for thesis advisers.


The Project has decided to drop the concept of a distance education course in hydrogeology because the PIsare unable to deliver it. The funds will be reallocated to the preparation and publishing of all curricula, and the noteswill be posted on the CARA website. It is suggested that the steps being proposed are exactly those that the Projectwould have to follow in any case, were it to go ahead with the distance education hydrogeology course, and that theSteering Committee should re-examine the possibility of delivering this output, beginning in Yr 5. If the hydrogeologycourse were the first to be published, the Project could then consider what subsequent steps are required to activate adistance learning course. It would be most logical to implement it from UCR; the Project could consider funding a staffmember at UCR to lead the process.

3.2.4 Applied Thesis Projects

The output expected of this component wasidentified above. However, it is the original LFA whichis clearer conceptually about the expectations:

• a comprehensive and tested participatorydevelopment methodology adapted to waterissues;

• students/faculty better prepared to engagecommunities and local authorities in problemsolving;

• research results/recommendations to key decision-makers for action;• case studies of how users and professionals can work together to solve problems.

The Project is in the first cycle of thesis projects, though with field studies little advanced. As a result, theextent to which these projects have the stakeholder focus, at both community and institutional levels, can only bedetermined by an analysis of the 22 research proposals19. Using the output indicator (see box), intended engagementof stakeholder is evident in less than 50% of the proposals (to what extent engagement occurred in proposal preparationis unclear, though the majority of proposals do not show evidence of any consultation); participatory developmentmethodology is completely lacking. Proposals show little intent of feeding results back to interested parties.

The short course on Participatory Development and Water Projects was delivered at USAC and UNAN inSeptember and November, 2001, respectively, largely in time for some of the concepts to have found their way intoresearch proposals. It may be delivered at UCR in 2002. However, the Project is still occupied in producing a Spanish-language field manual on participatory development methods, which may close the gap for students who are unableto translate concepts into practical methods20. It will be the comparison of the current and next round of researchproposals which will tell whether progress is being made on participatory research.

Quite apart from the context of stakeholder involvement, the issue of thesis projects is an important one fromthe point of view of students graduating in 2002. This is particularly relevant in Nicaragua, and to a lesser extent inGuatemala. It does not appear to be an issue for full-time students in Costa Rica. The value of part-time studies isinvestigated more thoroughly in section 5.4.1.

Two key persons were lost to the Nicaraguan program (one possibly temporarily, one permanently) in 2002.As a result, the thesis program at UNAN has been significantly delayed. The Project has made arrangements to bring

Output Indicator

• Congruence between stakeholder and studentappraisal of the student’s engagement ofstakeholder; with respect to: problemidentification, participation in planning and fieldactivities, knowledge of research results,relevance of results to stakeholders.


two Costa Rican consultants to Nicaragua to cover as much of the hydrogeological thesis supervision as possible. Thepermanent layoff was especially critical, as this person (the first woman graduate of UCR’s MSc program) was the day-to-day contact for all students. Her layoff was alleged to be as a result of shortage of funds. However, given the criticalneed for her expertise, and the emergency support that could have been expected of CARA, her loss speaks more toissues of institutional commitment to the program than anything else.

In Guatemala, internal methods of financial management, including lack of budgetary supervision by theCoordinator, resulted in a critical shortfall in funds available to students just at the moment when they were expectingto go to the field. Funds were received by USAC in March, but still had not been disbursed at the time of the evaluation.As the research trimester has now passed, students were unsure whether they could complete a full project within theremainder of the calendar year.

The Project intends holding a congreso in early 2003 to showcase the results of the MSc programs to theregion. The UNAN Director makes the point that the thesis reports are the tangible product which will be on display.However, if two-thirds of the PIs experience late submission of theses, this event is doubtful. The issue of a thesis-basedMSc is pursued further in section 5.4.1.

While conceptually it is perhaps an element of both the training program and development education, CIstudent activity in CA is grouped under applied research projects. Three CI students have completed MSc research inCA (at UCR), and four more will be undertaking similar research in Yr 4 (two of them in Mexico). UCR is verysupportive of this linkage, but notes that Canadian students come with high expectations of the PI, and that they donot always respect the collegiality which the Project’s partnership implies. Short notices of arrival, lack of pre-departure reports, and non-inclusion of PI staff as thesis-board members, were some of the comments made. It isincumbent upon the CIs to ensure that Canadian students are sensitive to issues of partnership before departure, andthat arrangements for Canadian students stretch to full consultation on PI collaboration on thesis work, andexpectations of debriefing seminars and reports prior to return to Canada.

3.2.5 Development Education

This component has yet to receive majorattention. Apart from minor activities, such as websitedevelopment and poster displays (the latter has beenused at campus events related to international fora), themain activity was participation in a CIDA sponsoreddocumentary on various CA projects prepared byCanadian Learning Television. This video is reported tohave been aired several times.

The Project intends to focus in Yr 4 on thedevelopment of a secondary school geographycurriculum on water issues, to compare Canadian andCA water issues.

3.2.6 Private Sector Involvement

Private sector involvement has been limited sofar to three companies either providing short-courses orinternships. The outcome of these contacts is not directlytangible, though one person noted that there have beensome software sales in Central America, and, indirectly,some business consultancies in South America (possiblystemming from student contacts made during earlier,

Output Indicators

• Number of CARA related articles appearing inuniversity newspapers

• Number of on-campus locations where CARAposters are placed

• Number of public presentations relating to CARAactivities

• Number of events where the CARA poster displayis presented and staffed

• Number of secondary school students exposed toCA water issues through CARA

Output Indicator

• Number of Canadian firms that have enhancedbusiness development in CA due to involvementin CARA; the proportion of those assisted whohave a) pursued and b) secured specific contracts

21 And internal forwarding at UCR.

22 UCR has now assigned him to other non-Project duties within ECAG; as this staff member was involved inteaching three MSc courses, this may impact on the delivery of Cycle 5.


IDRC-phase, courses, though more probably because of a world-wide reputation). This involvement, and that of theinternship, remain the only private-sector backup to the Project at large, with a net inflow of benefits to the PIs (e.g.the latter have received free software licences). While Canadian hydrogeology expertise could undoubtedly enhanceCA water-resource management, it does not yet appear to be a field of significant business opportunity.

One of the Canadian private-sector partners (Komex) was contracted to deliver a short-course at UCR at thebeginning of Yr 4. The identified UCR counterpart professor had already undertaken a three-month internship withthis company in 2001. As a result of the transmission to UCR21 of the Komex evaluation of this intern’s performance,the latter has withdrawn from the Project.22. UCR has now assigned an alternative counterpart, though at the time ofthe evaluation, UCR noted that UC had indicated that because the agreement involved the ex-intern as counterpart,it could not now fund the short-course at UCR. UC is contemplating its relocation to UNAN. UC also notes that thecourse did not proceed at UCR because of overly-late confirmation of dates.

3.2.7 Internationalization of Canadian Institutions

The output indicators for this component (seebox) speak to a more confined vision of the componentthan the output (/outcome) statement in section 3.2. Thedifficulty with the concept of internationalization offaculty and students is the intangibility of knowledgeunless, and somehow, it is demonstrated exactly. Thereis no doubt that both faculty and students have a broaderunderstanding of CA water issues as a result ofexchanges; to what extent this is significant in theoverall context of course content could not bedetermined. From discussions with CI staff, it appearsmore likely that graduate students are the main contributors, as they evolve, investigate and present, research findings.However, at least one CI professor indicated the use of CA case material in his teaching.

The activity focus of the workplan results in the majority of this component being conceptualized under IS,Training and Applied Thesis Projects, e.g. Yr 4 draft workplan. There is a need to revisit the purpose of thiscomponent, and redefine the expected results. These should be seen as separate from the crossover with IS, etc., eventhough it is the learning from these activities which provide the catalyst for Internationalization. Internationalizationis a strategic priority for UC, but the linkages are not necessarily in place to ensure this strengthens UC’s teaching andadvancement processes (it is only recently that some Faculties have awarded merit points for international involvement- until this occurs more broadly, Project-related internationalization at UC, and to a lesser extent, at UW, may not besustainable).

4 Project Management & Administration

4.1 The Project Manager

It was noted earlier that this Project was carefully crafted. The Project Manager has a management stylewhich shows a close attention to detail - the framework of outputs, outcomes and indicators is evidence of the thoughtand care which has gone into design. The value of the framework is seen in the separation of activities, though it hasalso been noted that there is some overlap between outputs and outcomes which may result in some confusion in

Output Indicators

• Extent to which CI faculty involved in CARAhave enriched their courses with Project relatedinformation

• Number of additional international projectsapplied for as a result of CARA

• Increase in number of applicants from CA forstudy at CIs


interpreting performance. However, this is a minor issue. The indicator-based approach has been extended to the PIs,where the Assistant Project Manager has undertaken participatory monitoring against the Capacity Building Strategies(CBSs). PIs are supposed to report to the Project Manager on progress in delivering on their CBSs. While they areperhaps less than conscientious in doing so, UC reporting to CIDA is on time, and uses the same performancemeasurement framework - as a result, reports are easy to follow and progress towards outcomes understood. Thecomprehensiveness of the framework allows a clear picture of progress towards Project goal.

The Project Manager has extremely high expectations of himself, and of the PIs, in achieving Projectoutcomes. It is clear that the changes which occurred at UCR at Project Inception were a disappointment to him, andthat as a consequence he assumed a greater personal load. While no doubt the changes at UCR were unfortunate, inthe context of the vision that had been developed, to a very great extent they were necessary if the MSc program wasto become properly grounded in ECAG. Equally unfortunate, however, is the extension of the Project Manager’s senseof disappointment to his apparent treatment of his immediate collaborators at UCR.

Early in the evaluation, UC made it clear thatan option for the future was to drop UCR from theProject if UCR didn’t make the necessary improvementsin institutional support; early in the visit to UCR, staffmade it clear that they resented the way in which theProject Manager made his demands. Where such mutualtensions exist, it becomes extremely difficult to makeprogress on any issue. The situation at UCR is as mucha consequence of leadership style in the previous IDRC-funded phases as it is an outcome of subsequentinstitutional support (or perceived lack of). During theevaluation, senior UCR staff indicated a) that UCR istaking steps to overcome the lack of permanenceassociated with annual contracts, and b) that thecoordination of all CARA activities would shortly revertto a single individual. These issues have been at the coreof UC demands; it is now up to the Project Manager torebuild bridges at UCR. Part of the latter will requirethat he see that UCR has made a significant contributionto IS of the other PIs, and that it is still an institutionwith a regional commitment. He should refrain frominvolving himself directly in internal UCR managementissues, and allow UCR the flexibility important tovaluing CARA within the framework of the otherprojects of which UCR is the beneficiary. The issue ofthe Komex course is one where he could have shownmore flexibility.

An extension of this issue is the manner inwhich UC, and the Project Manager in particular,disengage from the Project in the final years. Thecurrent crisis at UNAN, which is being solved withProject-funded consultants from Costa Rica (one ofwhom is the ex-UCR coordinator from the IDRCperiod), has the potential to spill significantly over into the next cycle of students. This may take up an excessiveamount of the Project Manager’s (and other UC/UW staff’s) time in providing short-term solutions to what isfundamentally an institutional issue (the role of current faculty-in-training will only become evident once they havereturned - future promises of institutional support must be balanced against the recent dismissal of the key UCR-trained

While it is nowhere explicit in the Tier 1Proposal, there appear to have been clear expectations ofregional leadership from UCR in both institutionalstrengthening and network development. Theseexpectations surfaced on several occasions during theCanadian portion of the University visits, especially indiscussion with staff who had participated in the IDRC-funded phases. The only clear indicator in the Tier 1Proposal is the identification of the UCR professor who wasto be UCR Team Leader and Central American Coordinator- again, anecdotal description defines much of the regionalvision as having been his.

This person retired at the moment the CARAProject began, allegedly as an outcome of CIDA notcontinuing the salary top-ups paid under the IDRC phase.He was also apparently instrumental in having hiscoordinating role at UCR split between two successors -whatever his purpose, in the long run it has had the effectof neutering ECAG, both internally and regionally. UCR’sown interest in bringing the MSc program back into theECAG mainstream, from its more personalized previousposition, has also resulted in normal academic politicsinfluencing decision-making (or the lack of).

These several factors have resulted in UCRfalling to the bottom of the pile in terms of UC favour. Thisposition has not been lost on UCR, which, at more than onelevel, complained about Canadian attitudes. UCR notes thatthe CIDA phase has more checks and balances than did theIDRC-funded phases, and that UC should not wish tocontinue earlier ‘free-wheeling’ ways. The resentment ofUC found at UCR during the evaluation stems from UCpressures on UCR (in an on-going recriminatoryatmosphere) to make the institutional changes UC views asnecessary.


hydrogeologist at UNAN) . It is important at this time for the Project Manager to step back from some of the internalissues, and force (allow) the PIs to manage them themselves. He should spend Yr 4 fostering the strategic mechanismsimportant to the broader outcome of the Project.

4.2 Project Committees

The Project is served by two committees: an Advisory Committee at UC which guides the Project Manageron matters of policy and hiring, and the Steering Committee (Comite Directivo), comprised of the three PIs, UC andUW, which makes substantive decisions on Project implementation. The UC Advisory Committee meets at few-monthlyintervals; the Steering Committee has been an annual event, though the frequency may be made semi-annual.

The main value of the Advisory Committee during 2002 will be to support the Project Manager in a) resolvingissues at UNAN, and b) the disengagement strategy. The Steering Committee will have to address issues ofsustainability, especially the form and purpose of the network in supporting inter-institutional collaboration on the MScprograms, and contribution to regional policy making on water management issues.

4.3 Budget

The Project budget, the original according to the Grant Agreement, and that actually administered, is indicatedin Table 2. Variances indicated are calculated against expenditures to 30 September 2001. The UPCD contribution is$3,995,766. At 30 September 2001, the Project had disbursed approximately 20% of its budget. Assuming adisbursement rate in the second semester equivalent to the first semester of 2001/02, the Yr 3 budget will still beunderspent by about 35%, and will take overall disbursement up to about 28%. Overall variances are generally uniformbetween Project components, including Management & Administration; in the context of the major components (IS,Training and Applied Field Projects) this is surprising, as one would have expected a greater concentration of relativeexpenditures in these components where principal attention has been paid. UC notes that underexpenditures are acombination of delayed activities, lower activity levels or unreported expenses. At the time of evaluation report writing,UC is implementing a workshop to improve financial reporting by UNAN and USAC; while UCR notes that it has alsobeen criticised for poor financial reporting, it is not attending the workshop, and the review of financial affairs at UCRduring the evaluation suggested that UCR was generally timely in its responses. UCR notes that UC often does notrespond to, or recall, UCR’s transmission of financial data. UC notes that it has never citicized UCR’s financialreporting.

Were annual expenditures to increase to an average of $750,000 (including Yr 3), there would beapproximately $1 m remaining at the start of Yr 6. Means to utilize unspent funds are suggested in subsequent sections.


Table 2. Central American Water ResourceManagement Network (CARA)

Five-Year Budget by Program Component - UPCDContribution

As per Grant Agreement

Yr 1 Yr 2 Yr 3 Yr 4 Yr 5 Yr 6 Totals

IS 3,407 268,039 213,889 139,253 54,840 35,807 715,235

Network 36,543 59,629 54,107 47,812 49,247 29,590 276,928

Training 13,921 126,572 278,561 272,713 280,894 152,621 1,125,282

Appl Thesis Proj 14,889 146,508 159,233 161,201 168,931 66,472 717,234

Development Ed 9,223 23,635 17,978 39,115 17,948 2,738 110,637

PS Involvement 2,578 7,966 8,205 8,451 8,705 5,977 41,882

Intnltn of CIs 26,803 55,216 56,872 58,577 60,335 24,858 282,661

Man & Admin 50,930 84,300 90,693 89,434 96,216 41,302 452,875

M&E 15,315 9,215 20,993 9,776 22,272 5,186 82,757

Subtotal 173,609 781,080 900,531 826,332 759,388 364,551 3,805,491

Cont 8,680 39,054 45,027 41,317 37,969 18,228 190,275

Total 182,289 820,134 945,558 867,649 797,357 382,779 3,995,766

Budget as administered

Actual Budget Variance Budget Variance Budget Budget Budget Totals Variance Var as %

IS 0 145,425 33,266 201,461 124,096 207,092 124,094 34,471 712,543 523,019 73.4%

Network 12,580 44,055 14,780 81,843 64,049 55,870 53,774 39,674 287,796 228,147 79.3%

Training 0 95,544 17,401 302,361 207,537 265,111 272,736 189,529 1,125,281 952,314 84.6%

Appl Thesis Proj 5,737 53,017 5,569 172,732 136,301 174,309 182,207 134,766 722,768 633,152 87.6%

Development Ed 5,157 19,608 11,969 18,939 13,095 41,113 18,963 6,856 110,636 91,996 83.2%

PS Involvement 0 9,441 757 8,205 4,576 8,451 8,705 7,079 41,881 29,568 70.6%

Intnltn of CIs 5,093 35,333 24,160 54,680 24,359 59,370 77,957 36,521 268,954 222,367 82.7%

Man & Admin 16,107 78,662 13,559 90,205 62,766 93,180 95,863 78,858 452,875 344,226 76.0%

M&E 6,989 9,085 621 32,912 28,380 9,776 10,069 13,924 82,755 62,770 75.9%

Subtotal 51,663 490,170 122,082 963,338 665,159 914,272 844,368 541,678 3,805,489 3,087,559 81.1%

Variance (%) 24.9% 69.0%

Variance against orig. budget (%) 29.8% 15.6% 73.9%

23 In the case of UCR, this person disappeared at the start.


The goal of the Project is: "To improvethe management of Central Americanwater resources”

5. Observations and Synthesis of Findings

5.1 Rationale

The water resource management focus of the Project speaks quite clearly to an increasingly pressing needin Central America: knowledge and effective management of water resources in a region of markedly variablephysiography and land use. Water is one of the four strategic areas of the CCAD, under which this body hasdefined three objectives:

• guarantee the protection of water sources;• ensure the long-term supply of water (in quantities and of a quality adequate for multiple uses);• encourage a complete economic valuation of regional water resources.

To date, much of the regional emphasis has been on surface waters. The Project seeks to extendunderstanding to subterranean water resources, where management issues are significantly more complex, andwhere current knowledge is lacking. The Project has focused on three national institutions which play relatedthough slightly different roles in research and training; in developing the regional knowledge base for waterresource management, this synergy promises to broaden the capacity in this area.

5.2 Contribution of Results to Meeting Project Goal

The Project is at a critical point. It has successfully established twoMSc programs at two CA universities, modelled on the one established atUCR (which itself depended on the UW pattern) with the support of thesame group of Canadian scientists. However, what may be termed second-generation issues, which have surfaced at UCR, may surface similarly at UNAN and USAC, and it will be thesuccessful resolution of these issues which will result in meeting the Project goal. Very largely, these issuessurround a successful institutionalization of each program beyond the vision or efforts of individual champions. Itis almost certain that the three key individuals, one at each institution, will have retired before the end of Yr 523.The second generation issues which have, in UC’s eyes, surfaced at and slowed the regional contribution of UCR,have almost certainly surfaced already at UNAN. UC, UW and UCR are training young faculty; and to a greatextent the sustainability of achievements to date depends on the re-integration of this staff into the three programs.However, greater staff numbers alone are not the only factor ensuring institutional commitments to the broaderinitiative at the regional level espoused in the network concept, and necessary for meeting the Project goal;institutions will have to be pro-active in seeking and providing the resources necessary.

At this point it is important to ask: ‘Where should be the main emphasis in the remaining years toincrease the likelihood of achieving the Project goal?’ A ranking has been developed to respond to this question(Table 3). Order of ranking should be read as the relative importance of components for continued investment ofProject resources in terms of contribution of probable outcome to Project goal. This ranking is developed from theevaluator’s overall vision of the state of the Project at this point in time, and takes into account the nature of therecommendations made in this report. The ranking indicates that the balance of investment should favour networkbuilding and training as the two most important, ongoing components; investment in the remaining componentswill, for a variety of reasons, have less effect on goal achievement. Readers may wish to consult the componentoutputs/outcomes in section 2.3 to help interpret these rankings.


Table 3. Relative importance of components for continued investment of Project resources in terms of contribution of probable outcome to Project goal

Project Component Ranking

Institutional Strengthening 3

Network Building 4

Training 4

Applied Thesis Projects 2

Development Education 0

Private Sector Involvement 1

Internationalization of CIs 0

Ranking

0 - No relevance to achieving Project goal1 - Increased emphasis may have slight effect on achieving Project goal2 - Increased emphasis will have a detectable though minor effect on achieving Project goal3 - Increased emphasis will result in significant probability of achieving Project goal4 - Increased emphasis will bring marked likelihood of achieving Project goal

5.3 Achievements of CARA in relation to UPCD Goal & Objectives

At this approximate mid-point in the Project, the progress towards achievement of UPCD objectives issignificant. To give a general indication of progress by UPCD objective, another ranking has been developed. Theresult may be found in Table 4. The Project still has sufficient time and resources to make further progress towardsits overall goal.

Table 4. Assessment of Project progress in achieving CIDA UPCD and ODA Objectives

UPCD Goals & Objectives Ranking

Goal: To increase, in a sustainable manner, the capacity of developing countries toeducate and train the human resources required to meet their priority development needs

4

Obj 1: To strengthen developing country institutions responsible for the training of humanresources

4

Obj 2: To contribute to the establishment of linkages fostering sustainable partnershipsbetween Canadian and developing country institutions

3

Obj 3: To contribute to the internationalization of Canadian Universities in all aspects oftheir mission and to the promotion of Canadian university expertise in higher education

3

Obj 4: To contribute to the development education of the Canadian University and thecommunity it serves

2

ODA Obj: Support for the full participation of women as equal partners in the sustainabledevelopment of their societies

3

24 This sample is too small for general confidence. A wider analysis across other MSc programs would have to bemade to be sure of such a conclusion. However, the extra demands especially and inevitably placed on mature women studentssuggests that this latter category ought to benefit from part-time arrangements.


ODA Obj: A commitment to help developing countries protect their environment andcontribute to addressing global and regional environmental issues

3

ODA Obj: Promote sustained and equitable economic growth by supporting private sectordevelopment in developing countries

1

Ranking

0 - no perceptible progress1 - minor change or progress, but which cannot specifically be attributed to the Program2 - minor change or progress, some of which is attributable to the Program3 - significant change, most of which is attributable to the Program4 - high probability of achieving outcomes

A series of recommendations, in section will support the Project in improving overall achievement ofUPCD objectives.

5.4 Concepts

Many of the recommendations made in this report relate to minor process or effectiveness issues.However, there are two specific areas of the Project where a conceptual evaluation might contribute to improvedrealization of certain outcomes. Both of these would imply a significant shift in the way Project resources areapplied.

5.4.1 Course-only M.Sc.

From the historic UCR data, the risk for women students of not graduating appears a high-level risk(>50%; 20% in males). While an effort has been made to accommodate female students through part-time studies,there is no indication that this has influenced graduation rates24. UCR points out that women do not require aMaster’s degree in order to work as hydrogeologists - five of the women who did not graduate, and who are namedin Table 5 work as consultants and public-sector employees. UCR makes the point that as geology graduates, theyhave already obtained some background in hydrogeology, perforation techniques, and management of waterresources. A hydrogeology MSc on top of a first degree in geology appears not to influence employability (except inthe University context), though would, presumably, improve an individual’s capacity in the field. Lack of a firstdegree in geology would make a Hydrogeology MSc a prerequisite for employment (though if the field is still asunder-defined as it appears, the region would see a Geology MSc also as adequate).

The fact that the five women non-graduates applied to the MSc program speaks clearly to a perceivedneed (either institutional or personal) for improved understanding of the field. It is the issue of the thesis thatdefines the outcome (graduation). One of the current underlying assumptions of the program is that an individualresearch project is essential to turning out a well-rounded hydrogeologist, and that the degree should not be grantedwithout it. However (and as is currently evident in Nicaragua), much time can be lost in defining thesis topics,such that the research does not get under way during the period allocated for it, but later on, when students areback in the classroom. What can then be done will inevitably suffer in terms of scope and quality.

The objective-level outcome relevant to this discussion is:


Table 5. Students, and their Graduation Record, UCR, Phases I-III (1995-2000)

Phase Student Nationality Attendance Graduated

I (1995-96) Jorge Cardenas Gloria Hincapie Mayra Blandino Oscar Cruz Claudia Enamorado

ColombianColombianNicaraguanNicaraguanHonduran

Full-time YesNoNoYesNo

Arturo Rodriguez Federico Arellano Alicia Gomez Sandra Arredondo Ana Lucia Castro

Costa RicanCosta RicanCosta RicanCosta RicanCosta Rican

Part-time YesNoNoNoYes

II (1997-98) Issac Herrera Jose Roberto Duarte Martha Espinoza Brenda Medal Karla Mora

GuatemalanSalvadorianNicaraguanNicaraguanCosta Rican

Full-time YesYesYesNoYes

III (1999-2000) Erick Motta Arturo Trochez Clara Agudelo Juana Rafael Vargas Rodrigo Calvo David Vega

GuatemalanHonduranColombianCosta RicanCosta RicanPanameñan

Full-time YesYesYesYesNoYes

Source: UCR 2002

“Graduates of PI M.Sc. programs ............influence water sector policies, plans and practices of theirrespective countries through solid technical knowledge, participatory development training and experienceat a community level, and the incorporation of a “partnership” ethos in their work.”

If a student has gone through a frustrating process whereby the thesis became just a means to graduationrather than the core of the experience-building (especially in a community), then it may be suggested that insistingon a thesis project added no value to developing technical knowledge. If, for whatever reason, the Project(/institution) cannot guarantee the timely human, financial and physical resources necessary for the thesis projectto be defined in Yr 1, and undertaken in the first months of Yr 2, then the program needs to be modified so thatclass loads are less at other times and the thesis work is undertaken over a longer, less-intensive period.

Alternatively, if the thesis trimester were to be converted into a solid, practical, community-based fieldschool (directed and led by a hydrogeologist, and of which more than one model could be envisaged), whichaddressed issues above and beyond practical classes linked to each of the course subjects, it is quite possible for theexperience gained to exceed what even an effective thesis experience could impart. It would also guarantee that allthose who achieved adequate marks graduated on time. Such an approach has the advantage that it could bedesigned to incorporate all the elements of participatory development and partnership, in response to a perceivedwater-resource need or issue.

Intensive course-only Masters programs (perhaps more common in less practical fields) are becomingrecognized as an efficient means of training. While recognizing that any natural-resource focussed program shouldcontain elements of practical work, it is suggested here that outcome 3, and the Project goal, does not automatically

25 This appears to be the model at UW, where it has been indicated (Steering Committee, 14 Feb 02) that any new PIstudents going to UW for studies would have to incorporate themselves in research fields already under way.

26 Mechanisms of matching funding could also be built in.


require that students undertake an individual research project to be effective water-resource managers. Moreintensive training could result in greater impact than half-hearted research.

Recommendations: That the partners review present program structure to determine whether greaterallowance need to be made for research time during current ‘teaching’ trimesters.

That the partners examine the advantages and disadvantages of replacing the thesistrimester with a community-based field school. Consideration should include benefits interms of expected outcome, alternative human and financial resources required.

5.4.2 Research

CARA is focussed on and structured towards creating conditions for effective delivery of masters’programs. In the research component, financial resources are allocated to students for their projects, which they arevery largely free to define how they wish. Professors supervise this research through their own commitment tohuman resource development. No incentives are provided.

Intellectual advancement is at the core of professorial quality. Apart from the incidental contact betweenprofessor and student, there is nothing in the Project which contributes to a professor’s own research agenda withinthe field of hydrogeology. While professors undoubtedly will have some influence in shaping the ‘what and how’ ofa student’s research activity, it is very much up to the student to push the agenda forward. Clearly, this is not a badthing, though if the student does not complete the thesis no intellectual property is generated.

It is suggested that a joint commitment of professor and student to the research program is a better model,i.e. that the student’s research contributes to the professor’s own research interests and intellectual gain. Bycommitting him/herself to the generation of research outputs, and incorporating the student(s) in that process25, theprofessor places a higher expectation of outcome on the student than currently exists in CARA. This can beachieved by the allocation of research grants, which include student stipends.

Because the institutional strengthening component is so focussed on MSc program delivery, this essentialelement of IS is lacking. However, within the responsibilities of academic staff to their institution lies that ofresearch. CARA is less than holistic in its approach to IS, and while it may be said that current expected outputsare adequate for training outcomes, if achieved, it is suggested that they fall short of the model of a Centre ofExcellence. IS should not stop at training future faculty to a higher postgraduate level. A Centre of Excellencerequires active research agendas which position academic staff at the forefront.

It is noted that the idea of a research fund recently surfaced, and that an amount of $20,000 has beenassigned to each institution for the remainder of the Project. The purpose and mechanisms of this funding havebeen defined and are articulated in correspondence between UC and CIDA, and UC and the PIs. It is recommendedthat this research funding be further considered, and some significant alterations be made to the way in whichperhaps some student support is allocated, i.e. that academic staff in the three PIs develop and put forward researchprojects for CARA funding26 which could be undertaken in the same two-year cycle of the MSc program. In casethe CIs believe that CIDA will not support a stronger research emphasis, the UPCD program recognizes theimportance of research to the broader function of a PI.


The opportunity exists here to establish a creative (and possibly competitive) mechanism which could alsofoster network linkages. It is recommended that a working paper be generated by the CARA Coordinators and UCon a research grant mechanism for circulation before the October Steering Committee meeting. If such a system isto run in parallel with the MSc programs, it will have to be in place by the end of calendar year 2002 (i.e. not onlythe mechanism, but also some approved projects). It is suggested that this mechanism does not need to be complex- adjustments can be made for future cycles, if necessary.

This discussion is independent of the preceding section (5.4.1), but offers an alternative approach tostrengthen the emphasis on quality thesis research.

6. List of Lessons Learned

The following are the principal higher-level lessons learned from the Project to date. Lower-level lessonsare implicit in the subsequent recommendations:

1. Topping up salaries from donor funds is detrimental to longer-term individual commitment and collegialharmony.

2. The sustainability of an MSc program requires at least three core staff members with similar disciplinarybackground; in a thesis-based program, each core staff member can probably manage no more than twostudent projects effectively

3. Champions serve program development well; whether they serve the institutionalization of that sameprogram equally well depends on their collegial vision.

4. A network only exists when its members actively pursue (including some of the financing of) thatassociation for the benefits they perceive to result.

5. The continuity in Canadian staffing through the IDRC phases and into the present CIDA Project, becauseof their understanding of and commitment to regional institutional development in improved waterresource management, has resulted in significant early progress in the Project. However, CIDA and IDRChave distinct modi operandi, including in their approaches to project management.

6. A Centres of Excellence model should integrate research support for the very staff upon whom high-quality teaching depends.

7. Conclusions and Recommendations

The evaluation has found that the CARA Project has made significant progress towards achieving its goal.The high commitment of the Canadian institutions has been noted, as has the overall sense of partnership betweenthe Canadian and Central American Institutions. The evaluation has noted that there are several areas which couldbe strengthened, include conceptual approaches to Institutional Strengthening, Network Building and Training.The following is the set of recommendations made to enhance the probability of achieving the Project goal.

Women’s Participation

In Project lifetime, USAC will only be able to redress its gender imbalance through the student intake.Two of USAC’s four next-cycle scholarships should be directed exclusively to women.

Network Building

That the Project foster the network within CCAD. With motivation provided by CCAD’s strategic focuson water as one of four areas, the Coordinator could serve a much more fundamental role in regionalinstitution building.


That the Project Manager visit the Post for updates and interchanges on each substantive country visit.CIDA has, for instance, specific and broader interest in CCAD, and any initiatives with this organizationcould benefit from the context of CIDA’s other programming.

Training

That UNAN should not contemplate hosting students from other countries until it has resolved the severalinstitutional constraints that currently affect the training program.

That, in the next UNAN cycle at least, all four scholarships should go to local students, and a rigorousselection process be applied to all applicants..

That the Project restore the imbalance incurred through the faculty-in-training contribution of UCRthrough the awarding of three scholarships, and then allocate at least one more scholarship per year toUCR than to UNAN and USAC in order to support this important regional service function

That the next Project Steering Committee meeting address the differences between PIs in administrationof student stipends and determine whether some of these differences can be eliminated when interpretedon the basis of the amounts received by the students.

That MSc course Coordinators at each PI ensure that lecturers include sufficient field practice for eachcourse.

That UC ensure that consultants contracted to deliver short courses be credible and well prepared - theprograms at each PI will still depend on short courses to deliver critical material.

That, by at least the end of calendar year 2003, the Project Steering Committee review Projectexpenditures to determine whether sufficient funds remain to support further scholarships in Cycle III,and whether a recommendation to this effect should be made to CIDA.

That the Steering Committee should re-examine the possibility of delivering a distance education coursein hydrogeology, the activity to recommence in Yr 5, once the hydrogeology course is published. It wouldbe most logical to implement it from UCR.

Applied Thesis Projects

That it is incumbent upon the CIs to ensure that Canadian students are sensitive to issues of partnershipbefore departure, and that arrangements for Canadian students stretch to full consultation on PIcollaboration on thesis work, and expectations of debriefing seminars and reports prior to return toCanada.

Internationalization

That UC revisit the purpose of this component, and redefine the expected results. These should be seen asseparate from the crossover with IS, and other components, even though it is the learning from theseactivities which provide the catalyst for Internationalization

Management

That the Project Manager rebuild bridges at UCR. UCR has made a significant contribution to IS of theother PIs, and is still an institution with a regional commitment.


That the Project Manager refrain from involving himself directly in internal UCR management issues,and allow UCR the flexibility important to valuing CARA within the framework of the other projects ofwhich UCR is the beneficiary.

That the Project Manager begin to consider UC’s disengagement strategy. He should step back from someof the internal issues, and allow the PIs to manage themselves. He should spend Yr 4 fostering thestrategic mechanisms important to the broader outcome of the Project.

That the UC Advisory Committee support the Project Manager in a) resolving issues at UNAN, and b) thedisengagement strategy.

That the Steering Committee address issues of sustainability, especially the form and purpose of thenetwork in a) supporting inter-institutional collaboration on the MSc programs, and b) contribution toregional policy making on water management issues.

Balance of the Remaining Years

That the balance of investment should favour network building and training as the two most important,ongoing components.

Concepts

That the partners review present program structure to determine whether greater allowance need to bemade for research time during current ‘teaching’ trimesters.

That the partners examine the advantages and disadvantages of replacing the thesis trimester with acommunity-based field school. Consideration should include benefits in terms of expected outcome,alternative human and financial resources required.

That increased, professorial-level research funding be reconsidered, and some significant alterations bemade to the way in which student support is allocated, i.e. that academic staff in the three PIs develop andput forward research projects for CARA funding which could be undertaken in the same two-year cycle ofthe MSc program.

That a working paper on a (possibly competitive) research granting mechanism be generated by theCARA Coordinators and UC for circulation before the October Steering Committee meeting. If such asystem is to run in parallel with the MSc programs, it will have to be in place by the end of calendar year2002

1 27See "Impact" of the Program, page 5 of the Grant Agreement's Annex A - Part 1, Original Tier One Proposal.


Appendix 1. Terms of Reference

PROGRAM DESCRIPTION

The Central American Water Resource Management Network Program aims to "lead to an increased capacity ofCentral American countries to educate and train human resources in the area of hydrogeology and water resourcemanagement. The creation of a Central American Water Resource Management Network will lead to improvedmanagement and protection of Central American water resources through the partnerships and collaborations ofhighly qualified water resource professionals from universities, public water institutions, non-governmentalorganizations, the private sector, international funding." 27

It was approved in a Grant Agreement between CIDA and the University of Calgary for the "Central American WaterResource Management Network" Project, (ICD/CIDA no. S55601) which was signed in December 1999, under theauspices of CIDA's Institutional Cooperation Division's "University Partnerships in Cooperation and Development"(UPCD) funding mechanism. The CIDA grant is for $3,995,000 million Canadian dollars and covers a five year periodplus three months for reporting, with this grant scheduled to terminate in March 2004.

The Canadian partners are:- the University of Calgary, Alberta (the lead institution); and- the University of Waterloo, Ontario.

The Central American partners are:- the Universidad de Costa Rica, San José, Costa Rica;- the Universidad Nacional Autonoma de Nicaragua, Managua, Nicaragua; and - the Universidad de San Carlos de Guatemala, Guatemala City, Guatemala.

Anticipated Outcome components of the program:

The main anticipated outcomes of the program are:

- institutional strengthening: the partner institutions are appropriately mandated and prepared to offer highquality Masters programs and professional short courses in hydrogeology and multidisciplinary watermanagement.

- network creation: the partner institutions benefit from resource sharing and information exchange,complementing each other's teaching and applied research strengths and an enhanced capacity to undertake othercollaborative research projects. Network members are connected to relevant information and similar or paralleltraining and research efforts through associations with outside institutes, agencies and networks.

- training: graduates from the various training programs influence the policies, plans and practices of relevantpublic and private sector institutions and/or strengthen the teaching and research capacities of the partnerinstitutions.

- community-based field research: the developed Participatory Development Methodology is applied in greaternumber of communities. Women are successfully involved and empowered by community activities. Graduatesincorporate participatory development methods into their work, influencing their colleagues and the waterresource management profession in the process. Involved communities identify water issues, seek technicalassistance, if needed, and organize for long term solutions. The public sector partners integrate results and


recommendations of studies, policies, plans and/or programs. The research results are credible and attractattention regionally and internationally generating new partnerships. Community-based water managementstrategies are created and partner institutions carry out projects which are integrated into such strategies.

- development education: Canadians learn from the partner institutions' experience in Central America waterissues and the understanding of local cultural, social, economic and legal realities necessary for addressing theseissues. Canadian institutions and the public are more knowledgeable in water issues and Canada's efforts toprovide assistance (ODA) in this regard.

- private sector involvement and collaboration: Canadian firms gain knowledge of Central America's needs andmarkets. Canadian and Central America's private sector technology and expertise in hydrogeology and waterresource management are increasingly marketed in Central America (and Latin America). Network activitieslead to professional associations and contracts between Central American and Canadian firms. The CentralAmerican private sector is stimulated and modernized.

- internationalization of Canadian institutions: students and faculty in the Canadian institutions demonstratecross-cultural sensitivity and embody a Central American "partnership" ethos in their work. Canadian studentsand faculty integrate central American water issues into curricula and research. Canadian institutions serve theglobal community and develop world-class reputations as centers for higher learning. Enrollment of Central andLatin American students in Canadian institutions is increased.

PURPOSE OF THE EVALUATION

CIDA requires that all projects funded under the UPCD Tier 1 program conduct a mid-term evaluation. Stakeholderswill use the results of the evaluation to make necessary adjustments to project approach and methodology.

SCOPE AND FOCUS

The focus of the evaluation should be on the reporting of results as they relate to the expected project outcomecomponents detailed above. The Mid-Term Evaluation will address the various aspects of program rationale,effectiveness, efficiency, results and achievements to date. Also, the review will produce a picture of the program withinthe context of the following questions:

Goals and objectives- What is the progress made toward the achievement of expected results and outcomes?

- How is the program design providing an appropriate investment balance for each of the seven components of theprogram?

- More precisely, what is the progress made toward the achievement of expected results and outcomes for each ofthe program's seven components?

Institutional Strengthening Component:- How does the program strategy provide an appropriate method for CIDA to support institutional capacity

strengthening and development needs in Central America?- What level of commitment and ownership towards the program's inputs, outputs, results and outcomes, is being

demonstrated by the Central American partners, as a measure of ascertaining the sustainability of the Program?

Network component:- Describe the progress made to develop a Central American Network.- Describe the progress made to support the Development of Computer Communications Links among Central

American universities and between Central America and the rest of the world.


- Comment on the increased internationalization of the partner organizations.

Training Component:- Describe the progress made to train graduates so they can influence the policies, plans and practices in their field

of research or strengthen the teaching and research capacities of the partner institutions.

Community-based field research Component:- Assess how and to what extent graduates incorporate participatory development methods into their work.

Development education Component:- Describe how Canadians learn from the partner institutions' experience in Central America's water issues and

better understand the local cultural, social, economic and legal realities necessary for addressing these issues.- Describe what has been done to sensitize Canadians in Canada to those same issues and to international

development in general.

Private sector involvement and collaboration Component:- Describe to what extent specialized Canadian firms' visibility and influence has increased in Central America,

thanks to the project.

Internationalization of Canadian Institutions Component:- Comment on the University of Calgary and the University of Waterloo's increased capacity in international

development work.- Determine the extent to which sustainable (permanent) linkages are being established between Canadian and

Central American institutions.

Management of the Program- Are there sufficient, qualified, and available Canadian and Central American, physical, and financial resources to

accomplish the planned program's components' objectives and activities?- Assess the extent to which program inputs are being supplied in a timely and complete manner.- Assess the efficiency of the various partners' management of the program.- Comment on the appropriateness of the management structure for the program.- Comment on the performance measurement strategy developed by the University of Calgary and/or the Central

American Institutions for the program. - Determine the effectiveness of internal program monitoring and feedback mechanisms.- Describe the progress made to ensure that the equal Participation of Women in Central American Universities

generally is encouraged.- Comment on linguistic (Spanish-English) aspects of the program.

Lessons Learned and recommendations- What experiences / lessons are being learned that might guide the remainder of the program?- Provide recommendations to assist the program to be more effectively implemented.

STAKEHOLDER INVOLVEMENT

Stakeholder participation is fundamental to CIDA evaluations. The Consultant is expected to conduct a participatoryevaluation providing for meaningful involvement by project partners, beneficiaries and other interested parties.Stakeholder participation is to be an integral component of evaluation design and planning; information collection;the development of findings; evaluation reporting; and results dissemination.

ACCOUNTABILITIES AND RESPONSIBILITIES


CIDA's assigned Program Officer will represent the Agency during the evaluation. She will prepare the terms ofreference, in consultation with the University of Calgary's CARA members, propose and hire a consultant, andcoordinate the evaluation.

The CARA members, lead by the Program coordinator, will participate in the elaboration of the terms of reference andthe choice of a consultant among the ones proposed by CIDA.

The Consultant is responsible for: 1) conducting the evaluation; 2) the day-to-day management of operations; 3) regularprogress reporting to CIDA's project manager; 4) the development of results; and, 5) the production of deliverables inaccordance with contractual requirements. The Consultant will report to CIDA's program officer.

METHODOLOGY

- Review program files and interview CIDA officer(s) involved with the program at ICD / CIDA, Hull.

- Review program files at the University of Calgary, and interview in person or by phone University of Calgary andUniversity of Waterloo officers, staff, faculty and students involved with the program.

- Travel to Central America: interview Universidad de Costa Rica, Universidad Nacional Autonoma de Nicaraguaand Universidad de San Carlos de Guatemala officers, staff, faculty and students if applicable involved in theprogram, and members of other universities, research institutes, and/or others involved with the program.

- By telephone, by correspondence, or in person, communicate with any other individual(s) involved with, oraffected by the program, as deemed necessary.

DELIVERABLES

The evaluator/reviewer will provide CIDA with the following:

- A written draft workplan which will address the following reporting elements: Overview of ProgramExpectations of EvaluationRoles and ResponsibilitiesEvaluation MethodologyEvaluation FrameworkInformation Collection and AnalysisReportingWork Scheduling

The draft workplan is to be submitted within four (4) weeks of the signing of the contract.

- A written evaluation workplan within one (1) week of receiving CIDA's comments on the draft workplan, theConsultant will produce a final evaluation workplan.

- A written draft Mid-Term Evaluation report that describes the evaluation and puts forward the evaluator'sfindings, recommendations and lessons learned. The presentation of results is to be intrinsically linked to theevaluation issues, establishing a flow of logic development derived from the information collected. The draft Mid-Term Evaluation report will be due within four (4) weeks of returning from mission.

- A written final Mid-Term Evaluation report, with table of contents, an abstract/summary, and in an annex, alisting of issues, recommendations, and persons who were interviewed. This report will be due within two (2)weeks of receiving CIDA's comments on draft report.


Recommended report format:Description of context, expected results and participants (4-6 pages)Description of evaluation methodology and processes (3-4 pages)Brief summary of project activities (1-2 pages)Discussion and analysis of implementation, management and initial operations (3-5 pages)Description and analysis of preliminary results (5-10 pages)Discussion of unexpected results and lessons learned (2-3 pages)Evaluative conclusions and recommendations for improving project performance (2-3 pages)

These deliverables should be submitted electronically, in English.

EVALUATOR QUALIFICATIONS

The evaluation will be carried out by a Canadian Consultant who meets the following requirements :

- is a reliable and effective evaluation manager with extensive experience in conducting evaluations and a provenrecord delivering professional results;

- is fluent in English and Spanish;- has experience working with Universities, and with the concept of strengthening institutional capacities and the

notion of empowerment;- is fully acquainted with CIDA's results-based management orientation and practices;- ideally, is familiar with the Central-American Environment;- ideally, has specific knowledge in water resource management and/or hydrogeology.


Appendix 2. Institutions Visited and Persons Contacted

Date Institution Contact

Mar 18-20 CIDA Ms Anne-Sophie Belzile, Project Officer, InstitutionalCooperation Division, Educational InstitutionsProgram, Canadian Partnership Branch.

Mar 25 University of Calgary Mr David Bethune, Project ManagerMs Gabriela Rotela, Assistant Project ManagerDr Larry Bentley, Project DirectorMr Lorne Jacques, Director, UC International CentreDr E. Horbulyk, Economist

Mar 26 University of Calgary Heydy Calderon, Nicaraguan graduate studentNatasha Cassineth, CIDA Intern

Mar 27 University of Waterloo Dr Ramon Aravena, Prof.Dr Nilson Guiguer, Pres., Waterloo HydrogeologicDr David Rudolph, Assoc. Prof.Dr Bruce Mitchell, Assoc. Vice-President AcademicMr Drew Knight, Director, International ProgramsDr Cathy Ryan, Associate Project Director (Technical) Mr David Manuel Navarete, studentJohn Pearsal, student

Apr 03 University of Costa Rica MSc Ana Lucia Castro, CARA CoordinatorMSc Sergio Paniagua, Dir. Comision de PosgradoMSc Mario Arias, Prof.MSc Gunter Schozinsky, MSc Program Coordinator

Apr 04 UCR MSc Rolando Mora, ProfMSc Ana Teresita Aguilar, Dir. Esc CA de GeologiaMarieta Bravo, Financial OfficerDr Fernando Silesky, Dean, Faculty of Engineering

Apr 05 CIDA Post, San JoseUCR

Mr David MorrisMaria Perez, Dean, Postgraduate School

Apr 08 CIRA, UNAN, Managua MSc Salvador Montenegro, Dir. CIRAMSc Marisa Espinosa, CARA & CA CoordinatorMSc Marta Espinoza, HydrogeologistSr Jaime Lopez Lowery, Financial Officer, CIRAMSc Elsie Lopez Lowery, Vice Rector General UNAN

Apr 09 CIRA, UNAN, Managua Valeria Delgado, UC studentNinoska Cho, CIRAJorge Pitty, CIRAMaria Elena Vargas, CIRAMaximina Altamirano, ex-UCR studentJorge Guatemala, ex-UCR student


Apr10 CIDA Post, Managua

USAC, Guatemala

Mr Jack AdamsMs Nalini Ablack, CIDA PSUMSc Jose Chonay, CARA CoordinatorMSc Miguel Angel Morales Cayax, ProfMSc Rolando Lara, ProfMSc Erick Motta, Prof

Apr 11 USAC, Guatemala MSc Edgar Franco, Dean, Fac. Agron.Ms Margarita Ramirez, Financial OfficerMSc Maxdelio Herrera, Ex- CA CoordinatorMSc Victor Cabrera, Ex-Prof.Marvin Zaguero, studentLuis Moran, student

Apr 12 CIDA Post, GuatemalaUSAC

Ms Helene RivardCarlos Juarez, student


Appendix 3. Propuesta de Funciones del Coordinador Centroamericano de la Red CARA

Los responsabilidades y actividades del Coordinador Centroamericano de la Red CARA incluyen:

8. Ser el representante oficial de la Red CARA en cualquier actividad, reunion o evento en la region;9. Trabajar en equipo con el Coordinador Canadiense y Gerente del Proyecto CARA (Bethune) y Vice-

Gerente (Rotela). Mantener comunicacion continua con Bethune y Rotela;10. Trabajar en equipo con los lideres de las cinco universidades de la Red CARA y mantener comunicacion

regular con los lideres/contactos de los miemvros universitarios;11. Coordinar actividades de la Red que involucren a varios miembros, por ejemplo: cursos cortos, talleres,

reuniones del Comite Directiivo de CARA, congresos, etc.12. Conocer los programas academicos y las actividades de CARA en cada universidad miembro. Seria

necesario visitar estas instituciones durante la primera mitad de 2002;13. Conocer los otros programas academicos en CA que tienen que ver con hidrgeologia y manejo de recursos

hidricos con el objetivo de incluir nuevos miembros a la Red (especialmente en El Salvador, Honduras yPanama). Seria necesario visitar estos paises/instituciones durante 2002. Elaborar y someter aconsideracion y aprobacion de los lideres, procedimientos, normas y reglamentos para la incorporacion denuevos miembros;

14. En colaboracion con el Coordinador Canadiense, desarrollar un plan para el sostenimiento de la Red unavez finalizado el proyecto ACDI;

15. Promocionar la Red CARA y realizar gestiones de financiamiento alternativo con agencias donantesinternacionales y con organizaciones hidrologicas locales y regionales;

16. Participar en la elaboracion y/o revision de los informes narrativos semi-anuales para ACDI;17. Coordinar la organizacion del I Congreso Regional de CARA a realizarse en Nicaragua, en Abril 2003.18. Ser co-editor (con Bethune y Rotela) del boletin semi-anual de CARA. Recolectar informacion relevante

sobre problemas hidricos en Centroamerica, conferencias, oportunidades y proyectos de tesis, paraincluirla en el boletin de CARA;

19. En equipo con Rotela, facilitar la recoleccion continua de datos (e interpretacion/presentacion) dentro delsistema de monitoreo de resultados;

20. Mas? Menos?


Appendix 4. M.Sc. Students, by Institution, 2001-2002.

UNAN

Name Background Institution Thesis Supervisors Topic

Rosa Maria GonzálezTapia

Lic. Química UNAN-ManaguaDirector: Adela CruzAsesores: Enoc Castillo Martha Espinoza

Estudio sobre la calidad del agua y riesgos decontaminación del campo de pozos Chaguitillo,Valle de Sébaco.

Thelma ZulemaSalvatierra Suárez

Lic. Biología CIRA-UNAN-Managua

Director: Salvador MontenegroG.Asesores: Miguel Cáceres Benedicto Valdez

Plan de gestión y desarrollo integral en lassubcuencas Las Playitas, Moyúa y Tecomapa, dela cuenca del Río Grande de Matagalpa.

Juana Del C. RuizMendieta Ing. Geóloga ANGPA

Director: Martha EspinozaAsesores: Marisa Espinosa Róger Mejía

Aplicación de un modelo numérico para el manejode la explotación de pozos de Llano Grande ySanta Clara, Valle de Apanás, Jinotega.

Zeneyda Del S. RizoOsorio Ing. Geóloga ANGPA

Director: Dionisio RodríguezAsesores: Ottoniel Arguello Enoc Castillo

Optimización del proceso de recarga del acuíferopor uso del suelo, en la subcuenca III de lacuenca sur del Lago de Managua.

Maria Luisa Talavera López Lic. BiologíaCIRA-UNAN-Managua

Director: Evelyn HookerAsesores: Sergio Gámez Martha Espinoza

Estrategias para preservar la calidad de agua delcampo de pozos del Valle de Apanás, Jinotega,Nicaragua.

Yelba Del Carmen FloresMeza

Ing. Geóloga INETERDirector: Martha EspinozaAsesores: Enoc Castillo Marisa Espinosa

Aplicación de un modelo matemático para laevaluación del acuífero del Valle de Sébaco.

Mayra Altamirano Carcache Lic. Biología UNAN-ManaguaDirector: Ma. Elena VargasAsesores: Ramón García Carmen Pong

Plan de acción participativo para potenciar ladisponibilidad, calidad y accesibilidad de agua en lacomunidad de Colonia Roque, Municipio de Tipitapa

Wilber Antonio Cuadra Jirón Ing. Agrónomo ONG

Director: Rodolfo JaénAsesores: Antonio Álvarez

Gestión del riesgo de contaminación de suelo en elacuífero León-Chinandega, por residuos de toxafenoaplicado al cultivo del algodón en Chichigalpa yPosoltega.

Francisco Mercado Jiménez Lic. Metereología MAGFOR

Director: Jorge RodríguezAsesores: Desirée Elizondo

Identificación de sitios vulnerables a la erosiónhídrica y propuestas par prácticas apropiadas de usoy manejo de suelos en la subcuenca III de la cuencasur del Lago de Managua.

Svetlana Dumailo Lic. Biología BICU

Director: Katherine VammenAsesores: Jorge Pitty Víctor Martínez

Propuesta de solución a los problemas desedimentación y contaminación de la Laguna deBluefields, mediante un plan de gerencia de lascuencas Río Escondido y Río Kukra, RAAS,Nicaragua.


Source: UNAN 2000.


UCR

Student/Nationality Thesis Project Thesis Supervisors Organizations Involved in Thesis

Randall AlpizarCostarricense

Modelo Numerico del Valle Aluvial Superior delRio Tempisque, Provincia de Guanacaste, CostaRica

Director: Gunter SchosinskyAsesora: Ana Lucia CastroAsesor: Rolando Mora

Instituto Costarricense de Acueductos yAlcantarrillados (AyA);Ecodesarrollo Papagayo

Hector Enrique Zuñiga MoraCostarricense

Caracterizacion Hidrogeologica del Acuifero SantaCruz, Guanacaste, Costa Rica

Director: Gunter SchosinskyAsesora: Ana Lucia CastroAsesor: Mario Arias

Instituto Costarricense de Acueductos yAlcantarrillados (AyA);

Paola Fuentes SchweizerCostarricense

Caraterizacion Hidrogeologica de la Cuenca de RioPotrero, Nicoya, Guanacaste, Costa Rica

Directora: Ana Lucia CastroAsesora: Gunter SchosinskyAsesor: Orlando Bravo Trejos

Instituto Costarricense de Acueductos yAlcantarrillados (AyA);Ministerio del Ambiente y Energia;Universidad de Costa Rica.

Jorge Guatemala HerreraNicaraguense

Uso de Metodos Geofisicos en la Delimitacion dela Posible Estela de Contaminacion Producida porlos Lixiviados del Botadero de Desechos La Joya,Granada, Nicaragua

Withdrawn

Maximina Altamirano EspinozaNicaraguense

Determinar la Contaminacion por Metales Pesadosen las Aguas Subterraneas de las Comunidades deLas Mangas, El Zapote, Roberto Centeno, SabanaLarga y Comarca La Union, Municipio de Sebaco,Depto de Matagalpa, Nicaragua

Withdrawn

Julio Jhonnassen TaracenaHernandezGuatemalteco

Caracterizacion hidrogeoquimica del AcuiferoPresente desde la Parte Alta de la Cuenca del RioGuacalate hasta la Poblacion de Alotenango,Sacatepequez, Guatemala

Directora: Ana Lucia CastroAsesora: Rolando Mora Asesor: Isaac Herrera(Guatemala)

Tomas Antonio Padilla CambaraGuatemalteco

Evaluacion del Potencial Hidrico en laMicrocuenca del Rio Cantil, para elAprovechamiento de las Aguas Subterraneas en laFinca Sabana Grande, El Rodeo, Escuintla,Guatemala

Directora: Ana Lucia CastroAsesora: Gunter SchosinskyAsesor: Rolando Mora


Eugenio Oliverio Orozco y OrozcoGuatemalteco

Potencial del Recurso Hidrico Subterraneo yModelo Matematico Preliminar del Acuifero delValle Aluvial del Rio Guacalate desde AntiguaGuatemala hasta Alotenango, Sacatepequez,Guatemala

Director: Gunter SchosinskyAsesora: Ana Lucia CastroAsesor: Asdrubal Vargas

Facultad de Agronomia, USAC

Source: UCR 2002

USAC

Student Thesis Project Supervisors

Marvin Alfonso Romero Santizo Valoracion Economica de Lago de Atitlan, Solola,Guatemala

H. Cardona (Principal)M. BarrientosJ. Skinner

Luis Fernando Moran Palma Estudio Hidrogeologico de la Montaña El Soco en elDepartamento de Chimaltenango

Erick Motta (Principal)A. RevolorioG. Santos

Marvin Roberto Salguero Barahona Reconocimiento Hidrogeologico con Fines de Riego,de la Cuenca del Rio Acome, Escuintla, Guatemala.

Erick Motta (Principal)G. SantosJ. Sanchez

Carlos Wilfredo Juarez Quim Calidad de Agua para Consumo Humano y Agricolaen la Microcuenca del Rio Balanya, Chimaltenango

A. Sacbaja (Principal)D. JuarezM. Barrientos

Norma E. Gil Rodas de Castillo Evaluacion Fisicoquimica y Micro biologica delAgua La Laguna de Ayarza

A. Sacbaja (Principal)A. ZuñigaP. Garcia

Source: USAC, 2002.


Appendix 5. Documents Consulted

Anon, undated. Plan Ambiental de la Region Centroamericana, Direccion General del Medio Ambiente delSistema para la Integracion Cenroamericana, pp32.

Anon, undated. Estrategia de Fortalecimiento de la Capacidad Institucional: Periodo de 1999 a 2004. ManejoSostenible de Suelo y Agua con Especialidad de Planificacion de Recursos Hidricos en Agronomia, Facultad deAgronomia, Programa de Estudios de Postgrado, Universidad de San Carlos.

Anon, undated. Estrategia para el Fortalecimiento de la Capacidad: Formacion de Profesionales en la Gerenciade los Recursos Acuaticos en Nicaragua, Proyecto CARA. CIRA, Universidad Nacional Autonoma de Nicaragua,1999-2000.

Anon, undated. Estado del Ambiente y Los Recursos Naturales en Centroamerica, 1998. ComisionCentroamericana de Ambiente y Desarrollo, pp179.

Anon, 1998. Central American Water Resource Management Network: Tier 1 Proposal, UPCD, submitted byUniversity of Calgary.

Bethune, D. Letter to CIDA dated 26 Apr 2000: Program 01888/S55601 (CARA) - Inception Mission report andProposed Modifications to Project

Anon, 1999. CARA Yr 1 Workplan, University of Calgary.

Anon, 2000. CARA Yr 2 Workplan, University of Calgary.

Bethune, D., 2001. CARA Yr 3 Workplan, University of Calgary

Anon, 2000. CARA Annual Narrative Report, 1 Dec 1999 - 31 Mar 2000, University of Calgary

Anon, 2000. CARA Semi-Annual Narrative Report, 1 Apr - 30 Sep 2000, University of Calgary

Anon, 2000. Valoracion y Estrategia de Capacidad de Fortalizamiento para la Maestria en Geologia con enfasisen Manejo de Recursos Hidricos e Hidrogeologia, Universidad de Costa Rica.

Anon, 2001. CARA Yr 2 Annual Narrative Report, 1 Apr 2000 - 31 Mar 2001, University of Calgary

Anon, 2001. CARA Semi-Annual Narrative Report, 1 Apr 2001 - 30 Sep 2001, University of Calgary

Astorga, A, et al., 2001. El Estado de la Gestion de la Calidad Ambiental Centroamericana, Sistema de laIntegracion Centroamericana, Comision Centroamericana de Ambiente y Desarrollo, pp167.

Anon, 2002. Presentacion del Programa de Maestria de Manejo Sostenible de Suelo y Agua con Especialidad enPlanificacion y Manejo de Recursos Hidricos en Agronomia. Proceso de Monitoreo y Evaluacion, USAC.

Anon, 2002. Informe de Monitoreo, Proyecto CARA-Nicaragua Año 2001. CIRA/UNAN-Managua.

Anon, 2002. Catalogo de Estudios, Programa de Maestria en Manejo Sostenible de Suelo y Agua conEspecialidad de Planificacion de Recursos Hidricos en Agronomia, Facultad de Agronomia, Programa deEstudios de Postgrado, Universidad de San Carlos, pp22.


Castro, A.L & G. Rotelo, 2002. Monitoreo y Evaluacion de los primeros tres años del proyecto CARA-Universidad de Costa Rica. Esc. CA de Geologia, UCR.

Bethune, D. and G. Rotela, 2002. Year 4 Workplan, University of Calgary.

Cuadra Jiron, W.A. 2002. Gestion del riesgo de contaminacion de suelo en el acuifero Leon-Chinandega, porresiduos de Toxafeno aplicado al cultivo de algodon en Chichigalpa y Posoltega. Protocolo de Investigacion,UNAN.

Ruiz, J. C., 2001. Aplicacion de un modelo numerico para el manejo de la explotacion de los pozos Llano Grandey Santa Clara, Valle de Apanas, Jinotega. Protocolo de Investigacion, UNAN.

Dumailo, S., 2002. Propuesta de solucion a los problemas de sedimentacion y contaminacion de la Laguna deBluefields, mediante un plan de gerencia de las cuencas Rio Escondido y Rio Kukra, Raas, Nicaragua. Protocolode Investigacion, UNAN.

Gonzalez, R.M., 2002. Estudio sobre la calidad del agua y riesgos de contaminacion del campo de pozosChaguitillo en el Valle de Sebaco. Protocolo de Investigacion, UNAN.

Meza, Y.F., 2002. Aplicacion de un modelo matematico para la evaluacion del acuifero del Valle de Sebaco.Protocolo de Investigacion, UNAN.

Osorio, Z.R., 2002. Optimizacion del proceso de recarga del acuifero de Managua por el uso del suelo(urbanizacion) en la sub-cuenca III de la cuenca sur del Lago de Managua. Protocolo de Investigacion, UNAN.

Mercado, J.F., 2002. Identificacion de sitios vulnerables a la erosion hidrica y propuestas para practicasapropiadas de use y manejo de suelos en la subcuenca III de la cuenca sur del Lago de Managua. Protocolo deInvestigacion, UNAN.

Salvatierra, T., 2002. Plan de gestion y desarrollo integral en subcuencas Las Playitas, Moyua y Tecomapa, de lacuenca del Rio Grande de Matagalpa. Protocolo de Investigacion, UNAN.

Altamirano, M.E., 2002. Plan de accion participativo para potenciar la disponibildad, calidad y accesibilidad deagua en la comunidad de Colonia Roque, Municipio de Tipitapa. Protocolo de Investigacion, UNAN.

Romero, M., 2002. Valoracion Economica de Lago de Atitlan, Solola, Guatemala. Protocolo de Investigacion,USAC.

Salguero, M., 2001. Reconocimiento Hidrogeologico con Fines de Riego, de la Cuenca del Rio Acome, Escuintla,Guatemala. Protocolo de Investigacion, USAC.

Juarez, W., (undated). Calidad de Agua para Consumo Humano y Agricola en la Microcuenca del Rio Balanya,Chimaltenango. Protocolo de Investigacion, USAC.

Moran, L., (2001). Estudio Hidrogeologico de la Montaña El Soco en el Departamento de Chimaltenango.Protocolo de Investigacion, USAC.

Gil, N., (undated). Evaluacion Fisicoquimica y Micro biologica del Agua La Laguna de Ayarza. Protocolo deInvestigacion, USAC.

Zuñiga, H., (2001). Caracterizacion Hidrogeologica del Acuifero Santa Cruz, Guanacaste, Costa Rica. Protocolode Investigacion, UCR.


Padilla, T., (2001). Evaluacion del Potencial Hidrico en la Microcuenca del Rio Cantil, para el Aprovechamientode las Aguas Subterraneas en la Finca Sabana Grande, El Rodeo, Escuintla, Guatemala. Protocolo deInvestigacion, UCR.

Taracena, J., (2001). Caracterizacion hidrogeoquimica del Acuifero Presente desde la Parte Alta de la Cuenca delRio Guacalate hasta la Poblacion de Alotenango, Sacatepequez, Guatemala. Protocolo de Investigacion, UCR.

Fuentes, P., (2001). Caracterizacion Hidrogeologica de la Cuenca de Rio Potrero, Nicoya, Guanacaste, CostaRica. Protocolo de Investigacion, UCR.

Alpizar, R., (2001). Modelo Numerico del Valle Aluvial Superior del Rio Tempisque, Provincia de Guanacaste,Costa Rica. Protocolo de Investigacion, UCR.

Orozco, E., (2001). Potencial del Recurso Hidrico Subterraneo y Modelo Matematico Preliminar del Acuifero delValle Aluvial del Rio Guacalate desde Antigua Guatemala hasta Alotenango, Sacatepequez, Guatemala.Protocolo de Investigacion, UCR

Altamirano, M., (2001). Determinar la Contaminacion por Metales Pesados en las Aguas Subterraneas de lasComunidades de Las Mangas, El Zapote, Roberto Centeno, Sabana Larga y Comarca La Union, Municipio deSebaco, Depto de Matagalpa, Nicaragua. Protocolo de Investigacion, UCR.

Guatemala, J., (2001). Uso de Metodos Geofisicos en la Delimitacion de la Posible Estela de ContaminacionProducida por los Lixiviados del Botadero de Desechos La Joya, Granada, Nicaragua. Protocolo de Investigacion,UCR.