evaluation of the environment and forests … › derec › ec › 35666236.pdf · the core team is...

EVALUATION OF THE ENVIRONMENT AND FORESTS REGULATIONS 2493/2000 and

2494/2000

Synthesis Report Volume I: Main Report

November 2004

Evaluation for the European Commission

Evaluation of the Environment and Forests Regulations 2493/2000 and 2494/2000 DRN-ADE-NCG-ECO

Synthesis Report November 2004

ii

Italy

Aide à la Décision

Economique Belgium

ECO-Consulting Group

Germany

Nordic Consulting Group Denmark

DRN, leading company: Headquarters Via Ippolito Nievo 62 00153 Rome, Italy Tel +39-06-581-6074 Fax +39-06-581-6390 [email protected] Belgium office Square E. Plasky 92 1030 Brussels, Belgium Tel: +32-2-732-4607 Tel/Fax +32-2-736-1663 [email protected]

This evaluation was commissioned by:

the Evaluation Unit common to:

EuropeAid Co-operation Office,

Directorate General for Development and

External Relations Directorate-General

The core team is composed by: Cornelia Sepp (team leader), Dolf Noppen, Steve Sepp, Stefan Mann, Karla

van Eynde and Giulia Pietrangieli (junior expert).

The national/regional experts are: Michiel Meijer, Cristina Ribeiro de Carvalho, Julio Victor Vidal, Lisel Alamilla, Bernal Herrera-Fernandez, Kidane Mengistu, Simon Kosgei Choge, Souleymane Guye, Kameni Roger

Foteu, Yohanes Muliastra, Nguyen Xuan Nguyen, Serey Rotha Ken, Girish Chand Bhardwaj. .

The consortium contract manager is: Paola Gosparini

The evaluation is managed by the evaluation unit who also chairs the steering group composed by members of the services (EuropeAid, Relex, DG Dev, DG Environment, DG Research and DG

Budget)

The opinions expressed in this document represent the authors’ points of view which are not necessarily shared by the European Commission or by the authorities of the countries

concerned.

mailto:[email protected]





iii

Table of Contents

0 EXECUTIVE SUMMARY ...................................................................................... 1 0.1 Background and Evaluation Framework................................................................................... 1 0.2 The Evaluation Process, its Phases, and Methodology .......................................................... 1 0.3 Assessment of the relevant EC strategic and policy framework............................................ 2 0.4 General characteristics of the Budget Line project portfolio ................................................. 2 0.5 Findings ........................................................................................................................................ 2

0.5.1 Achievement of the Regulations’ stated objectives – EQ 1a................................................................ 3 0.5.2 Contribution to national policy formulation and local livelihoods – EQ 1b ............................................ 3 0.5.3 Taking into account specific needs and interests of partner countries – EQ 2..................................... 3 0.5.4 Instruments and Procedures for implementation of the regulations - EQ 3.......................................... 4 0.5.5 Coordination, coherence and complementarity to other donors – EQ 4 .............................................. 4 0.5.6 Link to international Agreements and Processes - EQ 5 ..................................................................... 4 0.5.7 Relationship to other Commission development activities and policies – EQ 6 ................................... 4 0.5.8 Consideration of Cross-Cutting issues – EQ7 ..................................................................................... 5 0.5.9 Added Value of the two Regulations – EQ8......................................................................................... 5

0.6 Conclusions and Recommendations ........................................................................................ 5 0.6.1 Conclusions and Recommendations for the Forests Regulation ......................................................... 6

0.6.1.1 Conclusions ..................................................................................................................................... 6 0.6.1.2 Recommendations........................................................................................................................... 6

0.6.2 Conclusions and Recommendations for the Environment Regulation ................................................. 7 0.6.2.1 Conclusions ..................................................................................................................................... 7 0.6.2.2 Recommendations........................................................................................................................... 8

0.6.3 Conclusions and Recommendations for both Regulations .................................................................. 9 0.6.3.1 Conclusions ..................................................................................................................................... 9 0.6.3.2 Recommendations......................................................................................................................... 10

1 THE EVALUATION FRAMEWORK.................................................................... 11 1.1 Objective and scope of the evaluation .................................................................................... 11 1.2 The evaluation process and its phases................................................................................... 11 1.3 The Evaluation Questions......................................................................................................... 12 1.4 Methods of data collection and analysis................................................................................. 12 1.5 Methods of Judgement ............................................................................................................. 16 1.6 Approach to the presentation of findings, conclusions and recommendations ................ 16

2 EVALUATION BACKGROUND.......................................................................... 17 2.1 Political, Economic, Social and Cultural Dimensions affecting Environment and Forests

in 3rd World Countries ............................................................................................................... 17 2.2 Synthesis of the EC’s Strategies & Programmes................................................................... 18

3 GENERAL CHARACTERISTICS OF THE BUDGET LINE PROJECT PORTFOLIO ....................................................................................................... 19

3.1 Project portfolio of the Forests Regulation ............................................................................ 20 3.2 Project portfolio of the Environment Regulation .................................................................. 20

4 FINDINGS ........................................................................................................... 21 4.1 EQ 1a: Achievement of the stated objectives......................................................................... 21 4.2 EQ 1b: Contribution to national policy formulation and local livelihoods .......................... 27 4.3 EQ 2: Taking into account specific needs and interests of partner countries.................... 30 4.4 EQ 3: Instruments and procedures for implementation of the Regulations ....................... 32 4.5 EQ 4: Coordination, Coherence and complementarity to other donors .............................. 38 4.6 EQ 5: Link to international agreements and processes ........................................................ 41 4.7 EQ 6: Relationship to other Commission development activities and policies.................. 44 4.8 EQ 7: Consideration of cross-cutting issues.......................................................................... 48 4.9 EQ 8: Added value of the two Regulations ............................................................................. 51



iv

5 CONCLUSIONS AND RECOMMENDATIONS................................................... 55 5.1 Conclusions and recommendations for the Forests Regulation.......................................... 55

5.1.1 Conclusions for the Forests Regulation ............................................................................................. 55 5.1.1.1 Relevance...................................................................................................................................... 55 5.1.1.2 Effectiveness ................................................................................................................................. 56 5.1.1.3 Efficiency ....................................................................................................................................... 57 5.1.1.4 Impact/Sustainability...................................................................................................................... 57 5.1.1.5 Complementarity with other EC financing instruments .................................................................. 57

5.1.2 Recommendations for the Forest Regulation .................................................................................... 58 5.1.2.1 Recommendation at regulation and wider EC framework level...................................................... 58 5.1.2.2 Recommendations at Budget-Line administration level ................................................................. 59 5.1.2.3 Recommendations at project implementation level........................................................................ 59

5.2 Conclusions and recommendations for the Environment Regulation................................. 61 5.2.1 Conclusions for the Environment Regulation..................................................................................... 61

5.2.1.1 Relevance...................................................................................................................................... 61 5.2.1.2 Effectiveness ................................................................................................................................. 61 5.2.1.3 Efficiency ....................................................................................................................................... 62 5.2.1.4 Impact/Sustainability...................................................................................................................... 62

5.2.2 Recommendations for the Environment Regulation........................................................................... 62 5.2.2.1 Recommendation at regulation and wider EC framework level...................................................... 62 5.2.2.2 Recommendations at Budget-Line administration level ................................................................. 63

5.3 Conclusions and recommendations for both Regulations ................................................... 64 5.3.1 Conclusions for both Regulations ...................................................................................................... 64

5.3.1.1 Relevance...................................................................................................................................... 64 5.3.1.2 Effectiveness ................................................................................................................................. 64 5.3.1.3 Efficiency ....................................................................................................................................... 66 5.3.1.4 Impact /Sustainability..................................................................................................................... 66 5.3.1.5 Coordination, Coherence and Complementarity ............................................................................ 67

5.3.2 Recommendations concerning both Regulations............................................................................... 67 5.3.2.1 Recommendations at Budget-Line administration level ................................................................. 67 5.3.2.2 Recommendations at project implementation level........................................................................ 69

List of Figures:

Figure 1: The geographical distribution of Budget line commitments ................................................... 19 Figure 2: The geographical distribution of Forests Regulation commitments ....................................... 20 Figure 3: The geographical distribution of Environment Regulation commitments............................... 21 Figure 4: Forests Regulation - project distribution by thematic categories, 2000-2002 commitments . 22 Figure 5: Environment Regulation - project distribution by thematic categories, 2000-2002

commitments.......................................................................................................................... 23

List of Tables:

Table 1: Classification of the 97 grant contracts according to budget year, programme and applied

selection procedure................................................................................................................ 19 Table 2: Overall performance rating of projects visited during the field phase ..................................... 26 Table 3: Timeline of the first CfP ........................................................................................................... 33 Table 4: The Regulations’ stipulations on information exchange and (donor) coordination ................. 39 Table 5: Selected references to forest / environment relevant international agreements and processes

and multilateral agreements................................................................................................... 43 Table 6: Percentage of EC delegations highlighting individual cross-cutting issues ............................ 49 Table 7: Percentage of projects highlighting individual cross-cutting issues ........................................ 50



v

Table of Content of Volume 2: Annexes

Annex 1: Terms of Reference

Annex 2: List of Persons Contacted

Annex 3: Bibliography

Annex 4: Project Portfolio 2000-2002 - Forests and Environment Regulations

Annex 5: General characteristics of the B 7-620 Budget - Line project portfolio – in figures and numbers -

Annex 6: Evaluation Questions, Criteria and Indicators

Annex 7: Description of procedural Aspects

Annex 8: Analysis of Call For Proposals

Annex 9: Discussion of Regulations’ eligibility rules

Annex 10: Discussion of Regulations’ thematic scope, and the selection of priorities for implementation

Annex 11: Comparison of environment and forest related thematic foci in policy documents

Annex 12: Comparison of environment and forest related thematic foci in Strategic Guidelines

Annex 13: Analysis and interpretation of the received Delegation Questionnaires

Annex 14: Analysis and interpretation of the received Project Questionnaires

Annex 15: List of Projects visited during the field phase

Annex 16: Analysis of the Project Assessment Sheets – Field visits

Annex 17: Integration of forest/environment-relevant aspects into Regional and Country programmes

Annex 18: Analysis of Forest and Environment related projects under other financing instruments

Annex 19 Listing of Steering Group members



vi

Listing of Abbreviations

ACP African, Caribbean and Pacific Countries AFLEG African Forest Law Enforcement and Governance (regional process) ALA Asian, Latin American Countries CBD Convention on Biological Diversity CCD Convention to Combat Desertification CfP Call for Proposals CITES Convention on International Trade in Endangered Species of Wild Fauna and Flora COP Conference of the Parties CPF Collaborative Partnership on Forests CRIS Common RELEX Information System CSP Country Strategy Paper DAC Development Assistance Committee (of the OECD) DEL Delegations (internal abbreviation used to identify questionnaires) DG Directorate General (of the European Commission) EC European Communities ECOFAC Programme for Conservation and Use of Central African Forest Ecosystems EDF European Development Fund EQ Evaluation Question ETFAG European Tropical Forestry Advisors Group EU European Union FAO Food and Agriculture Organisation (of the UN) FLEGT Forest Law Enforcement, Governance and Trade G8 “Group of Eight” (regular summits of Canada, France, Germany, Great Britain, Italy, Japan,

Russian Federation, United States, European Union) GSP Generalized Scheme of Preferences HQ Headquarter IAF International Arrangement on Forests IFF Intergovernmental Forum on Forests IKM Information and Knowledge Management IPF Intergovernmental Panel on Forests ITTA International Tropical Timber Agreement ITTO International Tropical Timber Organisation M&E Monitoring and Evaluation MEA Multilateral Environmental Agreement MED Mediterranean Countries MDG Millennium Development Goals NFP National Forest Programme NGO Non-Governmental Organisation OVI Objectively Verifiable Indicators PAS Project Assessment Sheet PCM Project Cycle Management PROJ Project (internal abbreviation used to identify questionnaires) RAMSAR Convention on Wetlands RELEX External Relations (of the European Union) ROM Results-Oriented Monitoring RSP Regional Strategy Paper SADC South African Development Community SD Sustainable Development SFM Sustainable Forest Management SMAP Short and Medium-term Priority Environmental Action Programme SNRM Sustainable Natural Resource Management SWAP Sector-wide Approaches TACIS Technical Assistance for the Community of Independent States TFBL Tropical Forestry Budget Line UNCED United Nations Conference on Environment and Development UNFCCC United Nations Framework Convention on Climate Change UNFF United Nations Forum on Forests WTO World Trade Organisation



1

0 Executive Summary

0.1 Background and Evaluation Framework This report presents the outcome of the evaluation of the Environment and Forests Regulations1, which jointly provide a legal basis for disbursement of EC funds from the merged Budget Line B7-620. Both Regulations require that a comprehensive evaluation be submitted to the European Parliament by November 2004. This evaluation serves a double purpose, in that it

accounts to the legislative authority and wider public for the funds expended under the Environment and Forests Regulations

provides the relevant external co-operation services of the EC with (i) an ex post evaluation of the activities and (ii) proposals for the future of the Regulations (beyond their expiry in 2006).

The evaluation of the Environment and Forests Regulations covered a period from the coming into force of the Regulations in late 2000, to the end of the first trimester 2004. Given that the Regulations will expire by the end of 2006, the present exercise serves as a mid-term evaluation.

0.2 The Evaluation Process, its Phases, and Methodology The evaluation was prepared as a step-wise exercise, which consisted of the following phases:

Inception Phase. In the course of the inception phase, a variety of background information was examined by the evaluation team, and exploratory meetings were conducted with Commission staff in Brussels. The team reconstructed the “intervention logic” behind both Regulations, that is their underlying logical hierarchy of goals, purposes, activities, procedural foundation and so on, and further developed the Evaluation Questions by selecting suitable judgement criteria, indicators and appropriate modes of data-collection.

Desk Phase. During the desk phase, the evaluation team analysed the pertinent EU policy and procedural framework of the two Regulations. Quantitative evidence was collected on the project portfolio under Budget Line B7-620, and a statistical analysis prepared. The team further selected a sample of 35 projects in 16 countries for the ensuing field evaluation from the entirety of 97 projects identified under B7-620. Throughout the desk phase, instruments for data collection were developed/selected, fine-tuned and tested – including project assessment sheets (PAS) and questionnaires for use during the field phase.

Field Phase. During the field phase, the team visited selected projects on-site, conducted interviews with project staff and various stakeholders, and set up briefing and de-briefing meetings with the respective EC Delegations. In parallel with the field missions, a written poll was conducted, which addressed the entirety of projects under Budget-Line B7-620, as well as all EC Delegations in countries with B7-620 projects.

Synthesis Phase. The synthesis phase served to consolidate, analyse and discuss the full scope of findings in context. The findings have then been combined and structured to (i) answer the Evaluation Questions, (ii) draw Conclusions, and (iii) formulate concrete Recommendations for the future implementation of the Environment and Forests Regulations. Thus the answers to the EQs lead to the overall Conclusions which in turn lead to Recommendations.

The evaluation embodies three levels of assessment: The first level represents the Regulations’ design and content vis-à-vis the wider development policy framework. The second level addresses the practical enforcement of the Regulations, and administration of Budget Line B7-620 by the appropriate Commission services. The third level touches upon the practical implementation of individual measures by contractors in developing partner countries. At every major turn of the evaluation process, a report was prepared for consideration and approval by the AIDCO Evaluation Unit and the appointed Steering Group. 1 Council Regulations (EC) 2493/2000 “On measures to promote the full integration of the environmental dimension in the

development process of developing countries”, and 2494/2000 “On measures to promote the conservation and sustainable management of tropical forests and other forests in developing countries”



2

0.3 Assessment of the relevant EC strategic and policy framework Reconstruction of the Regulations’ intervention logic and analysis of the relevant policy framework revealed that the Environment and Forests Regulations are fully in line with all relevant policy documents and processes. The outcome of the policy analysis undertaken during the evaluation’s desk phase can be summarised as follows:

The examined EC policies in their entirety provide a logical hierarchy. The policy framework is coherent with, and relevant to, the international dialogue and

initiatives in respect of environmental and forest-related issues (MEA, international forest regime, etc.). This provides a precondition for the Regulations’ taking up international processes and regimes.

There exist two separate policies or strategies for forest and environment related activities under each of the various financing instruments.

Mainstreaming of environmental issues, which is one of the cross-cutting issues in EC development cooperation, is appropriately reflected in the Regulations.

The environmental and forest-policy frameworks have many thematic issues in common (thematic intersection).

The Environment and Forests Regulations are appropriately embedded in a wider contextual framework of EC policies and strategies, that is they reflect constituent strategies and objectives of both sectoral and cross-cutting policy documents.

A broad thematic intersection is apparent between the Environment and Forests Regulations.

Both Regulations display a comparatively high level of operationalisation, that is they present a logical hierarchy including policy goals, purposes and activities, and means of verification.

The Strategic Orientation as reflected in the guidelines is in line with the Regulations’ thematic scope, although direct references are in some cases difficult to identify.

Successive Strategic Orientations addressed similar thematic foci. Newly emerging issues, such as implementation of MEA or FLEGT, have been integrated

flexibly into the Strategic Orientation.

0.4 General characteristics of the Budget Line project portfolio Under the two Regulations a maximum amount of up to €342 million has been earmarked for expenditure over the total duration (2000-2006), with respective shares of €93 million for the environment and €249 million for the forests component. The identified and approved project-portfolio of the evaluation covers “direct grant projects” from 2000 (selected before the coming into force of the two Regulations), as well as projects under the first Call for Proposals2 and targeted projects (budget years 2001 and 2002). Following confirmation by the appropriate Commission services in Brussels (AIDCO F4), the project portfolio for the present evaluation comprises 97 projects, totalling approximately €116 million. The geographical distribution is as follows: about 36% of the project portfolio is implemented in Africa. Asia accounts for 20%. Projects in Latin America amount to about 35% of the total volume, whereas the Pacific region and global projects account for 2% and 7%, respectively. The following table below presents the distribution of projects across three consecutive budget years.

Budget 2000 Budget 2001 Budget 2002 Total (CFP/Direct) CFP Targeted CFP Targeted

ENV 14 9 1 6 5 35 FOR 14 20 2 20 6 62 Total 28 29 3 26 11 97

0.5 Findings The following paragraphs highlight the evaluation findings, which have been consecutively numbered (figures in square brackets). For a detailed explanation of individual findings, refer to the respective figure in the main report. 2 the second call for proposals being outside the evaluation’s time frame



3

0.5.1 Achievement of the Regulations’ stated objectives3 – EQ 1a The project portfolio is coherent with the thematic scope of the respective Regulations [1], but unbalanced in that it addresses topics mentioned in the Regulations only selectively and in part. This is due to the fact that according to the Commission’s interpretation of the Regulations, Commission services may narrow down the Regulations’ thematic scope through Strategic Orientation and Annual Programming. Successive Strategic Orientation exercises have concentrated on issues which occur under both Regulations (thematic overlap) [2]. The Forests Regulation’s geographical scope has been extended considerably (compared to the previous Regulation), so as to now include all types of forests and also the forest resources of partner countries outside the tropics [3]. At the same time, annual financial allocations have decreased substantially, a fact which might also provide an explanation for the observed thematic concentration over two successive Strategic Orientation cycles. Furthermore, both Regulations aim to promote innovation through pilot activities. Therefore, the project portfolio’s degree of “innovativeness” can help gauge the extent to which the Regulations’ objectives have been (or are being) achieved in practice [4]. It was found that projects mostly build on established and proven methods, but some approaches and tools are new, and possibly apt for replication in future. Further to the issue of the project quality, it was found that project design and planning [5] have improved in comparison with the findings of the 1998 evaluation of the Tropical Forest Budget Line. However, some flaws and bottlenecks persist – particularly with respect to the formulation of appropriate and measurable Objectively Verifiable Indicators (OVI). Such flaws notwithstanding, a definite majority of the projects visited displayed good overall performance, even though the projects’ efficiency was difficult to assess in some cases [7]; the last-mentioned point is due to a lack of transparency of the relative shares of co-funding, owing mostly to structural characteristics of the co-funding arrangement.

0.5.2 Contribution to national policy formulation and local livelihoods – EQ 1b In comparison with the 1998 evaluation of the previous Tropical Forest Budget Line, the current Forests Regulation has attained distinctly greater policy orientation [13]. Part of the positive overall judgement of project quality is that many projects focus proactively on contributions to national policy formulation processes [8] in their design. In practice, however, the picture looks somewhat different as few projects turned out to play an active role in policy formulation processes and fora [9]. There are a number of reasons for this. In some cases the partner countries’ unconducive political and legal frameworks [10] limited the projects’ opportunities. In about half of the partner countries, national sector administrations were found to oblivious of, and indifferent about projects implemented under B7-620 [11], sometimes because they were unaware of the projects’ existence, sometimes because they were uninvolved in the projects’ activities. Of course, an individual project’s impact on national (or, where decentralisation applies, regional) policy formulation and local livelihoods depends to a large extent on the type [14] and the modus operandi of the intervention. Prior to the now complete deconcentration of administrative responsibility for the Budget Line, EC Delegations did not serve as catalysts for B7-620 projects’ contributions to national policy formulation [12].

0.5.3 Taking into account specific needs and interests of partner countries – EQ 2 The evaluation established that projects under B7-620 reflect partner country policies adequately [15] in their project description and design. Once again, this finding illustrates the team’s positive overall assessment of the project portfolio. Likewise, a majority of projects under B7-620 follow up on the development of national policy frameworks [16] during their implementation. Projects were further found to consider local needs and interests in their majority [18], albeit with limited participation of local stakeholders in the design. With respect to the extent to which projects take into consideration national needs and priorities, the active involvement of the partner countries’ (national) sector administrations [17] remains a critical issue (even though partner countries’ sector administrations occasionally contribute to project implementation) - their involvement in project planning and design is restricted by the fact that previous issues of the Guidelines for Applicants under the Call for Proposals excluded sector administrations as ineligible.

3 For the sake of brevity and ease of reading, EQs are abbreviated and summarized in the Executive Summary. The

respective sections of the main report present the exact wording, rationale and the means of data collection and analysis in full detail for each Evaluation Question.



4

0.5.4 Instruments and Procedures for implementation of the regulations - EQ 3 Aside from procedural guidance in the Regulations themselves, administration of Budget-Line B7-620 is mainly determined by the budgetary framework which is uniformly applicable to the general budget of the European Communities. Enacted in 2002, the new procedures provide a greater degree of detail in their prescription [19] and are therefore in some aspects more restrictive than the old ones. The evaluation established that the elaborate procedures observed under the Call for Proposals produce high quality project proposals [20], but require high input from applicants as well as from the Commission Services. In a similar way, co-funding requirements of a minimum of 20% of the total budget in cash are perceived as too burdensome by many project implementers [32], and besides co-funding rules (e.g. admission of contributions in kind) are not evenly applied. In consequence, selection procedures exceeded the prescribed deadlines [21], as the first Call for Proposals has shown very clearly. During the first half of the Regulations’ period of validity, targeted projects [22] were predominantly being used to collaborate with international organisations and NGOs on an international scale. It was observed that programming exercises – as required under the Regulations and the pertinent procedural framework - do not adequately transfer and operationalise [24] the Regulations’ thematic scope, owing to their being contextually selective and focused on the thematic overlap between the two Regulations. The Guidelines for Applicants for the two past Calls for Proposals also narrowed down considerably the scope of eligible applicants [25]. In the evaluation team’s interpretation, this practice is hard to reconcile with the respective prescriptions in the Regulations. Likewise the Guidelines for Applicants (2001, 2003) ruled out small-scale projects [26], and defined maximum grant sizes for environment and forest projects (2003). The available instruments and procedures for selection do not allow any form of guaranteed continuation [27] or subsequent funding of a follow-up phase. This is due to the new budgetary framework which – since 2002 – substantially restricts the Commission’s discretion with respect to the award of grants. Such challenges notwithstanding, stakeholders involved in the implementation of B7-620 are mostly satisfied [23] with relevant information provided by the EC Services in Brussels. Addressing the implementation level, the evaluation found that the Forests Regulation’s requirement for preparatory studies [28] in advance of an application has not been met in most cases. Internal monitoring is left at the discretion of the grant-beneficiaries [30], while external monitoring & evaluation (initiated by the Commission services) does not fulfil the Regulations’ requirements [31]. This, and the fact that reporting formats are not standardised, in practice hinders institutional learning processes [29] within the Commission Services.

0.5.5 Coordination, coherence and complementarity to other donors – EQ 4 The Environment and Forests Regulations were found to be fully in tune with all relevant EC development cooperation policies and practices, which in turn are fully coherent with relevant international policy dialogues, initiatives and international regimes [33]. The Regulations themselves oblige the Commission proactively to seek information exchange and coordination [34] with Member States and other donors. The Commission and EC Member States exchange information on and coordinate programming exercises and budgetary allocations under B7-620 regularly, and through an institutionalised procedure [35]. However, donor coordination in respect of B7-620 projects is less assured at the level of partner countries [36], owing to the fact that in the past EC Delegations played no significant role in the administration of the Budget Line. Consequently other donors did not benefit from experiences and lessons learnt from EC measures under B7-620 [37]. But with deconcentration now a fact, this situation is likely to improve.

0.5.6 Link to international Agreements and Processes - EQ 5 The evaluation found a high degree of coherence between the two Regulations’ in terms of their design and content, including references to (i) sustainable development as a cross-cutting development paradigm [38], and (ii) all relevant forest/environment-related international agreements and processes [39]. By contrast, international processes and multilateral agreements are directly addressed by just about 10% of the projects implemented [40]; even so, most projects contribute indirectly.

0.5.7 Relationship to other Commission development activities and policies – EQ 6 Compared to the full scope of programmable aid provided by the EC, the proportion of overall EC development cooperation devoted to forests and environment is very limited [41]. More specifically, forest issues are not adequately reflected in CSPs and RSPs, whereas environmental



5

issues are [44]. The latter observation can be explained by the fact that mainstreaming is at the very core of the Environment Regulation. Mainstreaming of environment issues is also reflected in other EC sector policies (e.g. trade and development, sustainable transport development, rural development), although in an aggregated manner [43]. However, for both Environment and Forests there exist policy frameworks [42] which evenly apply to and guide environment and forest related development cooperation under different financing instruments. B7-620 projects in practice are detached from interventions under other financing instruments [46]. This general observation notwithstanding, it has been found that in particular research projects under B7-620 fulfil an important complementary role [47]. However, no institutionalised information exchange has been observed [48], and rarely are B7-620 projects taken up and continued under other financing instruments [45]. This finding invalidates to some extent the Regulations’ focus on promoting innovation through pilot measures.

0.5.8 Consideration of Cross-Cutting issues – EQ7 Cross-cutting issues in EC development cooperation such as, for example, good governance, human rights, gender issues and so on are reflected and addressed in both Regulations in an operational manner [49], down to the level of purpose and of activities to be implemented. Also at implementation level, cross-cutting issues are taken up as priorities and focal issues in successive issues of the Strategic Guidelines, and in the Guidelines for Applicants [50] under the CfP. The EC Delegations’ feedback indicates regional differences in how cross-cutting issues are prioritised [51] in practice in the national context. Projects were found to address cross-cutting issues even more selectively, in that they contribute mainly to Institutional Development and Capacity Strengthening [52].

0.5.9 Added Value of the two Regulations – EQ8 The two Regulations’ “added value” must be assessed in comparison with environment or forest related measures, as exist under other development cooperation instruments at the Commission’s disposal. Whereas environment projects exist under all geographic financing instruments, forest projects are found exclusively under EDF and ALA [54]. EDF covers a substantial number of geographically-distributed environment projects [55] while forest projects are few, and geographically concentrated besides. Few environment/forest related projects occur under ALA [56], and those that exist are regionally concentrated in Asia. They are, however, better funded than EDF projects. Environment/forest related measures under MEDA/TACIS play a negligible role [57]. Relevant projects occur also under horizontal financing instruments [58] – of which the NGO Budget Line B7-6000 clearly stands out as the most significant. The Environment and Forests Regulations offer the opportunity of testing innovative approaches and implementing applied research programmes. In this respect, the two Regulations provide a valuable complement for activities funded by DG Research [59]. The fact that a definite majority of B7-620 projects are well executed constitutes a realised precondition for the two Regulations achieving their potential “added value” in practice [53]. In summary, activities funded under the Forests Regulation (i) complement the limited and geographically concentrated interventions under geographical financing instruments, (ii) focus on participation of civil society and research organisations, promote (iii) innovative approaches and (iv) global initiatives [53-59]. Activities funded under the Environment Regulation (i) promote EC involvement in global initiatives and processes, (ii) provide opportunities for mainstreaming urban and industrial environmental issues.

0.6 Conclusions and Recommendations The Conclusions have been structured in line with the five DAC Evaluation criteria (Relevance, Effectiveness, Impact, and Sustainability) and the “3Cs” (Coordination, Complementarity and Coherence). In the following sections, conclusions are highlighted separately for each Regulation, and jointly for both. They have been consecutively numbered (roman numerals). Recommendations relate to different “levels of aggregation” and are accordingly classified according to whether they relate to (i) the wider procedural framework and the Regulations’ design (“R-WF”); (ii) administration of the budget line (“ABL”); or (iii) implementation of projects at partner-country level (“PI”). They are further rated as either 1st or 2nd priority, according to the evaluators’ professional judgement of their respective urgency and ease of implementation.



6

0.6.1 Conclusions and Recommendations for the Forests Regulation

0.6.1.1 Conclusions Relevance (I) The Forests Regulation is coherent with the EC Development Policy Framework, and reflects well the International Forest Regime. This is particularly highlighted by the fact that forest sector development is consistently perceived as functionally subordinate to sustainable development throughout the Regulation [findings 38, 42, 49]. (II) By contrast, a certain tension exists between the Regulation’s comprehensive intervention logic and numerous partner countries’ development priorities (i.e. their policies, interests and perceived needs). In consequence, forest issues tend to be undervalued in the partner countries’ strategic frameworks for development (where this holds true in a CSP, implementation of measures under Budget Line B7-620 is put at risk) [findings 10, 15, 16, 17, 18]. Effectiveness (III) Since the new budgetary framework came into force in 2002, procedural restrictions have reduced the Commission’s discretion to pursue strategic policy objectives through the Forests Regulation. This is because the previous practice of awarding direct grants for “targeted projects” has been restricted considerably [finding 22]. (IV) Even though measures implemented within the Forests Regulation’s purview are fully compatible with the Regulation’s thematic scope, the current project portfolio does not reflect the Forests Regulation’s thematic scope in a balanced manner – as the Regulation’s text suggests [findings 1, 2, 25]. (V) Annual financial allocations have decreased (compared to the previous TFBL), and are not commensurate with the Forests Regulation’s extended thematic and geographical scope [findings 3, 54-59]. Efficiency refer to section “Both Regulations” Impact/Sustainability (VI) The Forests Regulation’s impact materialises mainly at local level, although not necessarily during the implementation period of individual projects. This is because field implementation generally requires long implementation periods to achieve measurable impact. Of course, this aspect is largely determined by the type and purpose of a given intervention [finding 14]. (VII) The overall sustainable impact of projects under the Forests Regulation depends on whether they are taken up and continued by either partner countries, other donors, or (geographical) EC financing instruments. The latter depends on forest issues being adequately reflected in CSPs until the Regulation’s expiry in 2006. If they are not, there will be a reduced likelihood of further promising and innovative measures under other (geographical) financing instruments [findings 11, 36]. Complementarity with other EC financing instruments (VIII) Considering that forest-related development plays a negligible role under other financing instruments, the Forests Regulation is forced into a complementary role, even though it lacks the required capacity [findings 41, 44, 54-57]. (IX) As yet, the Forests Regulation’s complementarity with other financing instruments is largely coincidental, and does not result from strategic coordination of various financing instruments. The Forest Regulation’s potential to finance global projects with a strategic complementary focus, is therefore lost [findings 17, 21, 22, 44].

0.6.1.2 Recommendations Recommendations pertaining to the wider procedural framework and the Regulation’s design It is recommended that there should continue to be a Forests Regulation beyond 2006 [A] {conclusion I, VI, XVII} to address sustainable forest sector development on a separate, sector-specific legal basis, although in a manner complementary to other existing instruments. Pursuant to recommendation A, the evaluators recommend that financial allocations for sustainable forest sector development should be increased [B] {conclusion XXIII} under a future Forests Regulation, to service the Regulation’s important complementary role. This will require detailed documentation and dissemination of experiences and lessons learnt with a view to strengthening the case for more forest-related development cooperation by demonstrating past successes in forest sector development, especially in promoting poverty alleviation.



7

With the aim of improving both impact and efficiency, the evaluators recommend that the Commission should allow more flexible thematic focusing and further emphasize forests’ socio-economic significance [C] {conclusion IV, V}, and also that the previous focus on tropical forests should be restored [D] {conclusion V}. This would entail (i) repealing the requirement for a balanced coverage of the Regulation’s broad thematic scope, (ii) adding a requirement to focus on the socio-economic significance of SFM, and (iii) specifying a clear focus on tropical forests (including natural as well as secondary and plantation forests). It is further recommended that the Commission should enhance synergy between measures under the Forests Regulation and those under other financing instruments [E] {conclusion IX}, basically through a process of inter-Service consultation and consensual definition of the intended comparative advantages of B7-620 vis-à-vis other financing instruments. Administrative and procedural frameworks should be gradually adapted so as to better accommodate the Forests Regulation’s characteristics and comparative advantages. A further recommendation, that the Commission should integrate where appropriate forest sector development in CSP negotiations [F] {conclusion II, VII} points in a similar direction. Based on the evaluators’ interpretation of the Forests Regulation Art. 4 (4) 2nd indent (i.e. that conception of forest related measures under B7-620 is possible only in accordance with the CSP priorities), omission of forest-related issues in CSPs would not only prove counterproductive from a development policy standpoint, but would also restrict implementation of the Forests Regulation in practice. For this reason, adequate consideration of forest sector development needs to be made a part of CSP negotiations, and CSP negotiators need to be sensitized on forest issues and on the Regulation’s requirements. In this context, forests should not only be considered a (global) public good (consideration of which could be made a mandatory part of environmental mainstreaming). In addition, the socio-economic role and benefits of sustainable forestry (being an economic sector) must be emphasized. Recommendations pertaining to Budget-Line administration It is recommended that the Commission’s programmes should better reflect strategic goals and regional needs for forests in administration of B7-620 [G] {conclusion XVI}. To this end, the Call for Proposals procedure ought to be modified so as to better accommodate varying framework conditions in partner countries, and the respective countries’ needs and priorities. To this end, the CfP should be regionalised, and proactive, strategic interventions by the Commission facilitated. Recommendations pertaining to project implementation The evaluators recommend that the Commission take steps to ensure continuity and impact of promising projects [H] {conclusion VI, XXIII}, to better achieve measurable project success which is currently hindered by short implementation periods. In pursuit of this goal, appropriate mechanisms have to be installed at various levels, including: (i) project selection and design, (ii) monitoring & evaluation, (iii) facilitation of the project continuation, (iv) amendment of the budgetary framework so as to allow a second phase for exceptionally innovative and promising B7-620 projects.

0.6.2 Conclusions and Recommendations for the Environment Regulation

0.6.2.1 Conclusions Relevance (X) The Environment Regulation is coherent with the relevant EC policy framework and authoritative international regimes (e.g. Multilateral Environmental Agreements). The high degree of policy coherence provides an important prerequisite for environmental mainstreaming and donor coordination alike [findings 33, 38, 39, 42]. Effectiveness (XI) Procedural restrictions, brought about by the coming into force of the current financial regulation in 2002, reduce the Commission’s discretion to pursue mainstreaming activities at the global level by way of awarding direct grants for “targeted measures”. This limits considerably the Commission’s room for manoeuvre in addressing newly emerging issues of the international debate, and for exploitation of promising opportunities and project ideas in a flexible manner [finding 22]. (XII) Owing to the fact that successive Strategic Orientations since 2000 focused on



8

the thematic intersection between the Environment and Forests Regulations (e.g. with respect to conservation and sustainable management of natural resources), the thematic composition of the project portfolio blunts the Environment Regulation’s comprehensive mainstreaming strategy to some extent. Urban environment, hazardous chemicals, and environmental aspects of industrial development are mostly missing in the B7-620 project portfolio. It remains to be seen whether a greater thematic balance of environmental issues can be achieved until the Regulation expires in 2006 [findings 1, 2, 8]. (XIII) The financial allocation for environment, and the definition of thresholds for the minimum and maximum size of grants in the Guidelines for Applicants, further restrict effective mainstreaming [findings 3, 26]. Efficiency refer to section “Both Regulations” Impact/Sustainability For practical reasons, environmental mainstreaming has more to do with decision-making, planning, and allocation of resources than with practical implementation of (technical) measures at field level. (XIV) For this reason the Environment Regulation’s impact materialises more at various policy levels (national, as well as decentralised) than in respect of local livelihoods [finding 14]. (XV) Mainstreaming activities generally involve awareness raising and sensitization of various stakeholders across all levels. As such, mainstreaming by definition builds on multiplier effects, and therefore is a finite task [findings 44, 52]. However, this process is not yet complete, and justifies continued efforts, especially at the partner countries’ level. For this, B7-620 provides a flexible instrument.

0.6.2.2 Recommendations Recommendations pertaining to the wider procedural framework and the Regulation’s design It is recommended that the Commission should ensure the continued existence of an Environment Regulation beyond 2006, with a clearer focus on mainstreaming [I] {conclusion X, XIV}. More specifically, mainstreaming efforts should be focussed on partner countries, and BL B7-620 should remain open under the Environment Regulation for small-scale interventions in selected fields of support (e.g. information and knowledge management, participatory policy formulation, etc.). To give effect to the above recommendation, the evaluators suggest that the Commission increase financial allocations [J] {XIII}, commensurate with the Regulation’s broad geographical scope and in view of many partner countries’ still limited progress in mainstreaming environment-related issues. The appropriate Commission Services will need to base their lobbying for greater financial allocations on the outcome of the documentation of mainstreaming progress of B7-620 environment projects. Furthermore, it is recommended that the Commission continue to maintain a broad thematic scope but drop the current requirement for balanced coverage [K] {conclusion XII}, and in addition limit overlaps with forest-related issues [L] {conclusion XII} in practice. In the draft Environment Regulation the significance of urban and industrial environment should be highlighted, and clear criteria for selection of sustainable natural resource management projects under the Environment Regulation should be adopted. These recommendations seek to further improve efficiency and impact of environment projects under B7-620. Recommendations pertaining to Budget-Line administration In the evaluators’ view the Commission should closely follow up on mainstreaming progress [M] {conclusion XV} in order to make informed choices about the Environment Regulation’s future role. To this end clear criteria and indicators should be adopted, so as to better gauge environmental mainstreaming progress at different levels. Further to mainstreaming, it is recommended that the Commission focus the CfP procedure on global and regional mainstreaming needs [N] {conclusion XI, XII, XIV}. To complement the regional focus the Practical Guide on CfP procedures should be modified so as to make possible ad hoc CfP, detached from the annual programming cycle. It would then be possible to pursue strategic thematic foci at a global level more effectively.



9

Furthermore, the evaluators consider it desirable to allow for small-scale mainstreaming interventions in the Guidelines for Applicants [O] {conclusion XIII}, simply by dropping the minimum thresholds for project size defined in the Guidelines’ previous issue.

0.6.3 Conclusions and Recommendations for both Regulations

0.6.3.1 Conclusions Relevance (XVI) Both Regulations place much emphasis on addressing cross-cutting issues of development, which are appropriately reflected in successive Strategic Orientations. However, a certain discrepancy exists between the Regulations’ prescriptions and Strategic Orientations, on the one hand, and practical implementation on the other. Projects tend to address cross-cutting issues more selectively than one would expect given the emphasis placed on them. Therefore, mainstreaming of cross-cutting issues has not yet been consistently achieved and justifies renewed efforts, particularly with regard to African countries [findings 50-52]. Effectiveness (XVII) Given their good performance, projects under B7-620 in practice have a high probability of contributing to the Regulations’ stated objectives – even though interpretation and practical enforcement of eligibility criteria (and structural preferences for certain applicants) result in lost opportunities [findings 6, 20]. (XVIII) However, some weaknesses with regard to project performance persist – particularly in respect of appropriate formulation and verification of objectively verifiable indicators (OVIs) [findings 5, 11, 36]. (XIX) What further limits the projects’ contribution to the Budget Line’s effectiveness are the (unintended) effects of co-funding. Evaluation findings lead to the conclusion that, unless special care is taken to ensure transparency, co-funding by other donors tends to obscure the EC’s role as a major donor [findings 7, 32]. (XX) Furthermore, interpretation and practical enforcement of eligibility criteria, and the structural disadvantages for certain applicants result in lost opportunities [findings 17, 25]. Efficiency Based on the evaluation findings, conclusions about the Regulations’ or Budget Line’s efficiency can be drawn in two respects: first, with regard to the selection of project proposals, and second in respect of how project implementation is supervised, guided and followed up. (XXI) The first Call for Proposal resulted in delays and waste of resources, on both the Commission’s and the applicants’ part. This was mainly because the response to the first Call for Proposals widely exceeded the Commission’s capacity for screening and processing of applications, as well as the available funds [findings 20, 21]. (XXII) Moreover, as regards project implementation, lack of standardised M&E and reporting instruments impedes efficient control, rendering both supervision of projects and documentation of lessons learnt more difficult [finding 29]. Impact/Sustainability (XXIII) Despite the Regulations’ emphasis on innovation through pilot interventions, lack of mechanisms and arrangements for feeding B7-620 projects into geographical programming hinders achievement of impact and sustainability of pilot measures. Alternative options, such as taking over of project activities by national actors or other donors, have so far not been capitalised upon, mainly because EC Delegations had neither the mandate nor the means to facilitate donor coordination in respect of B7-620 projects prior to deconcentration [findings 36, 45]. (XXIV) So far, the Commission Services have not capitalized on lessons learnt and experiences, and hence have failed to exploit opportunities for institutionalised learning. This situation will probably improve now that deconcentration is complete and regional information-exchange between B7-620 projects has been institutionalised [finding 29]. Coordination, Coherence and Complementarity Various coordination arrangements are prescribed by the Regulations, as well as by other applicable legal norms. (XXV) The Commission Services at HQ level coordinate their activities internally, as well as vis-à-vis EC Member States and other donors [findings 34, 35]. (XXVI) Despite the framework in place, other donors do not benefit from experience and lessons learnt from B7-620 projects, partly because coordination and reporting measures focus on administrative and procedural aspects instead of on subject-matter information [finding 37]. (XXVII) So far,



10

adequate coordination (and, consequently, also coherence and complementarity) is not assured at partner-country level [finding 36].

0.6.3.2 Recommendations Recommendations pertaining to Budget-Line administration Three recommendations stand out as the most important, namely improve efficiency of CfP procedures [P] {conclusion XXI}; provide applicants with timely information on the CfP [Q] {conclusion XXI}; and drop the practice of excluding certain potential applicants [R] {conclusion XX}. These recommendations have in common an objective of improvements in project selection procedure. This could be brought about in several ways: (i) defining more clear-cut, regional priorities, so as to limit the number of proposals; (ii) investigating the possibility of adopting a two-step “concept paper” approach, (iii) adopting and monitoring clear-cut deadlines and milestones of CfP procedure in a more proactive manner, (iv) ensuring that all categories of partners listed as eligible under the Regulations are able to apply for funding. In pursuit of the above, it is further recommended that steps be taken to improve the role of southern partners [S] {conclusion XIX, XX} by defining criteria for more equitable partnerships between northern and southern partners in the Guidelines for Applicants; and to provide EC Delegations with adequate resources for the Budget Line’s administration [T] {conclusion XVII} in view of their increased responsibilities following deconcentration. With respect to the latter recommendation, it further appears desirable to improve Delegations’ technical capacities [U] {conclusion XXIV, XXVI, XXVII} and expertise so as to better match their increased workload. To this end, regional focal points ought to be established and supplied with appropriate sectoral expertise to enable them to act in a relay role for information gathering, documentation and dissemination of lessons learnt through workshops, publications and field visits. Recommendations pertaining to project implementation Despite the projects’ good overall performance, the implementation phase warrants further improvement through addressing certain, remaining bottlenecks. To this end, the evaluators recommend that steps be taken to improve the quality of OVIs [V] {conclusion XVIII, XXII} so as to be able to assess project effectiveness, efficiency and impact more reliably. This recommendation includes continuous adaptation and control of OVIs by the projects. The Commission might consider drafting a specific manual to provide guidance on the formulation of forest- and environment-related OVIs. On the issue of co-funding, it is recommended that the Commission make EC contributions more transparent and visible [W] {conclusion XIX}, with a view both to better gauging the Budget Line’s efficiency and to ensuring the EU’s visibility as a major donor. To this end standard contract forms need to include a clause obliging contractors to disclose any additional funding received from third parties in excess of the fixed co-funding ratio. It would further seem necessary to harmonise and improve existing M&E instruments [X] {conclusion XXIV} so as to ensure enhanced transparency and comparability of project progress. To this end, monitoring undertaken on the contractors’ initiative should be standardized and based upon OVIs, and the Commission Services should adhere more uniformly to the Regulations’ evaluation requirements in Budget Line administration. The latter recommendation would also provide a precondition to better capitalize on lessons learnt [Y] {conclusion XXVIII, XXII, XXIV}, based on the use of a standardized reporting format drawing on impact-oriented OVIs. A given project’s specific characteristics (e.g. in respect of cross-cutting issues) should be captured through an encoding system (reflecting the key objectives mentioned in the Regulations).



11

1 The Evaluation Framework

1.1 Objective and scope of the evaluation Since 2000, the two separate EC Regulations4 have jointly provided a legal basis for disbursement of EC funds from the merged Budget Line B7-620. Both Regulations require that a comprehensive evaluation be submitted to the European Parliament by November 2004. Acting on its legal mandate, the EuropeAid H/6 Evaluation Unit scheduled a joint evaluation of the Environment and Forests Regulations to start in 2003. This report presents the outcome of the evaluation, the objective of which is as follows5: “…to account to the legislative authority and wider public for the funds expended under the Environment and Forests Regulations; and in addition to provide the relevant external co-operation services of the European Commission:

• firstly with an overall independent and accountable evaluation of the activities financed by the EC under the two Regulations in the context of overall Community development co-operation;

• secondly with proposals for the future of the Regulations, including their possible modification or termination as stipulated in Articles 12 of 2494/2000 and 11 of 2493 regulations. “

The evaluation concentrates on the forest and environment measures or actions financed under BL-620. However, forest and environment-related activities under other financing instruments are examined on a more aggregated level, mainly in respect of coherence, coordination and complementarity with BL 620 projects. The present report covers mainly the period between 2000 and the first trimester of 2004. Within this period, two successive Strategic Orientations were completed, and two Calls for Proposals launched by the Commission Services responsible. Furthermore, the conditions which existed prior to the coming into force of the two Regulations, as well as information on the recently completed project selection under the second Call for Proposals (October 2004), are also taken into account. The project sample for the field phase consists of 35 projects in 16 countries (Belize, Brazil, Cameroon, Cambodia, Costa Rica, Ethiopia, Guatemala, Honduras, India, Indonesia, Kenya, Nicaragua, Panama, Peru, Senegal, Vietnam) under the 2000, 2001 and 2002 budgets. Projects funded from the 2003 and 2004 budgets were only selected in the last quarter of 2004 and hence could not be included in the sample.

1.2 The evaluation process and its phases

The present evaluation was conducted by a core team of five experts with professional backgrounds in forestry and related aid mechanisms, environment and related aid mechanisms, forest policy and law, environmental policy, social forestry and gender. It was assisted by a junior expert charged with the preparation of databases and documentation. During country missions in selected partner countries, the core team members cooperated with national and regional experts. The evaluation process was subdivided into four individual phases:

the inception phase, to clarify the key evaluation issues and the overall framework of the analysis, to review the tentative Evaluation Questions (as provided in the ToR) for subsequent approval by the SG, and to agree on the work plan, budget and evaluation team for the desk phase;

the desk phase, to reconstruct the hierarchy and logic of the objectives of the Commission´s interventions on environment and forests, to define judgement criteria

4 EC Reg. No. 2493/2000 “On measures to promote the full integration of the environmental dimension in the development

process of developing countries”; EC Reg. No. 2494/2000 “On measures to promote the conservation and sustainable management of tropical forests and other forests in developing countries”

5 ToR, part III, 15th para, p.8 see Annex 1



12

and indicators for each Evaluation Question and to propose the methodology for the field phase

the field phase, to gather data on site in developing partner countries, to collect additional information and cross-check previous outcomes of the desk phase; and

the synthesis phase, to consolidate and discuss all findings in context, and to deduce conclusions and recommendations from this basis.

1.3 The Evaluation Questions The evaluation was structured around detailed Evaluation Questions6 which, after having been drafted initially by the Evaluation Unit with the support of the Steering Group, underwent discussion, review and reformulation during the inception phase and early stages of the desk phase. Because both sets of EQs were very similar and in some cases even identical, they were streamlined and merged by the evaluation team in agreement with the Steering Group. Even so, free-standing conclusions and recommendations have been formulated for each Regulation separately. During the evaluation’s desk phase, one core task was the definition of judgement criteria and their accompanying indicators for each Evaluation Question, based upon the reconstruction of the Regulations’ intervention logic. The selection of criteria for each Evaluation Question was mainly guided by the Terms of Reference (in particular by the listing of “issues to be addressed”). The Evaluation Questions’ complexity sometimes calls for the use of “proxy indicators” which approach a given aspect from various angles. In such cases a set of indicators for a given criterion has been used. The drafting of indicators was additionally influenced by the team’s judgement of the availability, feasibility, and efficiency of different data-collection methods. The entire structure of the criteria and indicators for each Evaluation Question is shown in Annex 6. The review of and modifications to the Evaluation Questions, the selection and formulation of criteria, and the selection of indicators and data-collection instruments, have been presented as part of the Desk Report and Field Inception Report, and were subject to discussion and subsequent approval by the evaluation Steering Group. The Findings chapter is structured round the approved EQs with their respective criteria and indicators. Findings are consecutively numbered with figures in square brackets. At the level of findings the evaluators deliberately avoided value judgements. The Conclusions chapter is structured according to the five key DAC Evaluation Criteria (Relevance, Efficiency, Effectiveness, Impact and Sustainability) and 3Cs policies7. Conclusions relate direct to individual findings, and embody the evaluators’ professional judgment; they are presented separately for each Regulation (where appropriate), and then jointly for both. Recommendations are presented in a similar fashion (as requested by the ToR), separately and then jointly. Each Recommendation is deduced from a combination of conclusions, as highlighted by the respective references for each Recommendation.

1.4 Methods of data collection and analysis

Document analysis:

This method was used for the analysis of the Regulations’ wider policy framework during the desk phase of the evaluation. A first selection of relevant source documents was supplied by the Commission Services, and further documents were identified through internet searches (Annex 3). All selected documents were first read and then classified as either official policy documents, other EC documents, or background information originating from outside the EC institutions. Documents were then systematically searched for references to (i) environment and (ii) forests in the context of (sustainable) development cooperation. Relevant references were further classified as either (a) description or analysis of manifest problems, (b) strategies or objectives (including

6 In their current form and wording (see ANNEX 6), the EQs have been endorsed by the EC Steering Group 7 Coherence, complementarity, coordination



13

international processes or initiatives), or (c) concrete recommendations for action. All findings were recorded using the mind-map technique8. For each document “thematic foci” were identified, based on statements about response strategies or objectives relating to identified problems. Such foci were classified and listed as forest-related, environment-related, or intermediate. All documents thus processed were listed in a tabular matrix9 for enhanced comparability of the findings.

Exploratory meetings with Commission services’ staff in Brussels

At the beginning of the desk phase the core team members jointly visited the relevant Brussels-based Commission Services. In pairs they conducted semi-structured, focused interviews with members of AIDCO F4, RELEX, DG Research and others. This exercise served to solicit the views and insights of those involved in Budget Line administration. The outcome of the interviews was recorded in detail, and later transcribed into sets of protocols. The results have been fed into both the desk phase and the design of the field phase. During the final preparation of the synthesis report, the protocols were again used to extract information and to verify findings obtained during the field missions. For the listing of persons contacted, refer to Annex 2.

Approach to the identification of relevant environment and forest related measures implemented by the Commission (the “project portfolio”).

To identify relevant environment and forest related projects (both B7-620 projects and projects funded from other financing instruments), the CRIS database was screened10. For this purpose, DAC codes were selected, under which – according to the team’s professional judgement – forest and environment related projects seemed likely to occur. The resulting large sample of projects was further narrowed down in a second step, namely by excluding interventions which, judged by their title, did not obviously relate to environment and forests.11 Working with the CRIS database, the evaluation encountered a number of difficulties. Entries in the database may be made in different languages. Second, numerous misspellings impede the use of efficient search tools for a quick screening of the database. Third, several cases of double-listing came to light, but on the other hand some information was missing. In order to identify a complete and comprehensive list of relevant B7-620 projects, data collected from the database were compared to a list provided by the AIDCO F4 task managers. Two missing projects were thus identified. The resulting list contained all the contracts signed under Budget Lines B7-6200 and B7-6201 (merged in 2000 into the existing joint budget line). The following data were listed: contract number and title, sector (ENV/FOR), date of signature or payment, beneficiary, total financial commitment, payments made and the resulting balance. No projects beyond the budget year 2002 were identified, because the subsequent Call for Proposals occurred outside the evaluation’s time frame. The list was then referred back to AIDCO F4 staff, who 1) confirmed that it was complete; 2) provided additional details on the existence of monitoring or evaluation reports. They also amended some information extracted from CRIS, especially concerning the names of task managers, dates of contract signature and disbursements and the countries where the projects were based (especially for the regional projects). The resulting final list contained 97 interventions and constitutes the basis for this evaluation. The list of these projects is included in Annex 4. It has been structured according to the project’s origin (direct, call for, and targeted).

8 Mind mapping is a tool for structuring and illustrating one´s thoughts. Drawing a mind-map helps to systematize the

analysis of a subject and to visualize logical interrelations in a hierarchical structure 9 Refer to ANNEX 11+12 10 The Common RELEX Information System (CRIS) databases, the CRIS Saisie and the CRIS Consultation, are the main

sources of information on the European Commission’s programmes and projects world wide. They list EC interventions consecutively numbered, by name and region, intervention-sector (DAC-codes), and with detailed information on the projects’ technical as well as financial implementation-status (financial allocations as ‘committed’ and ‘paid’).

11 Although individual contacts in the relevant Commission Services expressed doubts about the accuracy and reliability of data recorded in CRIS, the Evaluation Unit stressed that, since it is the Commission’s official data-base, the evaluators should refer predominantly to CRIS. It should be noted, though, that a number of contacts at the Commission HQ in Brussels were found to maintain and use other, purely internal databases for their day-to-day administrative work.



14

Statistical evaluation of the project portfolio under B7-620:

Once the final list of projects had been obtained12, the evaluators relied on information sources other than the CRIS database (contract documents with ToR, and project sheets, supplied by the respective task managers). A new (internal) database was designed specifically for the evaluation, based upon an in-depth screening of the available project documentation. In doing so, the evaluators applied the following categories: geographical distribution, grant-beneficiary (contractor), financial volume, target group, mode of delivery, thematic foci of either “forest” or “environment” projects13.

On-site assessment of a sample of projects:

The on-site assessment of a sample of projects provided the core of the field phase. The selection of partner countries and projects followed an iterative approach with the aim of achieving a balanced sample commensurate with the identified project portfolio of 97 projects. In so doing the evaluators considered factors such as thematic foci, geographical distribution and financial allocation. Additionally practical considerations were made, including accessibility of project areas and feasibility of a field visit within the given timeframe or evaluation budget. The final sample included 35 projects (see Annex 15). The on-site assessment approach combined various instruments such as semi-structured interviews, document analyses, discussions with resource persons, site observations and others. To better structure the exercise and systematise documentation of the findings, a project assessment sheet (PAS) was used. The PAS is an instrument for project classification, standardised recording of basic data14, and inductive15 assessment of the projects – first for each of the DAC criteria, and then as a summary judgement of the project’s performance. As such, the PAS represents the “condensed” outcome of the on-site project assessments, and reflects the evaluation team members’ professional judgement. 28 PAS were produced in the course of the field phase. In respect of the remaining seven projects, preparation of PAS was judged impossible or useless for the following reasons: in one case, the project had been completed, and the entire project infrastructure dissolved; while the remaining projects either had commenced implementation only very recently, or had not started at all. In keeping with the approved evaluation methodology, the field missions’ outcome was dealt with in the following ways:

- Each country mission concluded with a debriefing addressed to the respective EC Delegation. Because these debriefings served only to inform individual EC Delegations about highly country-specific observations, there was no need to standardise the debriefing approach, or adopt a uniform reporting format. Consequently, the degree to which debriefings were captured in a minutes-of-meeting style document, varied considerably between country missions.

- For each country mission, the experts in charge prepared in-depth analyses, which served as internal working documents for the team during successive phases of the evaluation. Initially, separate “internal country reports” were produced for a number of missions, which applied a common approach and reporting format (based upon the Evaluation Questions), and streamlined the team members’ individual findings and observation. Following respective requests by the evaluation SG, the team later merged the first set of country reports into one “Field Phase Report – 1st batch”, and subsequently integrated the outcome of the second batch of country missions into it, so as to present one, consolidated Field Phase Report to the SG.

Written poll:

The written poll addressed individual projects as well as EC Delegations in partner countries. For this purpose two sets of questionnaires were conceived. Both sets integrated different questioning

12 for a full listing of projects, refer to ANNEX 4+5 13 According to Art. 4 FOR and Art. 3 ENV 14 Such as, for example, starting date of implementation, duration of implementation, budget etc. 15 The PAS provides sets of indicators (each with ordinal scales), which enable summary assessments in regard of a given

project’s quality of design and planning, relevance, effectiveness, efficiency, impact and sustainability.



15

techniques, such as structured and non-structured questions, multiple choice nominal answers, ordinal ratings, and others. As for some questions more than one answer (multiple choice) was allowed, the total number of answers to such questions may exceed the total number of respondents16. The two sets of questionnaires were tailored to reflect the project implementers’ individual experiences, as well as the EC Delegations’ accumulated experience with B7-620 projects. All 97 projects within the project portfolio received the questionnaire, as did all EC Delegations in countries where B7-620 projects are implemented. 40 projects responded to the written poll, as did almost all Delegations (28). Dissemination of the questionnaires was done by the Brussels-based Evaluation Unit.

Overview

The following table illustrates the step-wise evaluation process, and presents phases, methods of data collection, the output and respective SG meetings in a chronological sequence.

Phase Steps (chronologically) Internal documents

SG meetings

Reports

Team-building and internal streamlining of working modalities

Management and consultation of reference literature and background documentation

Launch

Note

Gathering, and subsequently consolidation of information about individual activities and interventions funded under Budget-Line B7-6200

First round of exploratory meetings with relevant EC services in Brussels Interview minutes

Intensive study of the two regulations, and subsequently reconstruction of the procedural and contextual intervention logic underlying EC Regulations No. 2493/2000 and 2494/2000

Mind-Maps

INC

EPTI

ON

PH

ASE

Detailed interpretation of the Evaluation Questions and drafting of respective judgement criteria.

Launch meeting

Inception meeting

Inception Note

Second round of exploratory meetings with relevant EC services in Brussels Interview minutes

Reconstructing the procedural and contextual intervention logic underlying EC Regulations No. 2493/2000 and 2494/2000

Systematic analysis of a selected number of relevant policy and procedural documents, to assess the wider policy context

Mind-Maps

Compilation of a full list of projects financed to date under the Budget-Line B7-6200, since the entry into force of the current regulations

Gathering and subsequently consolidation of information about individual activities and interventions (CRIS, project summary sheets, project documentation),

Establishment of a data bank with project information and a quantitative analysis of the 97 projects

Project database

Screening of available CSPs and RSPs, and compiling a matrix for quick reference

DES

K P

HA

SE

Detailed interpretation of the Evaluation Questions, formulation of judgement criteria, indicators and source of verification / methodological approach

Desk phase

meeting

Desk report

Final selection of 16 countries/ 28 projects (sample), and national experts

Fine-tuning of methodology for data collection on country and project level

Field inception

report

Launch of written polls to project contractors and Delegations

Screening of country specific framework conditions

FIEL

D P

HA

SE

On-site inspections and interviews with key-stakeholders (EC Delegations, donor representatives, target groups, project staff, etc.) – documentation of findings in Project Assessment Sheets (PAS)

PAS, internal Country Reports

Field phase

meeting

Country Debriefing

Notes

Field visit report

16 For a detailed presentation of the distribution of findings, refer to ANNEX 13+14



16

Phase Steps (chronologically) Internal documents

SG meetings

Reports

Evaluation of the written polls QuestionnairesSummaries

Screening and analysis of environment & forest interventions under other financing instruments (CRIS)

Matrix

Extended in-depth analysis of certain legal issues Internal papers

SYN

THES

IS P

HA

SE

Integration of outcome from previous phases through synthesis workshop by the team

Internal documentation

Synthesis meeting

Synthesis report

1.5 Methods of Judgement

The team’s approach may be characterised as a step-wise process of logical reasoning, proceeding from the level of individual findings which are then generalised and enriched by the teams professional judgement (i.e. conclusions) and finally transformed into concrete recommendations. Answering the Evaluation Questions prior to the formulation of aggregate conclusions and recommendations also helped standardise and streamline the team members’ individual perceptions and interpretations. In assessing the practical implementation of the Environment and Forests Regulations, the team has addressed different aspects which needed to be clearly distinguished. One must first distinguish between three assessment levels, (i) design of the Regulations, (ii) administration of Budget Line B7-620 by the Commission Services, (iii) implementation of concrete measures at partner country level. Furthermore a distinction has been made between the question of whether measures and procedures prescribed in the Regulations are considered appropriate by the evaluation team from a contextual point of view, and the question of whether the implementation routines followed by the Commission Services are in line with the prescriptions and intentions of the Regulations. Simply put: whether the “right things” are done, and whether “things are done right”. It must be noted that some of the issues addressed are open to interpretation, including different legal interpretations.

1.6 Approach to the presentation of findings, conclusions and recommendations

For the purposes of the presentation and subsequent discussion of findings, individual findings have been structured in accordance with the criteria and indicators selected for each Evaluation Question. At this stage no a priori distinction has been made between the two Regulations, nor are findings which apply equally to both Regulations presented separately. However, for each Evaluation Question, a summary answer is presented for both Regulations, as well as separately for the Environment and Forests Regulations. Conclusions & recommendations are presented separately for each Regulation, and then for both Regulations. As regards the presentation of conclusions and recommendations, one needs to distinguish clearly between different stages of aggregation. Findings emanated from the Evaluation Questions. Conclusions in respect of individual Evaluation Questions represent the “meso-level” of aggregation. By applying a rating system17 to these conclusions, the evaluation arrives at a summary judgement (“macro-conclusions”) of the Regulations’ merit and demerit as judged by the

17 Based on ordinal scaling



17

DAC criteria and 3Cs18. The aforementioned stepwise approach ensured a stringent causal relationship between findings, conclusions and recommendations. Conclusions and recommendations relate to three different levels: The first level reflects the Regulations’ design as legally binding norms, as well as the design of the legally binding procedural framework (such as, for example, the Financial Regulation). As such, conclusions and recommendations at the first level deal with the Regulations’ contextual as well as procedural intervention logic, their binding force, their relation to the overarching policy framework, and so on. The second level reflects the administration of Budget Line B7-620 (and the respective project portfolio) by the Commission Services. Conclusions and recommendations therefore relate to procedures for identification, selection, contracting, and oversight of project implementation. The third level reflects implementation of individual projects in the developing partner countries. Conclusions and recommendations at this level therefore draw on observations relating to individual projects’ quality of design, relevance, efficiency of implementation, impact, sustainability, and so forth.

2 Evaluation Background

2.1 Political, Economic, Social and Cultural Dimensions affecting Environment and Forests in 3rd World Countries

According to the Rio Declaration on Environment and Development (Principle 4), environmental protection constitutes a major component of sustainable development. In a global perspective, about 90 % of the 1.2 billion people living in extreme poverty directly depend on natural resources and a healthy environment for their daily survival19. For these people, natural resources mean shelter, space, food, medicine, fodder, building materials and fuel. This holds true particularly for forests which, given their multifunctional value, are a core element of rural livelihoods. In a global perspective, forests mitigate climatic extremes, harbour the lion’s share of terrestrial biodiversity, and help both to stabilize water resources and prevent drought and desertification. In a wider cross-sectoral perspective, “…integrating the environmental dimension in the development process…” (as stipulated by the Environmental Regulation) comprises preservation of environmental media (soils, air, water), and environmental management (SNRM, urban and technical environment, rehabilitation of environmental quality for future sustainable use). The underlying causes of environmental degradation and deforestation are inseparably linked with socio-economic and political as well as cultural parameters - of which poverty is the most significant one. Rural poverty often induces unsustainable exploitation of natural resources and increases environmental risks. Uncontrolled population growth, unhelpful political and legal framework conditions, market failures and the weakening of the established traditional, indigenous social fabric, as well as the respective norms and codes of conduct, exacerbate the problem.

Consequently, environmental protection and nature conservation in the development context are regarded as conditions for sustainable development. Often SNRM for human needs achieves environmental goals at the same time (and vice versa). The concept of Sustainable Development (SD) helps to achieve synergies between environmental protection and sustainable natural resource management; that is to say, within SD logic all development must take account of and integrate ecological, economic and social dimensions.

18 DAC-Criteria: relevance, effectiveness, efficiency, impact, sustainability; 3Cs: coordination, coherence, complementarity 19 UNFF (2004): Report on the fourth session (6 June 2003 and 3 to 14 May 2004). ECOSOC official records, supplement

No. 22. New York.



18

2.2 Synthesis of the EC’s Strategies & Programmes An analysis of the EC’s relevant development and co-operation strategies was carried out, with a screening of authoritative source documents20 across all levels of operationalisation, from the

level of EC policy documents to the level of practical guidelines. The Environment and Forests Regulations are located on a logical continuum half-way between (strategic) policy guidance and (practical) implementation. As legal norms, they govern disbursement of EU funding from Budget Line B7-620 and are directly binding on the Commission Services involved in the Budget Line’s administration. In a political sense, the Regulations also serve as transmission belts for translating policy objectives into one segment21 of EC’s environment- and forest-related development cooperation. A common characteristic of relevant EC strategies and programmes in respect of environmental protection and forest sector development is that they relate, or more precisely are subordinate, to a development cooperation rationale22. Following the development perspective, forests and environmental measures must serve human needs and interests. Therefore, enhanced environmental protection or optimised forest management are regarded as means to an end (achievement of sustainable development), rather than objectives in themselves. Accordingly, the “intervention logic” underlying sectoral EC documents such as the Environment and Forests Regulations relate directly to the Development Policy of the European Community.

Previous analyses23 during the evaluation’s desk phase established that

1. The examined EC policies in their entirety provide for a logical hierarchy 2. The policy framework is coherent with, and relevant to, the international dialogue

and initiatives in respect of environmental and forest-related issues (MEA, international forest regime, etc.)

3. There exist separate policy frameworks for forest- and environment-related activities, which guide the various financing instruments, including Budget Line B7-620

4. All policy documents consistently reflect environmental mainstreaming as a cross-cutting issue

5. A broad thematic overlap between forest-related and environmental issues in the policy framework is apparent

6. The Environment and Forests Regulations are appropriately embedded in a wider contextual framework of EC policies and strategies, i.e. they reflect constituent strategies and objectives of both sectoral and crosscutting policy documents

7. The Forests and Environment Regulations address similar issues 8. Both Regulations display a comparatively high level of operationalisation, i.e. they

present a logical hierarchy including policy goals, purposes and activities, and means of verification

9. Even though other financing instruments may also be broad in scope, the combination of (i) the decrease in annual financial allocations, (ii) the Regulations’ broad thematic scope, and (iii) the extension of the Forests Regulation’s geographical scope, reduces the Budget Line’s impact and visibility. This observation holds true if the full scope of activities is to be addressed (as in the evaluators’ interpretation the Regulations’ texts suggest), and the Commission’s freedom to select thematic foci is thereby restricted.

20 For the purpose of the present evaluation, documents have been selected which communicate, guide, direct or govern

the actions of either the EC, the European Council, or Parliament with respect to development cooperation, environmental protection and forest sector development.

21 Besides BL B7-620 environment and forest projects are also funded under other financing instruments 22 A socioeconomic / socio-political focus on poverty alleviation and improvement of (rural) livelihoods of target groups in

developing countries. 23 Reconstruction of the Intervention Logic of the Environment and Forests Regulations; Reconstruction of the hierarchy,

logic, related assumptions and intended impacts of the objectives of the EC interventions on environment and forests

Regulations

EC Communications

EC Sector Policies

EC Development Policy/

External Relation Policy

Strategic Guidelines

Priorities under Annual Workplan/Call for Proposals

Selection of Proposals

Policy Guideance

Operationalization



19

Africa36%

Asia20%

Global7%

Latin America35%

Pacific2%

10. The strategic orientation as reflected in the guidelines is in line with the Regulations’ thematic scope, although direct references are in some cases difficult to identify

11. Given the similarities of strategic orientation and focal areas in two successive periods (2000/2001 and 2002/2003), it remains doubtful whether all activities prescribed in the Regulations can still be addressed during their remaining period of validity.

12. Newly emerging issues, such as implementation of MEA or FLEGT, have been integrated flexibly into the strategic orientation

13. Civil society participation is a key objective of both the Environment and Forests Regulations

3 General characteristics of the Budget Line project portfolio Under the two Regulations a maximum amount of up to €342 million has been earmarked for expenditure over the total duration (year 2000 to 2006), with respective shares of €93 million for the environment and €249 million for the forests component. Between 2000 and 2002, 97 projects were financed, totalling €116.376.139. In addition €1.559.733 was spent on administrative costs (evaluation of proposals etc.). In spatial terms, Africa received the largest share of support from the budget line, followed by Latin America, Asia and the Pacific Region. The below Figure 1 illustrates the geographical distribution of budget line support. By and large, this geographical distribution of funds also holds true in terms of the number of projects implemented (Annex 5).

Figure 1: The geographical distribution of Budget line commitments

Global projects under the E&F budget line account for 7% of all interventions. It must be noted that this budget line is the only EC financing instrument to support global projects in the above mentioned fields. Selection procedures were modified during the evaluation period. Project proposals earmarked for the 2000 fiscal year were selected by decision of the Inter-service Steering Group, and are hereafter referred to as “direct”. With effect from 2001, the “Call for Proposals” has been the sole selection procedure (see chapter 3.1). Further projects representing special interests of the Commission have been directly selected, and are labelled as “targeted”.

Table 1: Classification of the 97 grant contracts according to budget year, programme and applied selection procedure

Budget 2000 Budget 2001 Budget 2002 Total (CFP/Direct) CFP Targeted CFP Targeted

ENV 14 9 1 6 5 35 FOR 14 20 2 20 6 62 Total 28 29 3 26 11 97

Accounting for slightly more than 70% of the total financial commitments under budget line B7-6200, NGOs are the single most important group of contractors (both Regulations). Of these, northern NGOs are the most important group, receiving the largest share of financial commitments with percentage shares varying between 54,7% (ENV Reg.) and 49,6% (FOR Reg.) respectively. Most of the southern NGOs benefiting directly as contractors from the Regulation commitments operate in Asia (38 %) and Latin America (22,4%), owing to the well developed NGO landscape in these regions. The funds contracted directly to African NGOs account for only 10% (both Regulations).24 24 However, in many cases projects are executed in partnership with local NGOs who contribute to project implementation



20

Africa35%

Asia23%

Global2%

Latin America39%

Pacific1%

International organisations apparently play an important role in the implementation of “targeted” global or regional projects under the Environment Regulation (21,2%), as opposed to a mere 9,8% under the Forests Regulation. Public agencies and research organisations are not among the contracting organisations for the Environment Regulation. The financial commitment per project varies around an average of € 1.1 Mio. Projects exceeding €2 Mio. mainly occur in Africa or Asia, and – for the most part – were found to be regional or targeted interventions (e.g. Brazil). It should further be noted that the EC contributes more to northern NGOs than to those located in (southern) developing countries, even though the latter’s co-financing contributions are higher.

3.1 Project portfolio of the Forests Regulation Between 2000 and 2002 sixty-two projects representing a total commitment of € 86.393.428 were approved under the Forests Regulation. Projects are currently implemented in 27 countries, with an additional 14 regional and 2 global projects.

Figure 2: The geographical distribution of Forests Regulation commitments

Projects were found to be evenly distributed across Latin America (21), Africa (20) and Asia (18). In comparison, interventions in the Pacific region (3) play a marginal role. Whereas seven regional projects have been identified in Africa, only three occur in Asia and Latin America. Although the geographical focus of the former Tropical Forest Budget Line (B7-6201) has been opened up considerably under the new Forests Regulation (so as to cover all types of forests in developing countries), this has had little apparent effect on the implementation level - with just one project launched outside the tropics, in Kyrgyzstan. Conservation and restoration of forests for their ecological value is a focal issue of the current project portfolio of the Forests Regulation, in terms of the funds allocated. It is particularly important in Africa (39%) and Latin America (36%), as opposed to Asia where only 24% of the allocated funds fell into this category. The category “knowledge and information generation and management” is a clear focus for projects implemented in Africa. Of the funds committed under the Forests Regulation, 40% go into this type of project, including five out of seven regional projects. Aside from supporting education and training, promoting “good governance” in the forest sector is a major concern in this category. Sustainable forest management is a major concern in Latin America (42%) and Asia (22%), while in Africa a mere 11% of the allocated funds are used to promote SFM. With an allocation of only 11% of the total financial commitments under the Forests Regulation, utilisation of forest products and downstream activities clearly plays a lesser role. Projects implemented under the Forests Regulation address the following target groups: local communities (50%), local decision makers (18%), national policy makers (13%), technical and administrative staff (6%), private small and medium enterprises (6%), research organisations (4%) and local NGOs (3%).

3.2 Project portfolio of the Environment Regulation During the evaluation period (2000-2002), thirty-five projects - representing a total commitment of €29.982.711 - were approved under the Environment Regulation.



21

Africa39%

Asia12%

Global21%

Latin America26%

Pacific2%

Figure 3: The geographical distribution of Environment Regulation commitments

Africa (12 projects) is the single most important recipient region, followed by Latin America (10 projects), and Asia (4 projects). Among them are thirteen projects with regional foci – which are mainly located in Africa (8) and Latin America (3), while only one project each was found in Asia and the Pacific. In addition, nine with global foci have been approved, one of which was never signed25. In terms of the financial allocation to certain thematic foci, Conservation of natural resources (41%) stands out as the single most important topic currently covered by the Regulation’s portfolio. Mainstreaming environmental issues (20%) is taken up by five global and two regional projects, dealing mostly with capacity building of national and local policy-makers, or - in one case - aiming to raise awareness and capacities of EC staff and other stakeholders in the ALA/MED region. Sustainable management of natural resources accounts for 16%, followed by implementation of international environmental policy frameworks (9%), improvement of urban environment (8%), and improvement of environmental aspects in other sectors (6%). Although local communities (29%) are still the primary target group, local decision-makers (22%) and national politicians (22%), as well as technical and administrative staff (13%) and research organisations (7%), are important. This is basically a result of the relatively high number of global and regional projects dealing chiefly with policy formulation and framework setting.

4 Findings In the following sections the individual findings are presented. The main section has been structured in line with the Evaluation Questions, for which sets of judgement criteria & indicators (officially approved by the Steering Group for the evaluation) had been defined during the earlier inception phase. The findings’ sub-headings reflect the structure of criteria and indicators defined for each Evaluation Question.

4.1 EQ 1a: Achievement of the stated objectives

EQ1a: How far, how effectively and how efficiently have the Commission-supported activities funded under the Environment/Forests Regulations contributed to the achievement of the stated objectives under the two Regulations?

Rationale/Introduction: Evaluation Question 1 is one of the most comprehensive of eight EQs, because it relates to the entirety of the Regulations’ objectives, as well as two out of five DAC criteria (effectiveness, efficiency). EQ1A captures no less than the overall performance of Budget Line B7-620. Firstly, the extent to which B7-620 projects further the regulations’ stated objectives (“… how far…”) depends on the projects’ adequately reflecting the regulations’ thematic scopes. Secondly, since both regulations emphasize the “innovative” or “pilot character” of B7-620, the projects’ effectiveness also depends on how well they meet this requirement. Thirdly, a good overall performance of B7-620 projects is a precondition for achieving the Regulations´ objectives.

Findings

25 Clean Air Initiative, World Bank (budget 2000)



Know ledge and information

generation and management

29%

Development of national and international

policy framew orks

1%

Conservation and restoration of forests for its ecological value

32%

Utilisation of forest products

and dow nstream

activities11%

Sustainable forest

management27%

[1] The project portfolio is coherent with both Regulations’ respective thematic scopes, but unbalanced Both Regulations stipulate in their respective articles (4[1] FOR and 3[1] ENV) that “…the activities to be carried out under this Regulation shall address in particular…”. The interpretation of the term “in particular” gives rise to discussions: does it mean that all thematic issues mentioned have to be addressed in more or less equal terms but that additional issues are accepted, or should it be interpreted as broad coverage of thematic issues by the Regulation, leaving selection to the Commission Services’ discretion? An attempt at interpretation from the team’s view can be found in Annex 10. Forests Regulation: According to the focus stated in the different project objectives26, the forest portfolio of 62 projects has been classified corresponding to the main thematic issues stated in clauses (a) to (e) of Article 4 of the Regulation.

Figure 4: Forests Regulation - project distribution by thematic categories, 2000-2002 commitments

Three categories stand out with approximately even shares. The two remaining categories, namely (i) utilisation of forest products and downstream activities, and (ii) development of national and international policy frameworks, have markedly smaller percentage shares in the portfolio of forest- related projects. “Conservation and restoration of forests for their ecological value”, which accounts for roughly one third of the projects, reveals considerable thematic overlap with the Environment Regulation which addressing similar issues.

Under each of the categories the Regulation mentions subcategories. Within their scope of activities, however, the projects cover several subcategories27 within different categories. For example, 55% of projects classified as “sustainable forest management” refer to “utilisation of forest products and downstream activities” as an important component within the scope of their activities. In this respect the projects under the Forests Regulation were found to relate simultaneously to an average of 3 to 4 of the categories under Art.4 of the Regulation. However, the following subcategories of certain themes are not addressed at all, or only by a single project: wood for energy, contribution to combating soil erosion, watershed management, support to the private sector, equitable trade.

26 The entire analysis is based upon the projects’ description a

implementation contracts... For a summary overview of individuand Article 4, clause 1 of the Forests Regulation, refer to tables

27 In Annex 5 a classification of the project activities according to th

the regulation is presented.

Example of Regulation overlap Of the four projects visited in India, only one (71) wasfunded under the Forests Regulation. However, thisproject contained so many “environmental” aspects thatit could just as well have been submitted under theEnvironment Regulation. On the other hand, project 25 –with its very strong focus on Non-Timber ForestProducts (medicinal and aromatic plants) - could veryeasily have been chosen to apply for funding under theForests Regulation. Please note: Figures in (brackets) reflect the internalnumbering of projects selected for evaluation (Annex 4)

22

nd design, as presented in Annex 1 of the respective al projects contribution to the achievement of Article 3

in Annex 5.

e subcategories mentioned in Art.4, para (a) to (e)[0] of



23

Environment Regulation: The priorities mentioned in Art. 3 (indent 1 to 16) have been grouped into six major categories (see Figure 5), so as to capture the scope of the Environment Regulation’s portfolio.

Figure 5: Environment Regulation - project distribution by thematic categories, 2000-2002 commitments

Classifying the 35 environmental projects accordingly, “Conservation of natural resources” stands out as the single most important topic currently covered by the Regulation’s portfolio, with environmental mainstreaming and sustainable management of natural resources following behind. The remaining categories each account for less than 10% of the project portfolio. As in the case of the projects under the Forests Regulation, the scope of the activities of one project may address several of the subcategories mentioned in Art. 3. Fourteen out of the sixteen sub-categories are covered by one or more projects. No projects could be identified in respect of the priorities “sustainable production and use of energy“ (indent 9) and “sustainable production and use of chemical products” (indent 10). On average, environmental projects were found to cover 2 subcategories. [2] The past Strategic Orientation28 exercises have emphasised the Regulations’ thematic intersection. The contextual scope of each Regulation is broad and there exists a certain thematic intersection (e.g. in the areas of protection of natural resources, biodiversity, climate change) between the two. The selection and definition of priorities through Strategic Orientation and Annual Programming exercises are covered by the broad contextual scope of the Regulations. However, it must be noted that (i) this prioritisation has concentrated on the issues which fall under the thematic intersection (“intermediate foci”) and (ii) the choice of priorities was similar for 2000-2001 and 2002-2003.Other issues have been selected to a lesser extent (e.g. for environment: urban environment, industries, use of chemicals/hazardous wastes, production and use of energy, fresh water resources; and for forestry, wood energy)29. [3] For the Forests Regulation, the geographical scope has been extended considerably (compared to the previous Regulation), while financial allocations have decreased The annual financial allocation under the previous Tropical Forestry Regulation stood at about € 50 millions. This figure has decreased to about € 41.5 million. At the same time, the Forests Regulation’s focus has been widened considerably so as to include “all types of forests”, including those in developing countries outside the tropics. Potentially this extension could have reduced the achievements of the Regulations’ stated objectives to isolated accomplishments, and could

28 Art 8 [2] Environment and 9[2] Forests Regulation stipulates that “every second year the Commission shall adopt, …

strategic guidelines and priorities for implementing the activities to be carried out in the following years.” 29 It must be noted that a basic endeavour underlying the Environment Regulation is to prevent, or limit quantitative as well

as qualitative environmental damage caused by human activities, and thus to safeguard environmental assets / quality and enhance environmental security. This basic notion directly motivates intersectoral mainstreaming activities. On the other hand, forestry is an economic sector, which nevertheless has exceptionally significant environmental and macro-economic implications.

Sustainable management

of natural resources

16%

Improvement of urban

environment8%

Implement. of int. Env. policy

frameworks9%

Conservation of natural resources

41%

Main-streaming of env. issues in nat. policies

and developt. coop.20%

Improvement of environmt.

aspects in other sectors

6%



have reduced the overall impact of measures conducted within the Regulations’ purview. In practice, however, only one project outside the tropics has so far been initiated. [4] Projects mostly build on established and proven methods, but some approaches and tools within their overall context are new, and possibly apt for replication in the future Both Regulations focus on “pilot” or “innovative” measures as part of their respective thematic scopes30. The Strategic Guidelines 2000-2001 and 2002-2003 all consistently refer explicitly to the “innovative” character of measures implemented under budget line B7-620 as one criterion for eligibility. As no common definition of innovation could be found, the following working definition has been adopted for the purposes of the present evaluation. Measures shall be considered “innovative” as long as they are either

novel in their entirety, or seek to transfer or adapt established methods and approaches to a new or different

context (spatial, social, or other). Findings31 revealed that few projects apply entirely new approaches. A definite majority of projects under B7-620 fall within the innovative definition’s second category. Only two out of 40 projects responding to the written poll reportedly designed their proposal from scratch in response to an Call for Proposals. 20 projects reportedly fell back on project designs from previous phases of the same project. It must be noted that innovative measures - particularly those within the first category - depend upon swift selection and a timely start to implementation to remain relevant. This requirement has not been adequately met so far under the selection procedures.

Though many projects under B7-620 are still at an early stage of implementation, in a more general perspective it would appear that approaches involving bringing about changes in attitudes or behavioural patterns on the part of target groups (or requiring practical proof of their economic feasibility) will in most cases not reach a sufficiently mature stage, within the period of implementation envisaged under B7-620, to be considered replicable. On the other hand, projects seeking to realise well defined, technical improvements in a generally supportive environment not only achieve their objectives within the normal funding periods under B7-620 but can in practice prove highly replicable. With respect to the projects’ replicable character, finthe evaluation of the written poll show that:

21 out of 40 projects seek to dismethodological and contextual aspects of

In 7 projects there were certainty that theover by other donors

4 projects reported a high probability odevelopment co-operation measures (hDelegations)

However, in some cases partner organisations and activities of projects under B7-620. This certainly prev

30 Article 4 [2] FOR; Article 3 [2], 3rd indent and Article 3 [3], last pa31 Findings from the analysis of (i) project descriptions, (ii) the co

The same holds true for the comparison of the evaluation tperception and information provided by EC delegations.

32 For a detailed documentation of findings, refer to ANNEX 16

Examples for projects building on testedapproaches with prior financing of other donors The first phase of a project in Kenya (80) started in1995. It was initially funded by the NetherlandsCommittee for IUCN until EC became the major donorin 2001. In Ethiopia (39), FARM-Africa and SOS Sahel havejoined forces to continue their activities of sustainableforest management funded, prior to the ECcommitment, by the Dutch and British Government.

24

dings32 from the country mission as well as

seminate proactively information on the their approaches approaches followed would in time be taken

f their approaches being continued by EC owever, this view was shared by only two

donors were not aware of the existence or ents replication of the project approach.

ragraph ENV untry missions, and (iii) the written poll differ slightly31. eam’s professional judgement with the project’s self-



25

[5] Project design and planning have improved, in comparison with the 1998 evaluation of the Tropical Forest Budget Line33, but some flaws and bottlenecks persist While the quality of project planning varies in specific aspects, the projects assessed during field phase generally displayed a relatively high level of quality in their design34. Compared to what was observed during the 1998 evaluation of the Tropical Forest Budget Line, the use of logical-framework matrices and PCM tools has become universal, and state-of-the-art project planning routines have been institutionalised. A remaining weakness in most cases is the definition of objectively verifiable indicators (OVI). In the field phase sample35, out of a total of 27 projects, six projects did not adequately define OVIs (and instead listed criteria, for example), while 13 projects defined qualitative indicators which were found to be difficult to verify and therefore not useful for monitoring purposes. (see also [30]) All 97 project proposals assessed elaborate on stakeholders, target groups and country-specific settings, although systematic analyses were mostly not conducted36. Rarely have applicants developed their project proposals from scratch after having responded to a new call for proposals (CfP)37. The team’s observations indicate that most of the projects visited looked back on some kind of “history” (pre-dating the CfP), be it a previous phase (supported by other donors, or under other financing instruments), a systematic research or project identification effort (often with external support) or long-standing presence and experience in the region, and hence thorough knowledge of local problems and priority needs. The above factors made possible, in most of the projects visited38, at least partial involvement of stakeholders and target group representatives in the project design – something that would have been far more difficult to achieve had the applicants initiated project design only after responding to a new CfP39. The field visits showed that the design of 14 projects was considered to fit well with the capacities of the target group, whereas 12 out of 2740 projects were found to have overestimated their capacities. This might also explain the fact that many projects seek a cost-neutral extension of their implementation period. On average projects under B7-620 are implemented over a 36-month period. Project duration thus becomes a major bottleneck. Natural resource management in particular requires long implementation periods of up to 10 years. This issue, already raised in the course of the previous evaluation in 1998, remains valid. Beneficiaries cannot expect to receive funding support for more than one phase41. Direct continuation of a project would therefore depend on a successful, new application under a CfP (which is neither likely nor indeed feasible). Even if they were successful, there would be an appreciable gap between the two phases, causing implementation-related problems. The other possibilities for continuation of a project are its being taken up by partner organisations, donors or under other EC financing instruments. [6] Most projects showed a good overall performance As a result of the enhanced project planning framework now in use and the strong selection process, the evaluation team considers that the projects visited are of high quality, and that the forest component seems to have improved significantly since the 1998 evaluation of the Tropical Forest Budget Line. The team’s overall performance rating of the visited projects using the DAC criteria of relevance, effectiveness, efficiency, impact and sustainability 42 is as follows: 33 The overall efficiency of the project planning process is affected by the selection requirements and procedures under the

Call for Proposals (CfP). These aspects, which affect all applicants evenly, are widely criticised and warrant separate discussion. For further detail about procedural aspects, refer to the answering of EQ 3

34 See PAS – 27-35 35 See PAS - 34 36 Quest-PROJ - 3 37 Quest-PROJ – 1 : 20 projects took up concepts/approaches of a previous project 38 PAS - 30 39 Individual beneficiaries reportedly did not involve target groups, because they did not wish to raise high expectations 40 PAS - 29 41 Financial Regulation 42 It must be noted that the team’s overall ratings were not based on thorough project evaluations. Rather, the country

missions produced “snap-shot”-like impressions of measures being implemented under B7-620, in keeping with the methodological premise that individual project visits would only serve as a sample providing further information and conclusions about the relevance and operation of the portfolio as a whole



26

Table 2: Overall performance rating of projects visited during the field phase

Excellent Good Moderate Inadequate Poor TOTAL

4 15 5 2 1 27 Excluding the recently started projects43, findings from the field phase justify a more detailed presentation in terms of effectiveness for the remainder 44:

Actual results differ from intended results; project purpose not

achieved

Certain intended results achieved, but purpose only to a limited degree

Results mostly achieved or possible to be achieved in the near

future

Project achieved intended results or is very likely to achieve

them in the near future 1 8 10 5

Where projects failed to implement activities or achieve results on schedule, this may be attributed to various factors:

Over-ambitious project planning (largely induced by the desire to be successful in the CfP competition)

Failure to translate project designs adequately into participatory annual plans of operation, to share responsibilities, or to adapt indicators properly

Concentration on single aspects within the project’s wider design, depending on the beneficiary’s focal capacity or specialisation, or on changing project environments

Inadequate support by the main partner organisation Force majeure (in Belize, a tropical storm wreaked havoc, affecting one project)

[7] Assessing the projects’ efficiency is in some cases hindered by a lack of transparency, owing mostly to structural characteristics of the co-funding arrangement Country missions to Belize, Brazil, Ethiopia and Kenya provide examples for how the practical assessment of efficiency is hindered by the current co-funding arrangement. In principle each intervention operates on a predetermined fixed budget, to which the Commission contributes a maximum of 80% of the total financial value. Beneficiaries are required to raise co-funding of at least 20%. The co-funding ratio may vary, however. However, the same beneficiary may, at the same time (and in the same area) receive further financial contributions from third parties (for similar purposes – sometimes for operations with even the same title). In such a case, it is practically impossible to distinguish between achievements accomplished under the 80/20 budget of the EC-supported project, and similar achievements in the same area funded from other sources. Therefore, the financial envelope available for a certain intervention may easily exceed the predetermined budget of the EC-supported measure (and, consequently, the co-funding ratio), while it is not possible to demonstrate how the additional funding translates into additional activities. Since it is impossible to attribute individual achievements to individual donors, the evaluation team was unable to establish or quantify the relative impact of EC funding in the field, that is in terms of outcomes achieved, products produced, or services rendered. In practice, the impact of EC contributions is obscured. This also has implications for the EC’s visibility as a major contributor.

Summarized answer to EQ1a: Both Regulations Commission-supported activities under B7-620 are coherent with both Regulations’ respective thematic scopes. However, the project portfolio does not evenly reflect both Regulations’ more comprehensive thematic scopes. The broad thematic intersection between the respective project portfolios under the two Regulations is due to, and reinforced by past Strategic Orientation exercises. For both Regulations, it can safely be stated that projects generally show a good performance -

43 most of which were funded from the second batch of the CfP 2001 44 PAS - 54



27

Examples how projects contribute to nationalpolicy formulation processes: On global level: a project implemented byUNITAR (21) assists partner countries bydeveloping capacities for preparing participation inintergovernmental and other internationalnegotiations (e.g. COPs). At country level: A project in Chile (13) providessupport to the integration of participatory planningand implementation models into the NationalStrategic Plan for Sustainable Development andthe National Action Program to FightDesertification. At local level: In Ethiopia, project (39) seeks toimprove the procedural framework conditions forthe forthcoming transfer of forest managementresponsibilities to rural communities.

although the formulation of OVIs leaves room for improvement. The latter weakness notwithstanding, findings show that projects’ design and planning has generally improved. Projects mostly build on proven methods, and some can be considered innovative, which is a focal endeavour both regulations have in common. Environment Regulation Most projects under the Environment Regulation address “conservation of natural resources”, however, noticeably few projects could be identified in respect of issues regarding trade, energy, industrial and urban environment. Forests Regulation Under the Forests Regulation’s project portfolio, “conservation and restoration of forests for their ecological value” stands out as the most important issue addressed. Extended geographical scope and decreased financial allocations reduced global impact of the Forest Regulation in comparison with the previous Tropical Forest Budget Line.

4.2 EQ 1b: Contribution to national policy formulation and local livelihoods

EQ1b: How far, how effectively and how efficiently have the Commission’s activities and interventions taken under the two Regulations contributed, through feedback of their results, to (i) national policy formulation with respect to natural resources and the environment and (ii) the enhancement of local livelihoods through sustainable natural resources management or environmental protection measures respectively.

Rationale

Evaluation Question 1b is no less comprehensive than 1a, although it shifts its focus from the regulations to the partner countries needs and requirements. These are defined on two levels, which nevertheless are closely interrelated. Improvements are to be achieved both in respects of conducive framework conditions for environmental protection and SNRM, and in regard of local target groups’ livelihood quality. This requires projects to either proactively influence policy reform process at national level, or to improve environmental quality / access to natural resources directly at the local level. Once again, the complex nature of the question necessitates definition of a set of criteria, and the use of proxy-indicators for their practical assessment.

Findings [8] Many projects focus proactively on contributions to national policy formulation processes in their design Environment Regulation 44 % of the projects under the Environment Regulation address “national policy makers” or “local decision makers” as target groups. Out of these, five global and one regional project contribute to national policy processes through capacity building of national and local policy makers. Another three projects (one global, two at country level) support partner countries pro-actively in implementing international environmental policy frameworks at national level (see box). Forests Regulation Of the projects under the Forests Regulation, 31% chose “national policy makers” or “local decision makers” as one of their target groups.



28

Many of these forest projects are field-based projects but comprise a policy-related component. They mainly deal with capacity building or information management. Only one global project (69) directly relates to international forest-related processes through enhancing information exchange. The EC’s support to the NFP-Facility Project, hosted by FAO, provides an example of a forest project contributing to national policy formulation45. [9] In practice few projects play an active role in policy formulation processes and fora Field visits revealed a number of cases where projects claiming a particular contribution to national policy formulation in their design failed to achieve this aim in practice. This can be attributed to the fact that NGOs highlight policy aspects as a key element of a competitive proposal but in reality feel uncomfortable with these requirements and concentrate mainly on field based activities. Furthermore many projects are located in quiet remote areas and lack access to national policy fora. In addition, trans-boundary projects under B7-620 face the challenge of splitting their resources and focusing on more than one country. This limits their options for influencing national policy processes, because they usually lack the “critical mass” and leverage required for engaging in national-level dialogue. National implementation partners do not share similar interests and motivations in all cases, which increases the need for internal coordination. [10] Unconducive political and legal frameworks limit the projects’ opportunities In a significant number of countries – predominantly in Latin America – the national policy frameworks are not conducive to, and positively hinder, environmental protection and sustainable forest sector development. While the perceptions of the two main categories of actor – projects and Delegations – may vary, their judgment was found to be noticeably consistent in this particular respect46. In countries where non-conducive frameworks prevail, the contributions of single projects to national policy formulation are limited unless they are integrated in broader networks combining efforts for a change. The field phase revealed good as well as bad examples of such integration, depending mainly on the contractors’ initiative. One project in Ethiopia provides a positive example, in that various donors’ projects pooled their resources, and thus gained greater political leverage at regional level. [11] In about half of the partner countries, national sector administrations are uninformed and/or uninvolved about projects implemented under B7-620 Half of the EC Delegations responding to the written poll selected the answer47 “largely indifferent” to characterise the respective sector administrations’ attitude towards projects implemented under B7-620. Discussions with key actors during the field missions confirmed the aforementioned finding, and pointed to lack of information (and, consequently, involvement & ownership) as the main reason. Some even did not know of the projects’ existence (e.g. Cameroon). The remaining EC Delegations responding to the question held the opinion that national sector administrations regard the projects as “generally positive”. No “negative” judgements were recorded. [12] EC Delegations did not serve as catalysts for B7-620 projects’ contributions to national policy formulation It must be noted that up to now EC Delegations were not involved in the administration of projects under the B7-620 budget line. Therefore they contributed neither to informing the projects about national policy formulation processes nor to involving them in those same processes. This will certainly change after the deconcentration process has taken place, considering the fact that about half of EC Delegations48,49 reported being present in national consultative fora. This will also increase the B7-620 Budget Line’s visibility vis-à-vis other budget lines and other financing instruments. 45 This project does not belong to the project portfolio of 97 projects but was assessed on request of the Steering

Committee. [0] 46 see documentation of findings in ANNEX 13+14 47 Quest-DEL – 3.2 48 Out of total of a total of 28 EC delegations responding to the questionnaire, 25 answered the questions relating to active

involvement in national policy formulation 49 relative to 28 = 100%



[13] In comparison with the 1998 evaluation of the previous Tropical Forest Budget Line, the current Forests Regulation has attained distinctly greater policy orientation Comparing the priorities mentioned in the 1995 Council Regulation EC No. 3062/95 “On Operations to Promote Tropical Forests” with the current Forests Regulation’s thematic scope, one finds that the current Regulation maintains an explicit focus on development of international as well as national policy frameworks (which was previously not the case). [14] Achievement of clear impacts on local livelihoods depends on the type of intervention Comparison of the projects’ choice of target groups reveals noteworthy differences between the Environment and Forests Regulations. 50% of the projects under the Forests Regulation chose “local communities” as a one target group - a choice made by only 29% of the projects under the Environment Regulation50. Given that most projects visited are still at an early stage of implementation, it is hard to measure (much less quantify) impacts on local livelihoods. Therefore, we talk more about the likely project contribution to issues such as poverty alleviation, empowerment of (rural) stakeholders, and creation of small and medium sized business and investment opportunities. Those projects most likely to achieve a measurable impact on local livelihoods within the projects’ implementation period incorporate “direct” measures (with immediate effects, such as income generation, enhanced agricultural productivity, diversification of production or market access) in their design. Howonly realise a significant contribution to improving tcontinuation of project activities beyond the current The team’s positive judgement of “direct impact” doof projects aimed at more “indirect impacts” on improvements in framework conditions may providedistant future.

Summarized answer to EQ1b: Both Regulations Policy-oriented project designs tend to focus on either ccontributions to international initiatives and processes. Iinto policy formulation processes and fora. In many formulation or local livelihoods will only achieve their fcan be assured beyond the current phase. Environment Regulation A significant proportion of environment projects addreand chose decision makers (national and local) as thfewer environment projects contribute directly to the etheir selection of local communities as target groups. Forests Regulation Forest projects tend to address national policy formulatCompared to the previous TFBL, the current Forests Rorientation. Compared to projects under the Environmcommunities as target groups.

50 Figures 8 & 12, DESK REPORT 51 PAS , question 60

Examples of direct impact - Transfer of management responsibilities for forests

from Government to local communities createsemployment and income opportunities (39)

- Introduction of multi-purpose trees on farmland andenhancement of marketing opportunities leads toan increase in income for local farmers (78)

Examples of indirect impact - Awareness creation and policy advocacy reduced

ongoing forest destruction in Kenya (80)

04 29

ever, a majority of projects under B7-620 will heir respective target groups’ livelihoods51, if a phase can be assured.

es not necessarily imply depreciating the value local livelihoods, of the kind that systematic – even if those impacts materialise in a more

apacity building of national decision makers, or n practice, however, few feed their results back cases, contributions to either national policy

ull impact, if a continuation of project activities

ss national policy formulation in their designs, eir most significant target group. By contrast, nhancement of local livelihoods, as shown by

ion as one component within a wider approach. egulation has attained distinctly greater policy ent Regulation, forest projects focus on local



30

4.3 EQ 2: Taking into account specific needs and interests of partner countries

EQ2: To what extent did the (i) design, and (ii) implementation of measures taken under the Forests/Environment Regulations take into account the specific needs and interest of partner countries?

Rationale

Sustainable impact and success of development cooperation interventions depends upon a high level of acceptance and ownership on the recipients’ part. The European Community’s Development Policy specifically highlights the need to improve impact through adaptation to the needs and capacities of partner countries52. The evaluation must take into account that partner countries’ needs and interests may be defined from different angles and by various stakeholders – e.g. government, civil society, indigenous groups.

Evaluation Findings [15] Projects under B7-620 reflect partner country policies adequately in their project description and design. Taking partner countries’ needs and interests into account forms part of the selection criteria applied under the CfP procedures. Accordingly, project designs specify options for contributing to achievement of the partner countries’ development priorities. These contributions and references are by no means limited to sectoral policies and priorities, but may include references to poverty alleviation, rural development, or various cross-cutting issues. Country Strategy Papers (CSP) do not only reflect partner countries needs and interests, but provide a negotiated framework for Community cooperation. This makes them a key source of information in the process of project design53. Art. 4 FOR refers to a requirement for coherence with CSPs, however, in a less than operational manner. Article 4 (4), 2nd indent, of the Forests Regulation (EC Regulation 2494/2000), stipulates that “Priorities shall be determined in accordance with…Community cooperation objectives as established by the Commission in jointly agreed country strategy papers”. Had this requirement been designed with the “Community cooperation objectives” (as listed in Art. 177 of the Treaty establishing the European Communities) in mind, it would be superfluous (because these – highly aggregated – objectives apply to all development activities, anyway, irrespective of sectoral characteristics). The evaluators’ interpretation of Art. 4 (4) 2nd indent is that coherence with sector-specific problem analyses and response strategies in the respective CSPs is required. If this were the case, three observations would seem pertinent:

selection of priorities for measures within the purview of the Forests Regulation must of necessity conform to and reflect the listing of objectives in the CSP;

failure to address forest sector development in a CSP might, given the current procedural framework, preclude implementation of forest-related measures under B7-620 altogether (even though this would seem counterproductive);

the Forests Regulation’s prescriptions in this particular respect have not been reflected in the identification and selection procedures so far followed under the Call for Proposals framework;

In the Environment Regulation, no such clear reference to the CSP is to be found, which reflects the Regulation’s focus on mainstreaming environmental protection in all sectors.

[16] Projects under B7-620 follow up on the development of national policy frameworks during implementation

52 COM (2000) 212 final, p. 30 53 Forest related interventions must conform to CSP priorities and objectives, see discussion in section [44]



31

About 80 % of the surveyed projects regularly follow up on the development of relevant policy frameworks54 - mostly on their own initiative and without support from, or coordination by, the respective EC Delegations. [17] Involvement of the partner countries’ (national) sector administrations remains a critical issue Particularly with respect to environmental protection and forests (as an example of a natural resource which serves multiple functions and, even if only in part, provides public goods), sector administrations must be considered key stakeholders, even though current socio-political and economic reform processes tend to reduce their relative significance for the governance of the sector. Widespread reform processes such as decentralization, democratisation, adoption of a market-based economic system and good governance practices increasingly tend to relegate sector administrations to a service function, and at the same time give non-state actors and claimants of tenure rights access to management functions and enjoyment of benefits. Consequently, project interventions must, of necessity, involve both sector administrations and non-state actors in a balanced manner, which will vary according to the local context. However, evaluation findings55 indicate that sector administrations play no significant role in the selection - and, consequently, the design - of proposals under either the Environment or Forests Regulations. On the other hand, they play an important role in the practical implementation of projects, owing to the fact that in many partner countries state administrations still claim exclusive rights of control (and even resource tenure or ownership)56.

Because an important part of the definition of the partner countries’ needs and interests – i.e. national development priorities, strategies and policies – takes place at a national policy level, under-representation of sector administrations at the selection and design stages constitutes a problem (see also [11]). However, the Budget Line offers the opportunity of creating “alliances for change” with civil-society stakeholders in those cases, where national sector administrations are either corrupt, or insufficiently committed to public sector reform, or lack the required capacities. [18] Local needs and interests are considered, with partial involvement of local stakeholders Findings57 consistently indicate that local needs, interests and priorities are considered in the design of measures. Stakeholders’ specific situations, capacities and needs are analysed, but the respective groups and populations are only partly involved in the process. Assessment of local needs is predominantly done in an informal manner, based upon expert judgements and previous knowledge of project sites and target groups. Many applicants conceive their proposals on the basis of other projects, previous implementation phases or extensive studies outside the context of a given Call for Proposals.

Summarized answer to EQ2: Both Regulations It can safely be stated, that projects under both Regulations in general reflect partner countries’ policies adequately in their design. During implementation a majority of projects under both Regulations follow up on the development of national policy frameworks. However, the failure to involve sector administrations reduced their interest, and thus their sense of ownership of B7-620 projects. Local needs and interests are mostly considered, however only partially with direct involvement of local stakeholders. Environment Regulation Because environmental issues are generally reflected in CSPs, these negotiated development cooperation frameworks provide a valuable source of information for the design and implementation of environmental projects.

54 Quest-PROJ – 10.4/10.5; Out of a total of 40 projects responding to the questionnaire, 33 answered the questions

relating to active involvement in national policy formulation. 55 question 12, Questionnaires “Projects” & “Delegations” – see ANNEX 13+14 57 The team’s judgment presented in the PAS, Questionnaires (Projects, Delegations)



32

Forests Regulation Failure to address forest issues adequately in CSPs makes it more difficult to capture specific needs and interests of partner countries in the design and implementation of forest projects.

4.4 EQ 3: Instruments and procedures for implementation of the Regulations

EQ3: To what extent did the instruments and procedures available for implementation of the Forests/Environment Regulations help or hinder achievement of its stated objectives?

Rationale/Introduction:

Most of the instruments and procedures guiding implementation of the Forests and Environment Regulations are provided by a general framework that is common to all other EC programmes for external aid and co-operation. This general framework changed58 during the course of the period covered by the present evaluation study. In addition to this general framework, the ENV & FOR Regulations 2493/2000 and 2494/2000 contain some specific procedural prescriptions for implementing the Environment and Forest Programmes and for management of Budget Line B7-620. The budget line is implemented through grant contracts, procurement contracts and the general administration, supervision and co-ordination activities, each of them following specific procedures. (Annex 7) Both the correspondence of the selected projects to the priorities of the Regulations and the quality of the projects depend also on the appropriateness of the selection procedures and instruments and their flexibility. Similarly, the human and physical recourses involved in the projects, their efficient use and the timeliness of the results and costs of each project are also a function of the procedures and instruments governing implementation of the activities financed under Budget Line B7-620 and related Regulations (e.g. monitoring systems).

Findings [19] The new procedures provide a greater degree of detail in their prescription and are therefore in some aspects more restrictive than the old ones. Newly-introduced or further-elaborated issues, under the new regulatory framework, include

standardised “Specimen Audit Certificate” and “Specimen Financial Guarantee” forms; beneficiaries being entitled to no more than one operating EC grant per financial

year; cancellation of the possibility of providing for a contingency reserve of up to 5% of the

total budget; reduced flexibility for the evaluation committee, at the time of award of grant, to

deviate from the original ranking of the proposal scores which are based on independent technical and financial quality evaluations’;

narrowing of the circumstances under which direct grants may be awarded; reduction of the deadline for submission of the final report from 6 to 3 months, if the

Contracting Authority is not a Commission Service.

58 Old: Until the 31st of December 2002, Financial Regulation of 21 December 1977, applicable to the general budget of

the European Communities, and in particular by Title IX of the Regulation on External Aid, that formed the basis of the procedural framework. Other subsidiary documents were: (1) the Vademecum on Grant Management adopted by the EC in July 1998 (COM (98) PV1395) and applicable since 1 January 1999 and (2) the Practical Guide to EC external aid contract procedures, published in January 2001. New: On the 1st of January 2003, a new Financial Regulation Nr. 1605/2002 came into force applicable to the general budget of the European Communities, the Commission Regulation Nr. 2342/2002 laying down the detailed rules for its implementation, and the Practical Guide to Contract Procedures financed from the General Budget of the European Communities in the context of external actions.



33

However, in some respects the new regulatory framework has become more flexible, or has added prescriptions in favour of contractors, e.g. the maximum period allowed for payment has been reduced from 60 days to 45 days. [20] Selection procedures of the CfP produce high quality project proposals, but require high input from applicants as well as EC Services The Environment and Forests Regulations59 provide that assistance “shall take the form of grants”, which are based on the Call for Proposal procedure60, and made with specific reference to the Financial Regulation61. Under the first Call for Proposals62 (issued early in 2001), 655 applications were received, 29 of which were selected for the 2001 budget. Another 26 were selected for funding for the 2002 budget. The team’s positive overall rating for the projects (refer to finding [6]) certainly reflects in part this robust selection process. However, quality comes at a price:

Under the first Call, 600 project proposals were not considered for funding. This

translates into a substantial financial loss for the unsuccessful applicants. Findings from the written poll63 indicate that – on a weighted average – applicants expend the equivalent of 6% of the total requested budget in preparing a project proposal, conforming to the demanding EC specifications and requirements. Even if it can be assumed that unsuccessful applicants have invested less in project preparation, there is still an important waste of resources (see Annex 8)

A definite majority of contractors consulted perceived the requirements of the Call for

Proposals as complicated and too demanding. Many applicants from developing countries react by forming alliances with partners from developed countries – in some cases reportedly perceiving this as a requirement in its own right. Many applicants feel obliged to seek additional assistance during preparation of proposals.

To manage such a large selection process, external as well as internal resources are

needed. Strategic orientation and definition of guidelines, as well as the selection procedure (in respect of projects exceeding financial thresholds prescribed in the Regulations) follow a number of complex and time-consuming administrative steps leading to a consensual decision, as required under the Management Procedure64 (see Annex 7)

[21] Selection procedures under the first Call for Proposals exceeded the prescribed deadlines Already at a very early stage, there had been a delay in the procedure caused by the modification of the Guidelines for Applicants and the deferral of the deadline for submission. In addition, the period between the deadline for submission and the opening session seems to have been unnecessarily long. Cumulative delays resulted in projects selected in the first batch (budget 2001) starting about one year after the deadline for submission, and those of the second batch (budget 2002) about two years.

Table 3: Timeline of the first CfP 11.1.01 1st Call for Proposals launched 14.5.01 Deadline for submission of the proposals (deferred from 27 March) – 636 proposals received 16./17.08.01 Opening session 24.9.01 Letter to 230 applicants, notification to have passed the first step (ACE) 26.9.01 Letter to 159 applicants, notification of the application being rejected 11.10.01 Letter to 247 applicants, request for missing information until 19.11. (later revised to 19.12). 28.12.01 Commission adopts the global commitment for financing 29 selected projects (on the budget of

59 Art. 6 ENV; Art. 7 FOR 60 pursuant to Art. 110 [1] of EC Regulation 1605/2002 61 Art. 8 [1] ENV and Art. 9 [1] FOR 62 Under the second Call for Proposals, issued late 2003, approximately 500 applications were received 63 Quest-PROJ – 6.5 In the team’s professional judgement, this figure lacks credibility, even though it is based upon direct

empiric evidence. 64 Art. 9 [2] ENV and Art. 10 [2] FOR, in conjunction with Articles 4 and 7 of Council Decision 1999/468/EC



34

2001) 17.12.02 Commission adopts the global pre-commitment for financing of another 26 selected projects (on

the budget of 2002) Contracts for the first batch (29 projects) were signed in the third (19) and fourth (10) quarter of 2002, those for the second batch likewise in 2003. Many contractors (and the Delegation staff concerned)65 complained that too much time generally elapses between: Submission of an application and notification of the successful or unsuccessful applicants66. In consequence, applicants tend to incur high transition costs, or else risk losing working relationships with local partners or acceptance by local target groups. Acceptance of a proposal and the first disbursement67: applicants are either forced to fall back on their own financial reserves (accepting a high risk), or else to accept delayed implementation of project activities or cutbacks in implementation activities to an absolute minimum while waiting for a funding source to open up.

[22] Targeted projects were predominantly being used to collaborate with international organisations and NGOs on an international scale According to interviews carried out at Commission Headquarters in Brussels, targeted projects (direct grants) provide an important instrument for implementing the Commission’s political agenda such as participation in international processes (e.g. promotion / implementation of MEA, or provision of support to National Forest Programme processes in pursuit of the international forest regime). Accordingly, out of 14 targeted projects currently being implemented, 8 involve collaboration between the Commission and international organisations or NGOs on an international scale. Three direct grants (1 ENV and 2 FOR) were awarded under the 2001 budget, and 11 (5 ENV and 6 FOR) under the 2002 budget. For the Environment Programme, 6 out of 21 grants (i.e. 28%) were awarded by means of the direct grant procedure during both years; for the Forests Programme the corresponding figures were 8 direct grants out of 48 contracts in total (i.e. 17 %). For both programmes together and over both years, there were 14 direct grants out of 69 contracts, or the 20% of the total number of contracts. In all cases, the Contracting Authority had to prepare an evaluation report justifying the manner in which the grant beneficiaries were identified and grant amounts were established, as well as the basis for the resulting contract award decision. With the coming into force of the new Financial Regulation68, the use of targeted projects for “strategic purposes” has been restricted69, to the extent that the new budgetary rules render the strategic use of targeted projects or direct grants more difficult. [23] Stakeholders involved in the implementation of B7-620 are mostly satisfied with relevant information provided by the EC Services in Brussels Both projects and Delegations praise the quality of information provided by the Commission Services in Brussels, and are mostly satisfied with the exchange of information (communication flows)70. About half of the responding projects were critical of their communication flows with EC Delegations. However, it must be noted that in the past, Delegations were only marginally involved in the management of B7-620. The current deconcentration process will change this situation. As yet, communication between projects and EC Delegations was largely up to the involved individuals’ initiative and to personal acquaintance.

65 Particularly with respect to the second batch under the first CfP 66 Quest-PROJ – 4H; Out of 40 projects responding to the written poll, 37 answered question 4H. Of these, 32 indicated

disapproval of the time elapsing between submission of their proposal, and eventual selection 67 Quest-PROJ – 4J and 4K; 68 Council Regulation (EC, Euratom) No 1605/2002 of 25 June 2002 “On the Financial Regulation applicable to the General

Budget of the European Communities”, in conjunction with Commission Regulation (EC, Euratom) No. 2342/2002 of 23 December 2002 “Laying down detailed rules for the implementation of Council Regulation (EC, Euratom) No. 1605/2002 on the Financial Regulation applicable to the general budget of the European Communities”.

69 Art. 110 [1] EC Regulation No. 1605/2002, in conjunction with Art. 168 [1] EC Regulation No. 2342/2002 70 Quest-PROJ – 4A,D; Quest-DEL – 6A,D



During the course of country missions during the field phase, national sector administrations in Belize, Cameroon, and Vietnam complained specifically about not having been informed by the Commission about applications and the respective selection of proposals. [24] Programming exercises – as required under the Regulations and the pertinent procedural framework - do not adequately transfer and operationalise the Regulations’ thematic scope The thematic scope of both Regulations is to be focused and operationalised by means of (biannual) Strategic Orientation and Annual Programming71, followed by the publication of a Call for Proposals. It was first observed that considerable thematic overlap existed between successive issues of the Strategic Guidelines (2000-2001 and 2002-200372) as well as between the Guidelines for Applicants (2001, 2003). Second, only two Calls for Proposals have so far been published, although the procedural framework requires the publication of one Call for each Annual Programme. Thirdly, the definitions of priority actions in both the Strategic Guidelines and Guidelines for Applicants did not directly correspond to actions listed in the Regulations – thus impeding direct comparison between the documents. One possible explanation for the aforementioned difficulties and imperfections might lie in the “inter-service consultation” exercise: many people are involved, each wishing to add something ‘very important’. [25] Guidelines narrow down the scope of eligible applicants Successive Guidelines for Applicants excluded certain applicants who are listed as “eligible” in the Regulations themselves73. The exclusion extends to sector administrations of the partner countries, profit making operators, and natural persons74. Whether this practice is in line with legal prescriptions of the Regulations, Council Decision 1999/468/EC and the budgetary framework (EC Regulation 1605/2002, in conjunction with EC Regulation 2342/2002) is indeed questionable. The analysis in Annex 9 gives room for the interpretation that the past practice of excluding certain applicants is not compatible with content and basic intentions of the Regulations. The exclusion of state administrations as contractors in the guidelines for applicants was controversial: It was justified by the Commission Services on the grounds that state administrations have access to other (e.g. geographical) financing instruments. It was also suggested that state administrations still can benefit as final beneficiaries, for example for capacity building. On the other hand, the team recorded criticisms in this regard from both ministry staff and EC Delegations during the field phase. According to the team the following arguments are advanced in favour of not excluding state administrations:

Article 4 (1) lit. a of the Forests Regulation defines the “development of appropriate national and international forest policy frameworks”, including “legal and fiscal measures”, as an express objective of EC support under the budget line;

projects addressing issues touching on the executive powers of a sovereign state (such as policy formulation and legal and institutional reform), require the relevant sector administration’s prominent and immediate involvement, which is not possible under the current guidelines.

71 Art. 6 ENV and Art. 7 FOR, in conjunction with Art. 8 [2] ENV an

1605/2002 72 extended to 2004 73 Art. 4 ENV and Art. 5 FOR 74 Section 2.1.1 CfP 2001, p. 6 and Section 2.1.1 CfP 2003, p. 5

Ruling out Government agencies from beingeligible for funding results in lost opportunities Covering more than 70% of the land area, Belize hasbenefited from three forest projects under B7-620 andthe previous TFBL, all working under unconducivepolitical framework conditions. Forest policy hashardly altered since the 1920s. To translate theprogress made in the forest sector into formalGovernment policy, the Forest Department prepareda project proposal to be funded under the 2003 CfP.Answering a request from the Forest Department,prior to submission of the proposal, the Commissionconfirmed their illegibility.

35

d Art. 9 [2] FOR, as well as Art. 110 [1] of EC Regulation



36

The team thinks that neither the involvement of state agencies as associates or partners of beneficiaries nor their direct involvement in “targeted measures”75 can compensate for the possibility of a direct contractual relationship with the EC. [26] Guidelines for Applicants (2001, 2003) rule out small-scale projects, and define maximum grant sizes for environment and forest projects (2003) The 2001 issue of the Guidelines for applicants defined a uniform minimum grant size of € 500,000 for environment and forest projects. In the 2003 Guidelines, the minimum grant size has been raised to € 1 million for forest projects. Additionally, maximum grant sizes have been introduced in the 2003 Guidelines for forest projects (less than € 3.5 million) and environment projects (less than € 2.5 million). These maximum amounts coincide with thresholds defined in the Environment and Forests Regulations, which stipulate that decisions on the award of grants in excess of the above-mentioned thresholds depend on the approval of the Management Committee76. [27] The available instruments and procedures for selection do not allow any form of guaranteed continuation or subsequent funding of a follow-up phase Refer to EQ1A [28] The Forests Regulation’s requirement for preparatory studies to precede an application is not met in most cases The Forests Regulation specifically requires preparation of environmental or social impact assessments, and financial and economic feasibility studies as integral components of a project proposal77, Findings indicate that hardly any projects fulfil this requirement78 in practice, even though individual projects – on their own initiative – addressed isolated aspects79. The lack of such studies has certainly contributed to an over-estimate of local stakeholders’ capacities in about 40 % of the projects visited80. This must be considered one of the key factors motivating requests for extension of project implementation periods, or ex post modifications of the project designs, as occurred in about one-third of the observed cases. (Refer to [18]). However, it remains open to question whether this requirement of the Forests Regulation can be met by applicants, (given the high investment necessary and the low probability of getting the contract), unless applicants succeed in fund-raising for the project-preparation phase. The Environment Regulation does not require any such preparatory studies to be furnished as part of the project proposal. [29] Failure to standardise reporting formats in practice hinders institutional learning processes on the part of the Commission Services Reports provided by the projects vary greatly in depth volume, and content. Consequently, processing the reports received by the Commission Services – specifically task managers in Brussels – is time-consuming and burdensome. It does not allow rapid appraisal of project progress, often necessitates clarification, and thereby contributes to delay in subsequent disbursements. Besides, many grant beneficiaries would welcome standardized report formats as they would provide them with a clear guideline on how to report (degree of detail, topics to include,…) and thereby increase their chances of submitting reports that are accepted. Reporting rarely draws on the initial Logframe and OVIs 75 since the newly introduced budgetary framework restricts the award of direct grants to “…duly substantiated exceptional cases…”. 76 Art. 8 [3, 4] ENV and Art. 9 [3, 4] FOR in conjunction with Art. 9 [2] ENV and Art. 10 [2] FOR; making reference to Art. 4

and Art. 7 of Council Decision 1999/486/EC 77 Art. 4 [5] FOR.

79 Quest-PROJ – 3; Out of 40 projects responding to the written poll, 34 answered the third question. Of these, 33

reportedly analysed “local needs and priorities”. Five projects prepared “economic and financial risk assessments”. Four projects provided social impact assessments. Only one project prepared an environmental impact assessment.

80 refer to PAS, question 29



37

[30] Internal monitoring by the project implementers is left to their discretion, and therefore does not produce comparable results of consistent quality. The standard project contract81 does not impose any requirements (apart from compulsory reporting which could be considered as a form of internal monitoring) on the internal monitoring and evaluation carried out by the beneficiary. However, as part of the Description of the Action (annex I of the standard project contract) the beneficiary must describe how internal M&E will be addressed. Accordingly the evaluation grid used for the technical quality assessment of the proposals includes this methodological element. Findings from the field phase show that internal monitoring is conducted by about 70% of the projects assessed, albeit largely at the projects’ own discretion. Since OVI are either generally lacking or ill-defined82, the quality of internal monitoring exercises in relation to the planned objectives seems doubtful. More specifically, in some cases internal monitoring did not relate directly to the original planning (as reflected in Logical Framework matrices – which moreover are neither followed up nor adapted to progress in project implementation). [31] External evaluation (carried out either by the Commission’s initiative, or on the contractors’ behalf), or external monitoring (as initiated by the Commission) does not fulfil the Regulations’ requirements Both Regulations stipulate regular external evaluation of measures funded under B7-620.83 This requirement, which according to the evaluators’ interpretation applies to all projects under B7-620 - has so far only partly been met (maybe partly due to the still early stage of implementation – to date, four projects out of 97 have been evaluated). Up until now, task managers based in EuropeAid Unit F4 mostly commissioned external evaluations at their own discretion, when they felt the need to do so. Internal monitoring (which is never independent) cannot compensate for a lack of external evaluations. Results Oriented Monitoring (ROM) as an external monitoring instrument is of little relevance for assessments of BL B7-620 measures. It is based upon random samples of projects, specific to neither financing instruments, nor sectors. Besides, ROM captures only projects in excess of 1 M Euro. However, in those cases where ROM happened to address projects within the B7-620 portfolio (15 projects – 5 ACP/10 ALA), the projects staff’s appreciation was generally positive. The ROM visits were perceived as constructive and useful; they provided an excellent opportunity for discussing project issues of concern and to discuss the appropriateness of certain modifications84. Reported deficiencies include bad timing (viz. too early in project implementation), excessive brevity. No discrepancies between the ROM findings and those of this evaluation have been detected. [32] Co-funding requirements of a minimum of 20% of the total budget in cash are perceived as too burdensome by many project implementers, and besides are not evenly applied Numerous grant beneficiaries reported concerns regarding financial requirements and financial management. Their criticism focused on (i) their required contribution of a minimum of 20% of the project’s total financial value, and the non-eligibility of contributions in kind for that purpose; (ii) the fact that advance disbursements by the EC cover only 80 % of the total approved EC contribution, (iv) the inconsistent application of some of the financial rules (different task managers or Delegation officers interpret some the rules differently).

81 Special and General Conditions 82 This finding received further support through experiences made in Latin America by the consulting company charged

with external Results Oriented Monitoring (ROM). 83 The Commission shall regularly evaluate activities financed …The Commission shall submit to the Committee … a

summary of the evaluations made. (Art 10 [2] FOR, Art 11 [2 ENV] 84 It is foreseen to sytematically integrate ROM results into the CRIS database, based upon a standardised reporting

format.



38

Summarized answer to EQ3: Both Regulations Compared to the previous evaluation of the TFBL, and especially to the wider procedural framework prior to the coming into force of the current financial regulation, the procedures applicable to the administration of projects under both Regulations have become more detailed, more elaborate, and generally more restrictive. On the one hand this does ensure high quality of project proposals received - and, consequently - of the project portfolio On the other hand procedural requirements under both Regulations were in practice time-consuming and resulted in a waste of resources, even though stakeholders are mostly satisfied with information provided by the Commission Services. Under the current procedural framework, some of the Regulation’s basic requirements have been effected only to a limited degree, owing to several factors: too selective programming, restrictions on the use of direct grants for “targeted” interventions, exclusion of eligible applicants, etc. A definite bottleneck exists with regard to continuation of innovative and promising activities. At project implementation level, opportunities for more systematic documentation of lessons learnt and of project progress have so far not been fully exploited, resulting in less-than-perfect institutional learning on the Commission Services’ part. Environment Regulation No findings specific to the Environment Regulation are presented. Forests Regulation The requirement for preparatory studies, specific to the Forests Regulation, is not met in most cases.

4.5 EQ 4: Coordination, Coherence and complementarity to other donors

EQ4: To what extend have the Commission’s measures taken under the Forests and Environment Regulations been adequately coordinated with, coherent with, and complementary to other donors?

Rationale

Donor coordination is a key element in achieving synergies and avoiding duplication. It takes place on two levels: first at HQ level to harmonize policy frameworks and strategies for environment and forest interventions and to provide mutual information on planned activities; second, at country level to supplement or complement field interventions. As projects funded under B7-620 are often of a pilot nature and with relatively short implementation periods, their impact and sustainability depends on whether their experience and approaches are drawn upon. How far donors (besides other financing instruments and partner country organisations) fulfil this role depends mainly on information flow and coordination. The key issues in this respect are a) coherence, b) coordination and c) complementarity.

Evaluation Findings [33] Streamlining of policy dialogue, initiatives and international regimes in EC development cooperation policies and practices facilitates coherence and donor coordination The characteristic feature of the Environment and Forests Regulations is their being “subordinate” to a development cooperation rationale (see chapter 1.1) which can be traced back to UNCED in Rio. The Regulations embody a complex, overarching system of political objectives and commitments, which they transfer into concrete prescriptions for the administration of the Commission’s forest- or environment-related development cooperation. Therefore, the coherence and complementarity of measures taken under the Environment and Forests Regulations with measures taken by other donors in the same area of intervention depend to a large extent on the coherence and complementarity of the Commission’s development policy objectives with those pursued by other donors.



39

What applies to the EC, certainly also applies to other donors, including the EC Member States and a variety of international organisations (mostly part of the United Nations system): Emanating from the concept of Sustainable Development, international processes, initiatives and regimes have been put in place, particularly in respect of the environment and forests. These include relevant multilateral agreements85, as well as the International Arrangement on Forests (IAF86). These processes, initiatives and regimes are increasingly being integrated and mainstreamed into the development policy objectives and strategies of the international donor community. From the present evaluation’s point of view, integration and mainstreaming appear particularly relevant in the following respects:

Integration of poverty issues in all policies related to the environment, sustainable development (of natural resources), food security, urban and rural development87

Integration of environmental concerns into all development cooperation policies88 Integration of relevant international processes, initiatives and obligations into

development cooperation (specifically in respect of MDG, MEA and the international forests Regime), with an aim of bringing about synergies in the implementation of MEA and sustainable natural resource management89

[34] The Regulations, as basic acts, themselves oblige the Commission proactively to seek information exchange and coordination with Member States and other donors The following table summarizes legal obligations in regard of information exchange and (donor) coordination:

Table 4: The Regulations’ stipulations on information exchange and (donor) coordination Requirement ENV FOR Improving coordination and the flow of information between Commission and Member State projects so as to put in place coherent action…

3 [5th indent]

Proper coordination and information between the Commission and the Member States in order to ensure coherence…

4 [3, 5th indent]

Activities … shall be coordinated with, …, national and international programmes and activities… 4 [6] When appropriate, activities shall be carried out within the framework of regional organisations and international programmes of co-operation

4 [7]

Opportunities may be sought for co-financing with other donors, especially with member states and the international organisations concerned. In this respect, coordination with the measures taken by other donors shall be sought

5 [3] 6 [3]

In order to attain … consistency and complementarity … and with the aim of … optimum efficiency …, the Commission shall, in liaison with Member States, take all coordination measures necessary, including in particular: the systematic exchange and analysis of information on activities financed or being considered for financing by the Community and the Member States; on the spot coordination of these activities by means of regular meetings and exchange of information between representatives of the Commission and of the Member states in the beneficiary countries

5 [5] 6 [5]

In order to obtain the greatest possible impact of the activities at global, national and local levels, the Commission, in liaison with the Member States, shall take any initiative for ensuring proper coordination and close collaboration with …, donors and other international organisations involved, in particular those forming part of the United Nations system

5 [6] 6 [6]

The Commission shall regularly evaluate activities financed …The Commission shall submit to the Committee … a summary of the evaluations made. The evaluation reports shall be available to any Member State, …

10 [2]

11 [2]

The Commission shall inform the Member States, at the latest one month after its decision, of the activities that have been approved, stating their cost and nature, the country concerned and the cooperation partners

10 [3]

11 [3]

Findings related to meeting these requirements are presented in following paragraphs.

85 e.g. Convention on Biodiversity, Convention to combat Desertification, Framework Convention on Climate Change 86 Also referred to as the “international forests Regime”, which consists basically of institutional (United Nations Forum on

Forests, Collaborative Partnership on Forests) as well as contextual (non-legally binding Forest Principles, IPF/IFF Proposals for Action) and procedural (National Forest Programmes) components

87 e.g. EU Development Policy, pp. 4, 7,18 88 e.g. Development Policy of the European Community, pp. 6, 7; Communication “Towards a Global Partnership for

Sustainable Development”, pp. 6, 9, 11, 14; etc. 89 e.g. Communication “Towards a Global Partnership for Sustainable Development”, pp. 13, 14; “General Guidelines for

Interventions under B7-6200 in 2000 and 2001”, p. 4; Communication “Forests and Development – The EC Approach, pp.8, 13, 14; etc.



40

[35] Commission and EC Member States exchange information on, and coordinate programming exercises and budgetary allocations under B7-620 regularly, and through an institutionalised procedure The Environment and Forests Regulations provide for the Commission being assisted by the appropriate geographically-determined committee responsible for development”90, by which provision the “Management Procedure” described in Council Decision EC/1999/46891 is applied to programming and budgetary allocations under Budget Line B7-620. Basically, the procedure comes down to a Commission obligation to present “…a draft of the measures to be taken…”92 (e.g. in the form of the draft guidelines and priorities) to the relevant management Committees (in this case the ALA, MEDA and EDF Committees) and to seek the appropriate committee’s opinion or approval. These Committees are composed of representatives of the Member States and are chaired by Commission representatives. Guidelines for the use of funds from Budget Line B7-620 were accordingly presented to these committees for the years 2000-2001, 2002-2003, extended for 2004, and finally 2005-2006. Following this same procedure, the ALA, MEDA and EDF Committees are also to be involved in the financing decisions. Specifically, if decisions have to be made on grants of € 2,5 million or more (€ 2,5 million being applicable for the environment programme; the forestry Regulation mentions € 3,5 million instead) for an individual activity, the relevant Commission must seek the opinion or approval of the relevant Committee(s)93. For financing decisions related to grants of less than the above-mentioned amounts, the Commission must succinctly inform the relevant Committee(s) on the decisions to be made and the information must be available to the Committee not later than one week before the decision is taken94. A number of informal working groups equally serve coordination and information exchange. ETFAG95, for example, provides opportunities for the Commission to coordinate its forest-related development cooperation activities with EC Member States. ETFAG’s role in drafting the Code of Conduct for Forest Sector Development Cooperation96 is an example for how coordination can be promoted in practice. In drafting the code, ETFAG referred to international processes and concepts97, such as SWAP98 and the international forests Regime. [36] Donor coordination in regard of B7-620 is not ensured at the level of partner countries, activities and experiences are being less well covered in coordination fora than those under other financing instruments Donor coordination in partner countries is not ensured in practice, whether by the beneficiaries under the CfP or by the Commission Services in Brussels. Adequate coordination and coherence depends chiefly on the EC Delegations’ pro-active involvement in (national) donor coordination fora, ideally steered by the respective partner countries’ national governments. Such structures exist in a number of the countries visited (e.g. Brazil, Cameroon, India, Indonesia, Kenya), and are currently emerging in some others (e.g. Cambodia). Even where consultative fora exist, projects implemented under Budget line B7-620 tend to be less well covered by them than do larger bilateral aid measures. There are several different reasons for this: prior to deconcentration, EC Delegations lacked necessary information on measures under

the Budget Line - this is likely to change - assuming of course that the Delegations henceforth have adequate resources for their increased workload99

90 Art. 9 [1] ENV and Art. 10 [1] FOR 91 Art. 4 and Art. 7 of Council Decision EC/1999/468 of 28 June 1999 “Laying down the Procedures for the exercise of

Implementing Powers conferred on the Commission” 92 Art. 4 [2] of Council Decision EC/1999/468 93 Art. 8 [3] ENV and Art. 9 [3] FOR 94 Art. 8 [4] ENV and Art. 9 [4] FOR 95 European Tropical Forestry Advisors Group 96 in response to the focus on forest sector governance reflected in the 1999 Council Resolution on Forests and

Development 97 Wells et al.: ODI Forestry Briefing No. 2, July 2002; p.1 98 Sector-wide Approaches 99 In Cameroon, India, Indonesia a recruitment process had started for an additional staff member who would have specific

responsibility for the B7-620 projects but fears were also expressed that the additional staff would not be adequate to take on board the additional workload arising from deconcentration



projects as small and short-lived as measures under B7-620 typically tend to be less well known to both Commission staff and other donors, and less “visible” than other projects;

projects under B7-620 tend to be more detached from national coordination frameworks, and therefore under-represented in the respective networks.

Rarely are projects taken over by other donors. [37] Other donors do not benefit from experiences and lessons learnt from EC measures under B7-620 Both regulations oblige the Commission to regularly evaluate B7-620 projects, and to share the reports with the appropriate management committee, Member States, the European Parliament and other interested parties. This obligation implies dissemination of experiences made and lessons learnt from the im(including other donors). There was no indicationdissemination seminars – that this requirementinstitutionalised external evaluation and standardisformats, it will be difficult to capture lessons learnt in Furthermore, discussions with other donor represeinformation on concrete measures under B7-620 ha

Summarized answer to EQ4: Both Regulations There are institutionalised procedures in place in information and coordinating programming exercises anAccordingly, Commission and Member States exchangexercises and budgetary allocations under B7-620 rRegulations to share experiences with the public andmainly because at partner country level, donor coordina Environment Regulations No findings specific to the Environment Regulation are p Forests Regulations No findings specific to the Forests Regulation are prese

4.6 EQ 5: Link to international agreements

EQ5: To what extent was (i) the design and (ii) imRegulations in line with, and supportive to foagreements and processes?

Rationale

Forest/environment-related international agreementfor all partner countries and donors, including the Epartners refer to the same overarching objectiveRelevance is assured, since both donor countrieinternational agreements are obliged to meet the res

Examples for lack of coordination In Cameroon, the EC delegation participates in thenational consultative forum for the NFP-process“Programme Sectoriel des Forêts et d’Environnement(PSFE)”. Through this mechanism, EDF projects arewell aligned with ongoing policy development. In thisrespect, projects under B7-620 face the systematicdisadvantage of not being adequately represented,and of not receiving information and guidance onongoing policy reform. A similar case has been observed in Indonesia, wherea coordination mechanism has been established,which brings together the projects’ technical advisorsand relevant Delegation staff. While this mechanismcould certainly also be used for B7-620 projects, it sofar only involves projects under the ALA budget line.

04 41

plementation of B7-620 to a wide audience - whether through publications or through

has been met so far. Given the lack of ed monitoring and reporting instruments and a systematic way.

ntatives during the field missions showed that s not been shared in most cases.

respect of both Regulations for exchanging d budgetary allocations at Headquarters level.

e information on, and coordinate programming egularly. But the obligation stipulated in the therefore with other donors is not fully met, tion often excludes B7-620 activities.

resented.

nted.

and processes

plementation of the Forests / Environment rest / environment relevant international

s and processes provide a common framework C. This coherence can facilitate dialogue, as all s (sustainable development) and strategies. s and recipient countries which have signed pective obligations.



42

Evaluation Findings [38] Both Regulations refer to sustainable development as a cross-cutting development paradigm The most characteristic feature of the Environment and Forests Regulations is their being “functionally subordinate” to a development cooperation rationale, that is to socio-economic and political poverty alleviation foci. The Regulations attribute to environmental and forest-related issues a decidedly political dimension, as they are linked to good governance (including promotion of private property, decentralisation and devolution of executive powers), structural adjustment (promotion of a market economic system), civil society involvement and empowerment of marginalised groups (women, ethnic minorities, forest-dependent people with their traditional knowledge systems), etc. This integration of socio-economic and socio-political objectives into environmental protection and forest sector development appears to work in a bi-directional sense: on the one hand, the above-mentioned policy issues must be addressed to achieve sectoral objectives, given the predominantly socio-economic and political “underlying causes” of environmental degradation, forest degradation and deforestation100. On the other hand, environment- and forest-related development interventions might well assume a pacemaker status, setting (sectoral) examples of policy, legal and institutional reform. These could then be disseminated to other sectors, and promoted through national consultative processes. Closely related to the aforementioned political dimension are the issues of integration and mainstreaming, in respect both of EC policies and strategies and of national policies and strategies in developing (partner) countries101. [39] Both Regulations are coherent, and relevant vis-à-vis forest/environment-related international agreements and processes in their design Both Regulations reflect challenges for specific international initiatives, and contain references to relevant multilateral agreements. The following table illustrates this relationship, and exemplifies only the most obvious references to individual multilateral agreements. In doing so, the table is by no means exhaustive, because, owing to the logical hierarchy established by the concept of sustainable development, every contribution to sustainable development and poverty alleviation would in principle be predicated on an international initiative, process, or multilateral agreement.

100 “Forests and Development – the EC Approach”, pp. 1-4 101 Integration of poverty issues in all policies related to environment, sustainable development (of natural resources), food

security, urban and rural development (e.g. EU Development Policy, pp. 4, 7,18); Integration of environmental concerns into all development cooperation policies (e.g. Development Policy of the European Community, pp. 6, 7; Communication “Towards a Global Partnership for Sustainable Development”, pp. 6, 9, 11, 14; etc.; Integration of relevant international processes, initiatives and obligations into development cooperation (specifically in respect of MDG, MEA and the international forest regime), with an aim of bringing about synergies in the implementation of MEA and sustainable natural resource management (e.g. Communication “Towards a Global Partnership for Sustainable Development”, pp. 13, 14; “General Guidelines for Interventions under B7-6200 in 2000 and 2001”, p. 4; Communication “Forests and Development – The EC Approach, pp.8, 13, 14; etc.)


Synthesis Report September 2004 43

Table 5: Selected references to forest / environment relevant international agreements and processes and multilateral agreements

Reference to forest / environment relevant international agreements and processes in the Regulations MEA/IA102/IFR103

(examples) Environment Regulation

Forest Regulation

The Community shall support developing countries in their efforts to integrate the environmental dimension into their development process… promoting the implementation of policies, strategies, … for the pursuit of sustainable development

General aspects of SD

1 [1]

The Community shall provide …assistance and … expertise to promote the conservation and sustainable management of tropical forests and other forests …, so as to meet economic, social and environmental demands … at local, national and global levels

Chapter 11 of Agenda 21, Forest Principles

1

…global environmental issues, in particular those covered by multilateral environmental agreements, such as climate change, desertification and biological diversity

General aspects of SD

3 [1, 1st indent]

…environmental impacts related to the integration of developing countries into the world economy WTO 3 [1, 3rd indent] …sustainable production and use of energy and …use of renewable energy sources, increased energy efficiency, energy saving and the replacement of especially damaging energy sources…

FCCC 3 [1, 9th indent]

…sustainable production and use of chemical products, in particular hazardous and toxic substances POPs …conservation of biological diversity, …, sustainable use of its components, the involvement of holders of traditional knowledge on the use of biological diversity, and the fair and equitable sharing of the benefits arising out of the utilisation of genetic resources

CBD 3 [1, 11th indent]

…coastal zone, estuary and wetland management RAMSAR 3 [1, 13th indent] …desertification CCD 3 [1, 14th indent] …the creation of instruments for sustainable development, …trade related instruments such as labelling and certification schemes and green trade initiatives

CITES 3 [2, 5th indent]

…information campaigns on hazardous substances, and toxic waste and pesticides in particular 3 [2, 7th indent] …(a) development of appropriate national and international forest policy frameworks… IPF/IFF, NFP 4 [1, 1st indent] …(b) conservation and restoration of forests … due to… their value for the preservation of biodiversity CBD 4 [1, 1st indent] …(d) …more efficient utilisation of forest products…, the sustainable use of wood as an energy source, … IPF/IFF, FLEGT,

FCCC 4 [1, 4th indent]

…encouraging environmentally and socially responsible private entrepreneurship in the forests products processing and marketing chain, in the context of agreed policies for private sector development…

IPF/IFF, NFP, FLEGT, ITTA, WTO

4 [3, 1st indent]

Activities…shall be coordinated with, and may provide support for, national and international programmes and activities…in particular the proposals for action …by the IPF/IFF process

IPF/IFF, NFP, FLEGT 4 [6]

…activities shall be carried out within the framework of…international programmes of cooperation and shall be supportive to the development of a global policy on the conservation and sustainable management of forests…

UNFF/CPF, ITTA 4 [7]

Lesson learning and dissemination of results,… including support of the implementation of international environmental agreements

3 [3]

…, the Commission, in liaison with the Member states, shall take any initiative for ensuring proper coordination and close collaboration with …, donors and other international organisations involved, in particular those … of the United Nations system

5 [6] 6 [6]

102 Multilateral Environmental Agreement (MEA) designates a legally binding agreement between signatory parties with a specifically environmental focus; other International Agreements (IA) may

nevertheless also relate to environmental / forest relevant issues in certain parts or aspects (also with binding force under international law) 103 International Forest Regime


Synthesis Report November 2004 44

Reference to international agreements and processes can easily be traced all the way downwards through the “chain of operationalisation”, to the level of the Guidelines for Applicants104. Likewise, the obvious success of integration and mainstreaming of environmental issues can be easily observed down to the level of the Strategic Guidelines under the Environment and Forests Regulations, and the Guidelines for Applicants. [40] International processes and multilateral agreements are directly addressed by about 10 % of the projects implemented, while most projects contribute indirectly 10 projects out of 97 funded by the B7-620 relate directly to either a specific MEA or to regional agreements or processes. Seven projects fall under the Environment Regulation, while three are within the Forest Regulation’s purview. Three projects address either FCCC or CBD and one of them simultaneously promotes CCD implementation in South-East Asia. One project relates to the CITES convention, and another contributes to a regional treaty105. Three projects deal either with general aspects of sustainable development (promoting the precautionary principle), or with processes emanating from an international policy dialogue (AFLEG). Most projects within the Regulations’ purview contribute indirectly to international initiatives, processes and concepts, because they promote individual aspects and facets of sustainable development (e.g. nature conservation/biodiversity, climate change, desertification).

Summarized answer to EQ5: Both Regulations Both Regulations refer to “sustainable development” as a cross-cutting concept and are in line with forest/environment-related international agreements and processes. With respect to implementation, about 10% of the projects under both Regulations support these processes directly, most projects contributing indirectly. Environment Regulation Seven out of 35 environment projects directly address international processes and /or MEA. The Environment Regulation focuses specifically on global environmental issues, including those related to world economic development, energy, industry, hazardous wastes etc. Forests Regulation Three out of 62 forest projects directly address international processes and/or MEA. The Forests Regulation highlights sectoral aspects, such as forest-biodiversity, the International Forest Regime, and socio-economic contributions of sustainable forest management.

4.7 EQ 6: Relationship to other Commission development activities and policies

EQ6: What evidence can be found that the Commission’s measures taken under the Forests/Environment Regulations have been constrained, supported by, or impacted on other Commission development activities or policies?

Rationale

This question addresses the relationship between the Regulations and other EC activities and policies. Internal coherence, complementarity and co-ordination are examined. Therefore, not only the impact of other EC activities and policies on the Regulations, but also the effects of the Regulations on other EC policies and activities have to be considered.

104 CfP for 2001, pp.3-4; CfP for 2003, extended to 2004, pp. 105 Barbados Programme of Action for Poverty Alleviation of Small Island Developing States



Evaluation Findings [41] Compared to the full scope of programmable aid provided by the EC, environment and forest interventions play a negligible role As budgetary allocations of the geographical financing instruments correspond to CSP priorities, environment and forest related measures are disproportionately small, compared to other focal development issues (such as, for example, development of transport infrastructure and structural adjustment106). [42] For both Environment and Forests, there exist uniform policy frameworks which evenly apply to and guide environment and forest related development cooperation under different financing instruments A) Forests Until 1998, forest related development cooperation had no uniform conceptual framework (“umbrella“), but was guided by a variety of concepts and strategies tailored to individual financing instruments. Since late 1998, a common foundation has been laid107 through the Commission’s initiative, culminating in a series of increasingly detailed and refined policy documents. This process of conceptualisation has been broad-based, so as to encompass (at the outset) forest-related issues even beyond the field of development cooperation. This common foundation is coherent with the Forests Regulation. B) Environment Building upon a commitment dating from the 1998 European Council in Cardiff, and making reference to obligations emanating from the EC Treaties, Lomé Convention and numerous MEAs to which the EC is a signatory or contributor, a similar process has been launched with respect to environment culminating in the Communication COM (2000) 264 from the Commission to the Council, the European Parliament and the Economic and Social Committee “Integrating environment and sustainable development into economic and development co-operation policy – Elements of a comprehensive strategy”. Building on this Communication, an Environmental integration strategy was adopted (SEC(2001) 609: Commission Staff Working Paper “Integrating the environment into economic and development co-operation”). The Environment Regulation is instrumental to this end. The above process of harmonious conceptualisation ensures policy coherence between the Regulations and the other instruments addressing forest and environment issues, and provides a sound basis for synergies between the various financing instruments, and – of necessity – supports practical implementation. [43] Mainstreaming of environment issues is reflected on in other EC sector policies (e.g. trade and development, sustainable transport development, rural development), although in an aggregated manner For all three areas mentioned above, policy documents and legislation have been adopted108 which address the need to mainstream environment issues in development cooperation. The respective policy commitments have been further clarified and operationalised through specific guidelines for use by development cooperation experts. Searching for relevant references, however, one finds that policy commitments are of a rather general nature, only citing the need to consider environmental concerns in sectoral development measures.

106 According to the sectoral break down of EDF-09, environment related measures account for 0.5 % of the total allocation,

whereas transport accounts for about 30 %. 107 Communication COM (1998) 649 from the Commission to the Council and the European Parliament of November 3rd,

1998 “On a Forestry Strategy for the European Union”; Communication from the Commission of November 4th, 1999 “Forests and Development – The EC approach”; Council Resolution of November 11th, 1999 “Forests and Development”

108 The European Community’s Development Policy – Statement by the Council and the Commission; Communication COM (2000) 422 from the Commission to the Council and the European Parliament “Promoting sustainable transport in development cooperation”; Council Regulation (EC) No. 2501/2001 of 10 December 2001 “Applying a scheme of generalised tariff preferences for the period from 1 January 2002 to 31 December 2004”; DG Development (2000): EC Policy and Approach to Rural Development” of 15 June 2000



As regards trade and development, the EC adopted a legal basis for addressing strategic policy issues in the Generalized Scheme of Tariff Preferences (GSP). The relevant basic acts establish, inter alia, a Special Incentive Arrangement for the Protection of the Environment, whereby partner countries conforming to internationally agreed standards and principles may enjoy preferential treatment of exports of raw and manufactured goods into the European Union. The “Special Incentive Arrangement for the Protection of the Environment” exclusively relates to the “sustainable management of tropical forests”. It should be noted, though, that mainstreaming of environmental issues in trade may hardly be achieved in practice by focusing entirely on tropical forest management. The respective instrument has been of limited practical relevance so far109. [44] Forest Issues are not adequately reflected in CSPs and RSPs – environment, being a cross-cutting issue, is better reflected To investigate how environment and forest issues are reflected in CSP (and RSP), a contextual analysis of 35 CSP and 12 RSP was conducted110. The findings show that forest-related issues are, for the most part, poorly reflected in the CSP111. It must be a matter for concern that, among 9 CSPs rated as “poor”, there are countries such as Rwanda, Liberia, and Ghana where people are highly dependent on the productive and protective functions of forests. Forestry has been selected as a “focal sector” in only one CSP (Indonesia), whereas in six CSPs it is an “element of a focal sector”. Situation analyses with respect to the forest sector are altogether missing in 18 out of 35 CSP, and quite shallow112 in a further 10. In six cases, a mention of B7-620 was the only reference to a response strategy. As environment is a cross-cutting issue it should appear in all CSPs. As regards how well they reflect environment, a majority113 of CSPs were rated as either “good” or “very good”. Only five CSPs were found to be either “inadequate”, or “poor”. Environment has been selected as a “focal sector” in three CSPs, whereas in 10 CSPs it is an “element of a focal sector”. Situation analyses with respect to the environment are missing in only five out of 35 CSPs, and rather shallow in a further 14. Only in two cases was a mention of B7-620 the only reference to a response strategy, whereas it was cited in 15 CSPs as one component in a wider context (see Annex 17) These observations are corroborated by empiric evidence from the written poll114.

Findings from the screening of 12 Regional Strategy Papers (RSPs) indicate that forest related issues are, for the most part, missing entirely in RSPs115. Not only do most RSPs remain silent on response strategies aimed at checking deforestation and forest degradation, but even situation analyses are missing in 8 out of 12 RSPs (and remain sketchy in another two). Forestry has been selected either as a “focal sector”, or as an “element of a focal sector” in only two RSPs (Central Africa, Andean Community).

109 So far, but one application has been received (by the P.R. of China), on which so far no final decision has been taken by

the Commission 110 categories used: Situation Analysis, Response Strategy, Past & Ongoing Cooperation 111 Out of 35 CSP assessed, 21 were ranked as “inadequate” or even “poor”, whereas only 8 were ranked as “good” or

“very good”. (see Annex 16) 112 Addressing but one of the 3 dimensions of sustainability 113 19 out of 35 CSP 114 Out of 28 EC delegations responding to the questionnaire, all answered question 2C. Of these, 13 were of the opinion

that environment and forest related issues are appropriately reflected in the CSPs. 14 EC delegations criticized CSPs for not taking environment and forest related issues into account, and 1 EC delegation was indifferent.

115 Out of 12 RSP assessed, 10 were ranked as “inadequate” or even “poor”, whereas only 2 were ranked as “good” or “very good”.

Example for general references to theenvironmental dimension in EC Sector PolicyDocuments COM (2000) 422: Promoting sustainable transport indevelopment cooperation; Section 3.1.3, pp. 13-14: “Transport’s impact on the environment must beminimised Addressing the environmental impact of transportstarts by ensuring national regulations are in line withtargets agreed in international conventions. Nationalregulations will also require updating to reflect themotorized vehicles, marine vessels and aircraft inuse. Non-motorized transport also merits promotion,for example, bicycles, a mode under-utilised in manyAfrican countries. These fundamental steps, whichaim to minimise environmental impact, will then pavethe way for a strategy to monitor and enforcecompliance with national standards.”



In respect of environment, eight out of 12 RSPs were rated as either “good” or “very good”. Only two RSPs were found to be “poor”, whereas another two were rated as “average”. Environment has been selected as a “focal sector” in four RSPs, whereas in another three it is an “element of a focal sector”. Situation analyses with respect to the environment were missing in only five out of 12 RSPs. The successful integration of environment aspects can be attributed to the EC’s mainstreaming strategy, which is also strongly reflected in the Environment Regulation. [45] Rarely are measures under B7-620 taken up and continued under other financing instruments Continuation of measures initiated under horizontal financing instruments has been observed only in rare, isolated cases. Only two EC Delegations knew about projects being taken up by other EC financing instruments. Four projects reported that their approach has been or will be used further in the context of EC development cooperation. This is not surprising, as no specific mechanisms are in place which would ensure permeability between horizontal and geographical financing instruments. Moreover, planning cycles and the long appraisal periods for individual projects in geographical financing instruments are substantial obstacles. [46] B7-620 projects in practice are detached from interventions under other financing instruments (for a detailed overview of interventions, refer to Annex 18) A) Forests: For the period 2000-2003, 35 countries were identified where forest-related projects are being implemented116 regardless of the financing instrument. In 26 of them a total of 48 B7-620 forest projects exists. In 21 out of 35 countries, B7-620 forest projects are the only EC-supported measures relating to forests. This means that only in five countries there is co-existence between B7-620 projects and projects implemented under other financing instruments. Of these, three “targeted” projects (in Brazil, Central Africa and Indonesia) have been initiated in support of existing interventions under other financing instruments. B) Environment: Applying the above procedure to environment projects, 48 countries have been identified, in 11 of which a total of 15 B7-620 environment projects exist. Only in one country are B7-620 environment projects the only EC-supported measures relating to environment. Projects under other financing instruments do not impact on B7-620 projects (and vice versa), despite considerable overlap of subject matter. Projects were found to be conducted without awareness of the others, even though in many cases there could be direct benefits from cooperation. [47] Research projects under B7-620 fulfil an imposupporting applied research Under the Fifth Framework Programme (1998-200projects have been funded in many countries, such asand South Africa. Country visits revealed no dirHowever, as research is eligible under B7-620 the burole. The project portfolio of B7-620 comprises sevethe forest Regulation. They are implemented by univFrance, Great Britain, the Netherlands and Belgium. 116 this was done screening the CRIS database for projects classif

Example of projects being implemented withoutexploiting potential synergies The CLEAN project (24) and the DEWATS project(26), both dealing with overlapping urbanenvironmental issues in India, have so far notcommunicated with a view to exploring potentialsynergies. There is a need to learn from each other’sexperiences. Commission Delegations shouldenhance the learning process they have funded bynot being just a funding source and an aidadministrator but also becoming involved in policy.Moreover, doing so will contribute to increasing ECvisibility.

47

rtant complementary role, particularly by

2) forest and environment related research , for example, Argentina, Brazil, China, India

ect interrelationship with B7-620 projects. dget line fulfils an important complementary n research related projects, all falling under ersities and research organisations based in

ied as Forestry



[48] There is no institutionalised information exchange The field visit findings suggest that no institutionalised arrangements for information exchange between projects under different financing instruments at country and regional level exist in practice. In 2004 information exchange meetings have been or will be organised at regional level but only for B7-620 projects117. As yet, EC Delegations do not facilitate the involvement of B7-620 projects - whether in national fora of the respective partner countries, or vis-à-vis other EC-supported projects under other financing instruments118 (despite direct involvement and close familiarity of the concerned Delegation staff). However, Budget line project staffs were found to be very keen to have direct contact with the Delegation, particularly favouring a Delegation-run forum where budget line project and Delegation staff can meet to share experiences119. It should be noted that, as of 2004, deconcentration of administration responsibilities is completed.

Summarized answer to EQ6: Both Regulations Even though various EC policies provide a comprehensive framework for environment and forest related development cooperation, environment and forest projects play a negligible role in the full range of programmable aid interventions. Rarely do projects under both Regulations interact in practice with other EC development activities. So far, little has been done in the way of institutionalised information exchange, facilitation through EC Delegations’ activities, or continuation under geographical financing instruments. Research projects under B7-620 complement other EC-supported research efforts. Environment Regulation Environmental mainstreaming is addressed in a variety of sectoral policies. Forests Regulation Forest sector development is poorly reflected in the CSP/RSP documents which govern EC support under geographical financing instruments, thereby also restricting coherence and information exchange with measures under the Forests Regulation.

4.8 EQ 7: Consideration of cross-cutting issues

EQ7: To what extent does the design and implementation of the Forests / Environment Regulations address cross-cutting issues, i.e. good governance, human rights, gender issues, wider environmental issues as well as institutional development and capacity strengthening?

Rationale

This question captures the contextual relation between interventions under B7-620 and cross-cutting development issues. The European Community’s Development Policy (referred to under ‘Coverage includes’) does not explicitly mention ‘cross-cutting issues’, but instead cites a central objective (poverty reduction), plus four strategic areas derived from the Maastricht Treaty, five guiding principles (mainstreaming) and two levels of action. The following cross-cutting issues are considered: (1) poverty reduction efforts; (2) support for institutional development and capacity-building; (3) gender equality; (4) enhancement of economic, social, political and cultural rights.

117 26 Project representatives of B7-620 projects in Africa convened in Addis Ababa in March 2004, the first meeting of this

kind. A similar exercise has been conducted in Lima (Peru), and will be done in New Delhi. In India, the EU is a member of a Joint Working Group on environment – but its role is more reactive than pro-active, as policy issues still tend to be handled in Brussels.

118 This observation is surprising, given the fact that a definite majority of delegation staff (23 out of 28) responding to the questionnaire characterised their own position as “being directly involved in the administration of B7-620 projects”. Likewise, 17 out of 28 said to be personally familiar with environment/forest related issues. About one half (13) further qualified their own familiarity with environment/forest issues as “better than average”.

119 Quest-PROJ – 6.4



Evaluation Findings [49] Cross-cutting issues of EC development cooperation are reflected and addressed in both Regulations EC development policy proceeds from the premise that “…development policy is a multidimensional process…”, which implies “…broad based equitable growth, social services, environment, gender issues, capacity and institutional building, private sector development, human rights and good governance…120”. This basic notion is consistently reflected in the Environment and Forests Regulations’ wider policy framework. The issues good governance, human rights, gender issues, wider environmental issues as well as institutional development and capacity strengthening are incorporated in both Regulations, and indirectly addressed on the level of objectives and measures to be taken under the two Regulations. [50] Cross-cutting issues are taken up as implementation priorities and foci at the implementation level, i.e. in successive issues of the Strategic Guidelines, and Guidelines for Applicants under the CfP Whereas cross-cutting issues are reflected in both successive Strategic Orientation cycles, references are more numerous, and more pronounced in the 2002-2003 Strategic Guidelines. Besides, a slight difference can further be detected between Strategic Guidelines under the Environment Regulation, and Strategic Guidelines under the Forests Regulation, to the extent that cross-cutting issues are more explicitly mentioned and are located at a higher level of the logical hierarchy in the forest-related set of guidelines121. Screening the Guidelines for Applicants, various references to cross-cutting issues (e.g. good governance, capacity building, and gender) can be identified either as implementation priorities122, or as selection criteria123. [51] The EC Delegations’ feedback hints at regional differences in how cross-cutting issues are prioritised in the national context In the written poll, EC Delegations were asked about the cross-cutting issues’ relevance124 in the respective partner countries’ national context. The answers indicated that differing importance is attributed to individual cross-cutting issues, reflecting the perceived needs in the country-specific development context. The finding in the following table allows further discussion of individual cross-cutting issues for each region, as well as summary comparisons between the regions.

Table 6: Percentage of EC delegations highlighting individual cross-cutting issues

Region Number of Delegations responding

Human rights

Inst. Dev. Capacity Strength.

Gender, marginalise

d groups

Good Governance

Env. Issues

Africa 11 18% 36% 73% 36% 55% 27% Asia 6 50% 83% 50% 67% 100% 83% Latin-America 11 36% 64% 73% 55% 73% 100%

Total/Average 28 32% 57% 68% 50% 71% 68%

On average, good governance, capacity strengthening and environmental issues are perceived as most relevant across all three regions, while human rights stands out with the lowest percentage of answers received.

120 Communication COM (2000) 212 from the Commission to the Council and the European Parliament: The European

Community’s Development Policy; p. 7 121 refer to Mind Map Presentations of the four Strategic Guidelines (Environment/Forests; 2000-2001/2002-2003) 122 Guidelines for Applicants under the CfP 2001: priority issues, pp. 3, 4; Guidelines for Applicants under the CfP 2003,

extended to 2004: priority issues, pp. 9, 10 123 Guidelines for Applicants under the CfP 2001 evaluation grid p. 13; Guidelines for Applicants under the CfP 2003,

extended to 2004 evaluation grid p. 17 124 Quest-DEL – 11.6



In Africa, capacity building is the most important issue, with good governance following close behind. Also noteworthy is the relatively low consideration accorded to environmental issues in Africa, which suggests insufficient understanding of the linkage between poverty and environmental degradation. In Asia, good governance clearly stands out as most important, with environment and institutional development following second. In Latin America, environmental issues are dominant, followed by good governance and capacity strengthening. The average percentage of answers for all cross-cutting issues is lowest in Africa, while Delegations in Asia and Latin America selected more cross-cutting issues as relevant. Furthermore, the variation in answers is highest in Africa, indicating that EC Delegations in that region answered that question more selectively than did EC Delegations in Asia or Latin America. [52] In practice projects contribute mainly to Institutional Development and Capacity Strengthening All projects address environmental issues in one way or another, either because they are environmental projects, or because sustainable forestry by definition promotes environmental benefits. Therefore, environmental issues are not shown as a separate category in the table below125. The findings in the table allow further discussion of individual cross-cutting issues for each region, as well as summary comparisons between the regions.

Table 7: Percentage of projects126 highlighting individual cross-cutting issues Region Number of

Projects responding

Human rights/Good Governance

Inst. Dev./ Capacity Strength.

Gender Indigenous, marginalised

groups

Africa 12 42% 83% 33% 42% Asia 9 67% 89% 89% 89%

L-America 12 42% 83% 42% 75%

Total/Average 33 48% 85% 52% 67%

On average, institutional development and capacity strengthening are perceived as most relevant in all three regions. In practice, institutional development and capacity strengthening are closely related. Projects dealing with institutional development automatically cover capacity building. Institutional development is predominantly focused on organising communities (e.g. Forest User Groups, Environmental Committees) for enhanced Sustainable Natural Resource Management. Capacity building through training and dissemination reaches out also to decentralised sector administrations, local governments, research institutions, etc. Interventions favouring indigenous and marginalized groups address mainly forest-dependent people in practice, as expressly mentioned in the Forests Regulation. The apparent variation between African projects on the one hand, and Asian or Latin American projects on the other, may presumably be attributed to terminological differences between the regions. “Indigenous peoples” as an established concept of the policy dialogue is more prevalent in Latin America and Asia. In practice, projects dealing with indigenous or marginalized groups, and projects addressing human rights and good governance are quite similar. During the field phase, projects dealing with human rights or good governance were encountered chiefly in Latin America and South-East Asia. In Latin America projects focus on indigenous people, land-titling and land ownership. One project

125 In the table, certain categories (human rights/good governance; institutional development/capacity strengthening) have

been clustered, because these issues are practically linked from the projects’ point of view, whereas EC delegations have a more decidedly “political” viewpoint.

126 Quest-PROJ - 11

Examples of projects addressing the cross-cutting issue of “Good Governance” A project in Cambodia (96) seeks to empower fringecommunities and settlers in the buffer zone of PhnomSamkos and Phnom Aural wildlife sanctuaries in theCardamom Mountains, vis-à-vis sector-administrations, the military and commercialconcessionaires. In Brazil a project (50) supports demarcation oftraditional lands, claimed by indigenous communities.In several cases, where the communities claimingland rights within state park boundaries, the disputewas decided in favour of the communities.



in South-East Asia seeks to mitigate land-use or natural resource use conflicts involving indigenous people. All projects addressing human rights also promote good governance. About half of the projects contribute to gender issues in practice. In a regional perspective, projects in Asia clearly stand out with the highest percentage, more than twice the corresponding figure for projects in Africa. Findings from the written poll were further backed by the team’s observations during the field phase127.

Summarized answer to EQ7: Both Regulations In both Regulations’ design cross-cutting issues are adequately reflected, down to the level of measures to be implemented. This basic commitment is also well reflected in the Commission’s Strategic Orientation and programming exercises, as implementation priorities in the respective guidelines. However, the partner countries’ perceptions of cross-cutting issues and their respective needs vary region-wise. However, projects address cross-cutting issues more selectively in practice, with capacity strengthening and institutional development being the most important issues. Environment Regulation No findings specific to the Environment Regulation are presented. Forests Regulation Projects addressing indigenous and marginalized groups directly occur mainly under the Forests Regulation, due to the fact that many indigenous peoples in fact are either forest-dwellers, or very dependent on forest resources.

4.9 EQ 8: Added value of the two Regulations

EQ8: Have activities funded under the two Regulations represented “added value” in relation to other funding instruments for developing cooperation?

Rationale

This question examines the “added value” of having separate Regulations and a corresponding, joint budget line on forestry and environment. To answer this question, one must ask first which other financing instruments also address forest and environment-related issues. Furthermore the comparative advantage of B7-620 in concert with geographical as well as with other horizontal financing instruments has to be discussed. The portfolio of forest and environment related projects under these financing instruments has been screened with this question in view. The following sections separately address and present findings for the examined geographical, as well as horizontal financing, instruments.

Evaluation Findings

[53] The fact that a definite majority of B7-620 projects are well executed provides a precondition for the two Regulations realizing their potential “added value” in practice The field assessment of B7-620 projects resulted in the finding that a definite majority of measures are well executed. This observation is consistent with the EC Delegations’ decidedly positive assessment of the budget line (also refer to finding [6]). [54] Whereas environment projects occur under all geographic financing instruments, forest projects are to be found exclusively under EDF and ALA EDF and the budget lines for ALA, MEDA, and TACIS are geographical financing instruments from which forest and environment related measures are supported. Whereas environment projects are to be found under all these instruments, forest projects are to be found exclusively under EDF and

127 see PAS, Question 26



ALA. The portfolios of geographical financing instruments (EDF, ALA, MEDA, and TACIS) have been screened128 for relevant projects. This exercise revealed significant differences between the financing instruments, as well as between environment and forest related measures. For a detailed overview, refer to Annex 18. As forest projects under EDF and ALA mainly concentrate on some of the few remaining large rain forest areas (Congo Basin and Indonesia), B7-620 provides the only means of intervening in other countries, where the productive and protective functions of forests play a vital role in local livelihoods. This limited and concentrated support for the forestry sector provided by the other EC-financing instruments considerably increases the “added value” of the Forests Regulation and its measures (field projects) in the sense that they are highly complementary. This was confirmed by resource persons129 encountered during the field missions, who universally appreciated the Regulations’ value as instruments for promoting environmental protection and sustainable forest sector development, especially at grass-roots level. In a more general perspective it should be noted that aid budgets available under all geographical financing instruments tend to decrease as a developing country improves its status to a middle-income country. For middle-income countries with a high global environmental significance (such as Brazil and Malaysia), B7-620 offers the only opportunity of providing environment- and forest-related support, irrespective of geographical EC support. [55] EDF displays a substantial number of geographically distributed environment projects – while forest projects are few, and spatially concentrated Interventions under the 9th EDF are programmed in accordance with principles set out in the EU’s Development Policy (with its six focal areas) and the Cotonou Agreement130. Forest and environment are mostly missing in the National Indicative Programmes under the 9th EDF, or addressed as “non-focal sectors”131, as confirmed by the “Sectoral Breakdown of the 9th EDF programmed in CSP for ACP Countries”132. However, the relative percentage share of allocations for environment and forest related measures in Regional Indicative Programming increased over three successive EDF cycles (from 19 % in the 7th EDF to 20-25 % in the 9th EDF133), although the absolute financial volume remains low (€11 to 13.75 million). A) Forest related interventions During the period 2000-2003, 7 national forest projects funded from the EDF were operational (in 6 countries, plus one regional project in Central Africa), with an average financial volume of about € 5 million (8th and 9th EDF)134. Of these, three projects provided support to ECOFAC within the overall context of the G8 Congo Basin Initiative. More specifically, ECOFAC projects addressed forest conservation and reform of forest policy and administration. B) Environment related interventions During the abovementioned reference period, 36 environment projects funded from the EDF were operational (in 23 countries, plus eight regional projects), with an average financial volume of € 3.5 million (8th and 9th EDF)135. Two thematic foci can be distinguished (policy/administrative reform; protected area management), as well as two focal regions of intervention (SADC and Western Africa/Sahel). The strength of EDF is its potential to fund large projects with a better leverage on national policy formulation. 129 EC delegation staff, project personnel, sector administrations 130 Article 1 states that the principles of sustainable management of natural resources and the environment shall be applied

and integrated at every level of the partnership. Protection and sustainable management of natural resources such as Tropical Forest ecosystems is defined as a thematic and crosscutting issue under Article 32 of the Agreement

131 See also analysis of CSP in EQ 7 132 Official chart published by DG DEV; environment and forest related allocations account for about 32 million Euro

(equivalent to 0.5% of the total financial volume), of which 28 million Euro have been allocated to the development of a sewage plant in Mauritius

133 Joint answer to Written Questions […] given by Mr Nielson on behalf of the Commission(27 September 2002) 134 based on selection of projects from CRIS database, see Annex 18 135 based on selection of projects from CRIS database, see Annex 18



[56] Few environment/forest related projects occur under ALA, and moreover these are regionally concentrated in Asia. They are, however, better funded than EDF projects. Administration of EC interventions in Asia and Latin America are governed on a joint legal basis136. The effective ALA Regulation lists environmental protection as one out of six focal areas137 for Community cooperation with Asian and Latin American countries. References to environmental issues are also to be found among the listing of specific priorities for Community cooperation, which mention, inter alia, “protection of the environment and natural resources, and sustainable development”. Furthermore, 10% of the weighted average financial allocation for the period 1991 to 1995 was to be earmarked for “protecting the environment, in particular tropical forests” as a transitional prescription. A) Forest related interventions During the reference period 2000-2003, four national forest projects funded from the ALA were operational with an average financial volume of € 13.7 million. Of these, three projects with a focus on participatory forest management and illegal logging are being implemented in Indonesia, while one project in China addresses natural forest management. B) Environment related interventions During the above-mentioned reference period, 13 environment projects funded from the ALA were operational (in 8 countries, plus one regional project in Asia and two in Latin America), with an average financial volume of € 15.5 million. No clear thematic focus can be distinguished for Asia, whereas interventions in Latin America focused on environmental policy and administrative reform, and rural development. It stands to reason that projects dealing with policy and administrative reform at the same time promote environmental mainstreaming in the respective partner country’s national context. Funding is often channelled through specific programmes: (i) Asia Pro-Eco, designed to strengthen the environmental dialogue between Asia and Europe through exchange of policies and technologies (ii) Asia Urbs, which aims at encouraging a sustainable approach to urban management and promote the involvement of civil society organisations138, and (iii) UrbAL (Local Urban Development in Latin America)139 . Like EDF, the strength of the ALA budget lines is its potential to manage large projects with a better leverage on national policy formulation. [57] Environment/forest related measures under MEDA/TACIS play a negligible role The MEDA Regional Strategy foresees a total budget of € 15 million for environmental activities for the years 2002-2004, while the TACIS Programme foresees a total budget of € 21 Million for the years 2002-2003. MEDA countries are eligible for the SMAP (Short and Medium Term Priority Environmental Action Programme), launched in 1997 and reconfirmed in 2002140. Under both financing instruments, no forest projects were operative within the reference period 2000-2003. A total of 9 environment projects was implemented, of which 6 fall under MEDA, while TACIS accounts for 3. Except for one protected area project in Egypt, all MEDA projects were concerned with the preparation of, and follow up to, national or regional conferences. Environment projects under TACIS were cross-border measures, typically concerned with either environmental protection and natural resource management, or environmental networking (as a contribution to policy and administrative reform).

136 Council Regulation (EEC) No. 443/92 of 25 February 1992 “On financial and technical assistance to, and economic

cooperation with, the developing countries in Asia and Latin America”; the draft new Regulation, currently under discussion (COM [2002] 340 final “Proposal for a Regulation of the European Parliament and of the Council concerning Community cooperation with Asian and Latin American countries and amending Council Regulation [EC] No. 2258/96 – presented by the Commission”) will follow an entirely different approach, whereby mostly procedural prescriptions for the programming exercise will be made. Therefore, specific thematic references to, inter alia, environment, are not contained in the draft. The proposed new Regulation cannot be compared to the current one.

137 Art. 1 of Council Regulation (EEC) No. 443/92 138 1998-2005 time frame. CfP 2004 deadlines in March and June 139 II phase: 2000-200. Overall budget 50.000.000 over a period of 5 years 140 Its stated priority fields are: (i) Integrated Water Management (ii) Integrated Waste Management (ii) Hot Spots (iv)

Integrated Coastal Management (v) Combating Desertification.



[58] Relevant projects occur also under horizontal financing instruments – of which the NGO Budget Line B7-6000 clearly stands out as the most significant Of the budget lines listed under title B7-6 “other cooperation measures”, only the NGO budget line was found to have an important number of forest or environment projects in its portfolio. In the basic act141 which governs the administration of B7-6000, environment and forests are not explicitly mentioned, although reference is made to sustainable development142. A) Forest related interventions During the reference period 2000-2003, 6 national forest projects funded from the NGO Budget Line were operational (in 6 countries), with an average financial volume of € 0.4 million. These projects are mostly concerned with community based, small scale approaches to sustainable natural resource management, often involving Non-Timber Forest Products as well as agro-forestry schemes. B) Environment related interventions During the abovementioned reference period, 39 environment projects funded from the NGO Budget Line were operational (in 24 countries), with an average financial volume of € 0.6 million. These projects mostly focus on rural development themes. Compared to the B7-620 projects, the NGO budget line fulfils a sort of “seedbed” function for enhancing local livelihoods. [59] DG Research funded programmes International cooperation in the field of environmental research and technological development (RTD) has been pursued under the Fifth Framework Programme (1998-2002). Whereas the lion’s share of these funds goes to European countries, eligible countries or regions also include TACIS, MEDA and developing countries143. The objectives and priorities of the CfPs differ according to the region. They include environmental issues in respect of sustainable development. Countries covered by the present evaluation, which received research funding include Argentina, Brazil, China, India and South Africa. In Argentina 9 of the 31 programmes funded were environment-related. In Brazil the corresponding figures are 10 out of 37; in China 13 out of 42; in India 9 out of 24 and in South Africa 11 out of 40. The Sixth Framework programme (2002-2006) addresses environmental objectives also for Developing, Mediterranean, TACIS and CARDS countries. One of the thematic issues of the International Scientific Cooperation Policy with developing countries includes the rational use of Natural Resources. The 2003 CfP envisages two environmental priorities: (i) managing humid and semi-humid coastal ecosystems, and (ii) reconciling multiple demands on coastal zones.

Summarized answer to EQ8: Both Regulations No common findings for both regulations are presented. Environment Regulation Environment projects are to be found under EDF, ALA, MEDA and TACIS instruments. In summary, the added value of activities funded under the Environment Regulation materialises through (i) promotion of EC involvement in global initiatives and processes, (ii) provision of opportunities for mainstreaming urban and industrial environmental issues complementary to other financing instruments.

141 Council Regulation (EC) No. 1658/98 of 17 July 1998 “On co-financing operations with European non-governmental

development organisations [NGOs] in fields of interest to the developing countries” 142 Art. 2 [1] 2nd para, 2nd indent of EC Regulation 1658/98 143 Apart from accession countries and newly industrialised countries.



Forests Regulation Forest projects occur exclusively under EDF and ALA. As forest projects under EDF and ALA mainly concentrate on some of the few remaining large rainforest areas (Congo Basin and Indonesia), B7-620 provides the only means of intervening in other countries, where the productive and protective functions of forests play a vital role in local livelihoods. Among horizontal Budget Lines, the NGO Line provides most opportunities for addressing environment- and forest-related issues outside B7-620. In summary, the added value of activities funded under the Forests Regulation materialises through (i) a complement to the limited and geographically concentrated interventions under geographical financing instruments, (ii) focusing on participation of civil society and research organisations, promotion of (iii) innovative approaches and (iv) global initiatives.

5 Conclusions and Recommendations The conclusions and recommendations have been categorized into (i) those specific to the Forests Regulation (ii) those specific to the Environment Regulation and (iii) those relating to both Regulations. Reference to the respective findings is given in edged brackets. The conclusions have been drawn from combinations of findings. Below, they are structured according to the DAC-criteria144. The present evaluation relies on the following adaptation of the DAC criteria’s definitions.

Relevance: Whether the objectives of the Regulations are in line with EC, international and national priorities

Effectiveness: The extent to which the objectives of the Regulations have been achieved

Efficiency: The productivity of the Regulations’ implementation process Impact: All positive and negative, intended and unintended, direct and indirect

changes and effects directly attributable to the Regulations’ measures Sustainability: Whether the positive effects will continue after support under the

Regulations has been concluded The 3Cs: The performance in regard to coordination, coherence and complementarity

vis-à-vis other EC policies and financing instruments, the interventions of other donors and those of the partner countries

Recommendations have been further structured so as to illustrate and categorize the respective hierarchical level: (i) Regulations and wider EC-framework level (e.g. budgetary rules and rules for the exercise of implementing powers = “R-WF”); (ii) administration of the budget line level (=“ABL”); (iii) project implementation level (= “PI”). Because individual recommendations may pertain to more than just one hierarchical level, for each regulation an individual classification is presented in the respective matrices, additional to their respective section’s sub-heading. The evaluators have chosen to prioritise their recommendations as first or second priority level. Roman numerals indicate references to conclusions. The rating reflects the evaluators’ professional judgement of each recommendation’s relative importance.

5.1 Conclusions and recommendations for the Forests Regulation

5.1.1 Conclusions for the Forests Regulation

5.1.1.1 Relevance

I. The Forests Regulation is coherent with the EC Development Policy Framework and reflects the International Forest Regime {R-WF}

144 AIDCO Evaluation Unit H6 (2001): Evaluation in the European Commission – A Guide to the Evaluation Procedures and

Structures currently operational in the Commission’s External Co-operation Programmes; pp. 11-15.



The Forests Regulation directly refers to Sustainable Development as the main conceptual basis for forest related development cooperation [38]. Consequently, cross-cutting development goals are well reflected in the Regulation [49]. As regards issues specifically related to sustainable forest sector development, the Forests Regulation takes up the international consensus about Sustainable Forest Management (which emanates from the Rio and post-Rio process), as well as the substance of EC umbrella policies for forest development cooperation [42].

II. There exists a certain tension between the Forests Regulation’s intervention logic, and numerous partner countries’ policies, interests, and perceived needs {R-WF} Forest sector development does not seem to be a priority for most partner countries: the forest sector is omitted in most CSPs [44] which represent a negotiated framework for EC cooperation with partner countries’, based on their policies, interests and perceived needs. According to the evaluators’ interpretation, the Forests Regulation stipulates that strategic programming for forest-related measures under B7-620 must be done in accordance with the sectoral references145 in the CSPs. Omission of forest issues in the CSPs renders this requirement impossible to fulfil in practice [44] and limits the Regulations’ potential for complementing other financing instruments’ focal sectors with forest-related issues. Where unconducive framework conditions prevail, official policies are an inappropriate reference for the partner countries´ interests and needs [10]. (see conclusion III) Most projects within the purview of the Forests Regulation take account in their design of the absence in the partner countries of high-level political interest or a conducive framework, and concentrate on local needs and interests in their implementation [15], [16], [17], [18].

5.1.1.2 Effectiveness

III. Procedural restrictions reduce the Commission’s capacity to pursue strategic policy objectives through the Forests Regulation {ABL} The Commission’s options for pro-active ad hoc support for individual measures have been reduced by the adoption of the new procedural (financial) framework, particularly with respect to the award of direct grants. Furthermore, the biannual programming cycle for strategic programming limits the Commission’s flexibility in responding to emerging issues of the international forest political dialogue. This, in combination with the restrictions placed on the award of direct grants, limits the Commission’s room for manoeuvre [22]. Taken together, this aspect of added value of B7-620 vis-à-vis geographical financing instruments has been reduced [22]. (see also XIII and IX)

IV. The project portfolio does not reflect the Forests Regulation’s thematic scope in a balanced manner {ABL} According to the evaluators’ interpretation, the full thematic scope described in Article 4 (Forests Regulation) and Article 3 (Environment Regulation) has to be addressed by the Commission in a “balanced manner” (i.e. at least once in a Strategic Orientation over the Regulations’ entire lifespan). This contrasts with the demand driven implementation modalities. However, projects in practice tend to implement a broader variety of activities in parallel, than their stated objectives suggest. The Forests Regulation calls for balanced consideration of its thematic scope through Strategic Orientation and Annual Programming [1]. So far, only a limited number of priorities out of the Regulations’ broad thematic scope, notably those within the Forest Regulation’s broad thematic overlap with the Environment Regulation [2] have been addressed. The priorities were similar in the two successive Strategic Orientations. In consequence, it remains doubtful whether the required thematic balance can still be achieved during the Regulation’s remaining life-span. Although successive Strategic Orientation cycles and Guidelines for Applicants clearly focused on policy-related activities (such as, for example, supporting partner countries through the implementation of MEA), this is not reflected by the project portfolio, which features many field-oriented projects. In practice, most projects pursue forest conservation, sustainable management and knowledge generation [1]. This might be caused, at least partly, by the exclusion of certain 145 For the purposes of this evaluation, the CSPs’ sectoral content shall be understood as (i) appropriate problem analysis,

(ii) definition of response strategies, (iii) references to EC or other donors’ cooperation strategies and instruments



eligible applicants, notably state sector administrations [25]. The observed discrepancy might further be due to the demand driven Call for Proposals approach and uncertainty surrounding project selection, which can be a disincentive for the significant investment needed to develop multi-stakeholder policy oriented projects. The eligible applicants, such as NGOs, Research Organisations etc. contributed to policy issues, but in a more indirect way. They concentrated on individual components related to policy issues within a project’s wider context [1].

V. The financial allocation is not commensurate with the Forests Regulation’s extended geographical scope {R-WF} Whereas the Regulation’s scope has been extended so as also to include developing countries outside the tropics, the annual financial allocation has decreased in comparison with the previous Tropical Forestry Budget Line [3]. Moreover other financing instruments do not appear to have compensated for this decrease ([54] - [59]).

5.1.1.3 Efficiency For detailed discussion of efficiency, refer to section “Both Regulations” below.

5.1.1.4 Impact/Sustainability

VI. The Forests Regulation’s impact materializes mainly at local level, although not necessarily during the implementation period of individual projects {PI} The Forests Regulation has, in it current form, been enacted only recently, in 2000. Important aspects (e.g. geographical scope) have been extended, compared to the previous TFBL. Practical implementation started only in 2001. Furthermore, the procedural framework for grant administration was amended in 2002, to become effective on January 1st, 2003. Most projects within the portfolio are therefore still in their early infancy. Taken together, these factors restrict impact assessments. Statements about project sustainability must therefore, of necessity, remain uncertain to a large extent, especially where framework conditions are not yet in place. The abovementioned limitations notwithstanding, about half of the projects under the Forests Regulation can be safely expected to achieve impacts on local livelihoods, because they specifically chose “local communities” as their focal target group. This applies particularly to those cases where projects will provide direct benefits during their implementation phase [14].

VII. The Forest Regulation’s sustainable impact depends on whether the framework conditions for the implementation of measures can be improved until the Regulation’s expiry in 2006 {PI} In about half of the partner countries, national sector administrations lack ownership of B7-620 projects [11], and are therefore reluctant to take over and continue measures initiated under B7-620. Furthermore, EC Delegations have not in the past assumed a coordinating role on projects within the Forests Regulation’s purview [36]. In consequence, B7-620 forest projects faced imperfect conditions for sustainable continuation of measures, and lack a supportive national framework. This situation is likely to improve, because deconcentration of administrative responsibility for B7-620 is now complete. Sustainability could be further promoted by addressing forest issues more comprehensively in the CSP.

5.1.1.5 Complementarity with other EC financing instruments

VIII. Currently, the Forests Regulation is forced into a complementary role, even though it lacks the required capacity {R-WF/ABL} Because forest-related issues are practically non-existent in most CSPs, only a small proportion of aid under geographical financing instruments is devoted to the forest sector (albeit with bigger budgets and longer implementation periods) [54], [55], [56] or else it is entirely missing [57]. In some cases, omission of forest issues under geographical financing instruments has been justified explicitly by referring to B7-620 as the only means to support forest measures [44]. In view of the



Forests Regulation’s limited financial capacities, this notion must be considered as insufficient given the importance of forests for development and poverty alleviation. However, a limited number of “targeted” forest projects effectively supplement measures under geographical financing instruments, as exemplified by interventions in the Congo Basin. As regards other horizontal budget lines, the NGO Budget Line is the only financing instrument under which forest projects occur in practice, although in a small number of cases and with very limited financial volumes [58]. Furthermore, the NGO Budget Line requires that the contractor is a EU based NGO The Forests Regulation further supplements research activities undertaken by DG Research, owing to the fact that research has been integrated as an eligible measure under the Forests Regulation.

IX. Problems arising from the Budget-Line’s administration in practice reduce the Forests Regulation’s added value vis-à-vis other financing instruments {ABL} The comparative advantages listed below, as had already been identified during the previous evaluation of the TFBL in 1998, are insufficiently realized:

initiation of innovative pilot approaches is constrained by the time consuming CfP [21]

direct support to almost all stakeholders is in practice obstructed by a narrowing down of the scope of eligible applicants in the Guidelines for Applicants [17],

the former option to support for forestry activities irrespective of CSP content, is restricted because determination of forest related interventions under B7-620 must, in the evaluators’ view, adhere to the CSP [44]

the possibility of a more proactive approach to implementing priorities defined by the Commission’s development policy is limited by restrictions placed on the use of direct grants (which also affects global projects) [22]

co-funding arrangements tend to reduce or obscure the Commission’s visibility as a major donor, because other contributors often claim the respective interventions as their own vis-à-vis the general public [7].

However, B7-620 remains the only financing instrument for funding global projects / strategic projects which can fulfil an important role in addressing issues of international concern.

5.1.2 Recommendations for the Forest Regulation The evaluators have chosen to prioritise their recommendations as first or second priority level. Roman numerals indicate references to conclusions.

5.1.2.1 Recommendation at regulation and wider EC framework level

PRIORITY: LEVEL: A Ensure the existence of a Forests Regulation beyond 2006 1st R-WF

RATIONALE: The existence of a separate, sector-specific legal basis – complementary to the existing instruments - to address sustainable forest sector development is justified by the following factors: (i) the important role of forests in poverty alleviation and in sustainable development, (ii) the official commitment of the Commission to international processes, conventions and treaties which require subsequent action, and (iii) the experience gained (conclusion I, XVII, VI). Such an instrument must, of necessity, display the following features: Swiftness in addressing issues of global concern, implementation of innovative approaches, interventions irrespective of priorities for EC development cooperation mentioned in the CSPs).



OPERATIONAL APPLICATION: Either initiate (Commission Services, HQ-level) drafting of a successor Forests Regulation no later than 2005; or – in case the Commission’s proposal for the merger of all current instruments for external action under the Development Cooperation and Economic Cooperation Instrument146 is adopted – ensure adequate consideration and visibility of sustainable forest sector development in the future DCECI.

PRIORITY: LEVEL: B Commit more funds to sustainable forest sector development under a future Forests Regulation 2nd R-WF

RATIONALE: Given the fact that other financing instruments to date address forest issues only in a limited number of countries and with decreasing budgetary allocation, B7-620 funded forest-related projects play an important complementary role. (conclusion XXIII) However, to realize this role effectively, a higher financial allocation is necessary. OPERATIONAL APPLICATION: Commission Services (HQ level) to lobby for an increased financial allocation, drawing on (i) detailed documentation of experiences & lessons learnt in terms of results and impacts achieved under B7-620, (ii) and, on this basis, a report for dissemination among the Commission Services and the Member States, with particular emphasis on the linkage between forest sector development and poverty alleviation.

PRIORITY: LEVEL: C Strengthen the socio-economic significance of forests and assure a more balanced coverage of all thematic issues 2nd R-WF

RATIONALE: Forests are crucial for poverty alleviation, particularly in rural areas, and yield a broad variety of ecological benefits. However, forestry constitutes an economic (sub-)sector. Therefore, forest-related measures must seek a balance between socio-economic improvements of local livelihoods, and the protection and conservation of forest resources. The social and economic significance of Sustainable Forest Management needs to be further highlighted, and stressed in a new Regulation. The compulsory implementation modalities (e.g. Call for Proposals) are not conducive to the required balanced coverage. (conclusion V, IV) OPERATIONAL APPLICATION: In drafting the new Forests Regulation, (i) remove or tone down the requirement for balanced coverage of thematic aspects, (ii) add a requirement to emphasize more the socio-economic significance of sustainable forest management.

PRIORITY: LEVEL: D Restore the Forests Regulation’s focus on tropical forests 2nd R-WF

RATIONALE: The great majority of forest-related hot-spots exist in the tropics and sub-tropics, and the need for assistance in tropical countries tends to be more pressing than elsewhere. Furthermore, secondary forests as well as plantation forests tend to be overlooked in forest-related development cooperation. At the same time, the financial allocation for forest-sector development has decreased, compared with the position under the previous TFBL-Regulation. Unless financial allocations can be increased, a pragmatic limitation on the geographical focus so as to favour increased impact is required, even though the international forest policy dialogue deliberately renounced the exclusive focus on tropical forests. As yet few proposals have been received from non-tropical countries, and a shift of focus would still be relatively easy to accomplish. (conclusion V) OPERATIONAL APPLICATION: Specify in new Forests Regulation a clear focus on tropical forests, including natural as well as secondary and plantation forests.

PRIORITY: LEVEL: E Enhance synergy between measures under the Forests Regulation and those under other financing instruments. 2nd R-WF

RATIONALE: It is imperative to identify more clearly the comparative advantages and specific strengths of the various financing instruments. This implies also a better understanding of their mutual interrelations. The sum total of the activities would thus become greater than the individual parts, resulting in increased effectiveness and visibility of the EC in forest sector development (conclusion IX). Introduction of the new budgetary framework from 2002 narrowed down the Budget Line’s flexibility to a considerable degree, and thus restricted also some of the Regulation’s comparative advantages. OPERATIONAL APPLICATION: Highlight, in collaboration with the IQSG, the intended comparative advantages of B7-620 activities vis-à-vis other financing instruments, with the aim of removing obstacles to the achievement the Budget-Line’s following opportunities: (i) relatively fast speed of appraisal process, (ii) initiation of innovative pilot approaches, (iii) direct support to almost all 146 pursuant to COM (2004) 626 of September 29th, 2004



implementing agencies with a focus on southern organisations, (iv) swift implementation of priorities defined by EC policies and strategies, (v) support to global projects, (vi) support to forest and environment activities even if they do not rank among the official priorities of the governments of the recipient countries. Once these characteristics have been commonly approved, the administrative and procedural frameworks should be gradually adapted in order to facilitate realisation of the comparative advantages.

PRIORITY: LEVEL: F Integrate sustainable forest sector development in CSP negotiations, where appropriate. 2nd R-WF

RATIONALE: The observed widespread omission of forest-related issues in current CSPs not only underestimates and even distorts forests’ significance on national, regional and global scales, it also results in the Forests Regulation being forced into a compensatory role vis-à-vis geographical financing instruments – a role the Forests Regulation cannot realistically fulfil. At the same time, the formal requirement – according to the evaluators’ interpretation - in Art. 4 [4] makes forest-related measures under B7-620 dependent on their reflecting sector-specific problem analyses and/or response strategies in the respective CSP. This might result in deadlock, effectively precluding forest measures under B7-620 in most countries. (conclusion VII, II) Were forest issues better reflected in CSPs, framework conditions for forest related development cooperation would be more conducive – and the odds of B7-620 projects achieving sustainable impact would thus be generally enhanced. OPERATIONAL APPLICATION: Forest sector development needs to be adequately addressed in all CSP negotiation processes, and CSP negotiators need to be sensitized on forest issues and the Regulation’s requirements. In this context, forests should not only be considered a (global) public good (consideration of which could be made a mandatory part of environmental mainstreaming). In addition, the socio-economic role and benefits of sustainable forestry (being an economic sector) must be emphasized.

5.1.2.2 Recommendations at Budget-Line administration level

PRIORITY: LEVEL: G Better reflect strategic goals and regional needs for forests. 1st ABL /R-WF

RATIONALE: CfPs should be used as an instrument to focus better on regional needs and to support the EC in meeting its strategic obligations by making better use of the comparative advantages of the different financing instruments. In the team’s opinion the following adjustments are possible under the present Regulatory framework. Economic and ecological framework conditions in the Commission’s partner countries vary greatly, and thus require better adapted responses. The same holds true for various cross-cutting issues. To better meet regional needs and reflect regional priorities, the CfP procedure should be regionalized. (XVI) It should be possible to pursue a strategic thematic focus, particularly to promote global initiatives and processes, and to achieve synergies through specific collaboration with ongoing projects under geographical financing instruments (Congo Basin, etc.). This would make possible more strategic use of the B7-620 forest projects budget line so as to improve currently-unfavourable framework conditions at partner county level. OPERATIONAL APPLICATION: A) Regionalize the CfP (define regions according to their common, sector-specific ecological, social and economic properties; identify regional priorities with involvement of the Delegations). B) Facilitate proactive, strategic interventions, there being two options for bringing this about: (i) allowing for targeted activities aside from CfP by repealing the Regulation’s exclusive focus on grants, or (ii) supplementing the Practical Guide on the CfP procedures so as to facilitate ad hoc CfP in addition to the annual programming.

5.1.2.3 Recommendations at project implementation level

PRIORITY: LEVEL: H Ensure continuity and impact of promising projects 1st PI/R-WF



RATIONALE: On average, B7-620 projects are relatively short, particularly when taking into account the fact that forest-related interventions by nature require longer implementation periods. Therefore particular care is needed right from the beginning. Projects seeking a change of attitude of the target group at local level should be especially carefully screened and closely monitored in respect of their practicability within the proposed time-frame. OPERATIONAL APPLICATION: A set of respective mechanisms needs to be installed at the following different levels: (i) project selection and design (check for realistic approach and corresponding OVIs); (ii) monitoring and evaluation (assure flexible adaptation of approach, and focus on exit strategies); (iii) EC Delegations should facilitate take-over by other financing instruments, donors, or the partner country itself; (iv) work towards an amendment of the Financial Regulation, so as to allow additional funding for second phase for exceptionally innovative and promising grant projects, without a Call for Proposals procedure (but based upon the outcome of the projects’ evaluation).

5.2 Conclusions and recommendations for the Environment Regulation

5.2.1 Conclusions for the Environment Regulation

5.2.1.1 Relevance

X. The Environment Regulation is coherent with the relevant EC policy framework and authoritative international regimes {R-WF} The EC has adopted an “environmental mainstreaming approach” in the recent past, whereby the environmental dimension has to be integrated into all aspects of development cooperation [42]. The Environment Regulation is instrumental to this end [38]. The EC’s understanding of what constitutes the said “environmental dimension” is mostly determined by international processes, initiatives and multilateral agreements, to which the EC is a party [39]. These international regimes reflect the concept of Sustainable Development, introduced into the international policy dialogue at the 1992 UNCED in Rio de Janeiro [33].


XI. Procedural restrictions reduce the Commission’s discretion to pursue mainstreaming activities at the global level {ABL} The new procedural framework restricts the use of targeted projects for proactive measures and initiatives taken by the Commission in accordance with the limitations prescribed in the budgetary framework for the award of directs grants [22]. In practice, targeted projects are henceforth restricted to co-funding of multi-donor programmes run by international organisations or, in rare cases, to funding of organisation with a de facto or de jure monopoly.

XII. The thematic composition of the project portfolio blunts the Environment Regulation’s comprehensive mainstreaming strategy {ABL/PI} Environmental mainstreaming takes place mostly at global and regional levels, resulting in building up the of national (and decentralised) policy-making capacity [8]. The Environment Regulation’s approach to environmental mainstreaming is comprehensive, as reflected by the broad scope of its prescribed activities. In practice, however, mainstreaming activities under the Environment Regulation focus on conservation or sustainable management of natural resources [1]. These two focal thematic areas give rise to considerable thematic overlap with B7-620 forest projects [2], whereas other thematic components147 of environmental mainstreaming are either underrepresented or completely missing [1]. This is due to Strategic Orientation and Annual Programming, as reflected in successive issues of the Strategic Guidelines and Guidelines for Applicants [2].

147 Notably: Environmental problems related to industries, management of freshwater resources, sustainable production

and use of chemicals / energy.



XIII. The financial allocation for environment, and the definition of thresholds for the minimum size of grants, restrict effective mainstreaming {ABL} The available financial envelope148 for environmental measures is disproportionately small, compared to the Environment Regulation’s comprehensive mainstreaming approach [3]. In view of this basic limitation, measures which promote mainstreaming in the most cost-effective way gain particular importance, such as training, seminars, handbooks, awareness campaigns. These measures commonly involve relatively small budgets. However, by defining a minimum grant size of € 0.5 million for environment projects in the Guidelines for Applicants, implementation of the aforementioned small-scale measures has been effectively ruled out, even though no such minimum thresholds are prescribed in the Regulations [26]. Implementation of a larger number of small-sized projects would increase the Commission’s workload, and holds implications for Commission services’ staffing and resource allocation.

5.2.1.3 Efficiency For detailed discussion of efficiency, refer to section “Both Regulations” below.

5.2.1.4 Impact/Sustainability XIV. The Environment Regulation’s impact materialises more at the policy level than in

respect of local livelihoods {PI} Projects within the Environment Regulation’s purview predominantly address local, regional and national policy-makers and decision-makers, as well as technical and administrative staff. Fewer projects address local communities directly [14]. Therefore, direct impacts on local livelihoods are achieved less frequently than under the Forests Regulation. This is most probably due to the Environment Regulation’s focus on mainstreaming.

XV. Mainstreaming by definition builds on multiplier effects, and therefore is a finite task {ABL/PI} Mainstreaming implies that environmental issues are reflected in other sectors and contexts, and that various actors and stakeholders (within both the EC and partner countries) are sensitised to the environmental dimension in the development process. Even though this process is not yet complete, [52] initial success is evident as shown by the reflection of environmental issues in a clear majority of the assessed CSPs [44].

5.2.2 Recommendations for the Environment Regulation The evaluators have chosen to prioritise their recommendations as first or second priority level. Roman numerals indicate references to conclusions.

5.2.2.1 Recommendation at regulation and wider EC framework level

PRIORITY: LEVEL: I Ensure an Environment Regulation’s continued existence beyond 2006, with a clearer focus on mainstreaming. 1st R-WF

RATIONALE: Limited funds and the Environment Regulation’s broad thematic scope call for an implementation strategy that has clearer focus on mainstreaming (as already reflected in the Regulation’s title). As mainstreaming at EC level is now increasingly being implemented, mainstreaming activities should concentrate at the level of partner countries. Furthermore, the budget line should be open to selected small-scale interventions (such as IKM, participatory policy formulation etc.) (conclusion X, XIV, XV). OPERATIONAL APPLICATION: Either initiate (Commission Services, HQ-level) drafting of a successor Environment Regulation no later than 2005; or – in case the Commission’s proposal for the merger of all current instruments for external action under the Development Cooperation and Economic Cooperation Instrument149 is adopted – ensure adequate consideration and visibility of environmental mainstreaming and the Environment Regulation’s thematic scope in the future DCECI.

148 Art. 7 ENV; Euro 93 million during the period 2000-2006 149 pursuant to COM (2004) 626 of September 29th, 2004



PRIORITY: LEVEL: J Increase budgetary commitments. 2ND WF/R-WF

RATIONALE: Notwithstanding the need to carefully gauge the mainstreaming process’s progress and requirements, it stands to reason that an increase in budgetary commitments will be required on account of the large geographical scope and the limited progress of some partner countries in mainstreaming environment-related issues. OPERATIONAL APPLICATION: Commission Services (HQ level) to lobby for an increased financial allocation based on the outcome of the monitoring of progress.

PRIORITY: LEVEL: K Address broad thematic scope, but drop the requirement of a balanced coverage, and emphasize the need for measures regarding urban and industrial environment.

2ND R-WF

RATIONALE: At present, the obvious preoccupation with natural resource conservation in rural areas distorts the Environment Regulation’s broad thematic scope. According to recent population-growth statistics from the UN, more than half the world’s population will live in urban areas by 2007. Given this trend, a shift of focus is clearly required for environmental measures under B7-620. Mainstreaming activities should thus increasingly address water and air quality, sanitation and health, waste treatment, sustainable energy, technical and chemical hazards, and the like. However, this will not apply evenly to different regions, and underpins the need to focus strategic orientation (and, consequently, the CfP) more visibly on regional characteristics and needs. (conclusion XII) OPERATIONAL APPLICATION: In drafting the new Environment Regulation, (i) remove or tone down the requirement for balanced coverage of thematic aspects, (ii) emphasize mainstreaming through a broad range of thematic options, (iii) highlight further urban and industrial environment.

PRIORITY: LEVEL: L Limit overlaps with forest related issues. 2ND R-WF

RATIONALE: Considering the limited funds available for environmental measures, it appears crucial to avoid obvious overlaps with the Forests Regulation (e.g. SNRM and conservation of forest ecosystems) so as to leave more room for the above-mentioned issues. This necessitates more operational and clearly defined guidance in the Environment Regulation on the circumstances under which environment-related measures can focus on SNRM (or other issues prone to thematic overlap). (conclusion XII) OPERATIONAL APPLICATION: Define clear-cut criteria for the selection of SNRM projects under the Environment Regulation. Reflect criteria in the Guidelines for Applicants.


PRIORITY: LEVEL: M Closely follow up on mainstreaming progress. 1ST ABL

RATIONALE: Determining the exact nature as well as the progress of environmental mainstreaming at various levels (EC-HQ, Delegations, regions, partner countries, stakeholders, etc.) requires a clear understanding of (i) what mainstreaming means in practice, (ii) how the progress of mainstreaming and, consequently, implementation success in respect of the Environment Regulation can be gauged. This is necessary to make informed choices about the Regulation’s future role and existence. (conclusion XV) OPERATIONAL APPLICATION: Define, monitor, and document criteria & indicators to gauge the progress of environmental mainstreaming at different levels. This should be made a task of the appropriate Help-Desk, as well as definition of tools for IKM and dissemination (at project/partner country level) in respect thereof.

PRIORITY: LEVEL: N Focus the CfP procedure on global and regional mainstreaming needs. 2ND ABL

RATIONALE: Individual environment projects, while subject to the overall mainstreaming rationale, must address specific deficits at regional, country and stakeholder level. Since deficits in relation to cross-cutting issues vary regionally, responses should also be regionalized and reflected in Strategic Orientation. In Africa, for example, mainstreaming would particularly emphasize the link between environmental degradation and poverty, whereas in Asia measures should first and



foremost aim at mitigating any negative consequences of (infra-)structural development: urban and industrial pollution, hazardous wastes, sanitation, etc.. To complement the regional focus, the Practical Guide on CfP procedures should be modified so as to make possible ad hoc CfP, detached from the annual programming cycle. It would thus be possible to pursue strategic thematic foci at a global level without running the risk of violating budgetary restrictions on the use of direct grants. (conclusion XI, XIV, XII) OPERATIONAL APPLICATION: A) Regionalize the CfP (define regions according to their common, sector-specific ecological, social and economic properties; identify regional priorities, with the involvement of the Delegations). B) Facilitate proactive, strategic interventions, there being two options for bringing this about: (i) allowing for targeted activities aside from the CfP by repealing the Regulation’s exclusive focus on grants, or (ii) supplementing the Practical Guide on the CfP procedures so as to facilitate ad hoc CfP in addition to the annual programming.

PRIORITY: LEVEL: O Allow for small-scale mainstreaming interventions in the Guidelines for Applicants. 1ST ABL

RATIONALE: Mainstreaming measures can be implemented in an efficient manner, even though their funding may be limited and implementation periods relatively short. Reducing the current minimum grant size of € 0,5 million would open up the budget line to small-scale environmental interventions, for example in relation to IKM. (conclusion XIII) OPERATIONAL APPLICATION: Drop the minimum threshold in the next guidelines for applicants.

5.3 Conclusions and recommendations for both Regulations

5.3.1 Conclusions for both Regulations

5.3.1.1 Relevance

XVI. Mainstreaming of cross-cutting issues has not yet been consistently achieved and justifies renewed efforts {ABL/PI} Both Regulations provide a highly relevant legal basis for addressing cross-cutting issues150 [49], which is so far taken up by and reflected in Strategic Orientation exercises [50]. As regards practical implementation, most projects focus on capacity strengthening and institutional development, and contribute to several others, owing to their broad scope [52]. Regional differences are noticeable in respect of perception of individual cross-cutting issues, this conclusion was confirmed by the EC Delegations’ judgement of their respective countries’ situation and level of development. Particularly noteworthy is the underestimation of the inherent linkage between poverty and environmental degradation, as becomes apparent from the Africa-based EC Delegations’ feedback to the written poll [51]. This raises doubts about the success of environmental mainstreaming, particularly in African countries.


XVII. Given their good performance, projects under B7-620 have a high probability of contributing to the Regulations’ stated objectives {PI} During the field phase, a project sample was assessed applying the DAC criteria, and an overall rating of projects under B7-620 was formulated on this basis. A large proportion of projects (19 out of 27) were rated as either “excellent” or “good”, which indicates that they will contribute significantly to the achievement of the two Regulations’ stated objectives. The evaluators are confident that a definite majority of the projects will achieve their intended results, even though this may require extension of their respective implementation periods [6]. Compared to the findings of the 1998 evaluation of TFBL-projects (irrespective of its sectoral focus), overall project quality (in terms of project planning and design) has improved. This improvement is much to the credit of the Call for Proposals Procedure (introduced under the new Financial Regulation), whereby strict

150 broad based equitable growth, social services, environment, gender issues, capacity and institutional building, private

sector development, human rights and good governance



external selection by predetermined quality criteria (e.g. use of logical frameworks and PCM) is required [20].

XVIII. Remaining weaknesses with regard to project performance {PI} A remaining weakness related to project design is the definition of the OVIs. Although regularly applied, they remain persistently weak, and can often not be verified [5]. This also impedes monitoring and evaluation (see also XXII below), and obstructs flexible adaptation and control of project implementation. In regard to impact and sustainability the bottleneck is the relatively short implementation period. Ordinarily, forest and environmental projects tend to run for longer periods, than do projects in other sectors. Implementation of forest field-projects e.g. may well stretch into decaB7-620 are short (42 months) compared to the tophases) under other financing instruments. This notin terms of purpose and impact (while they neverthimplementation phase), but also restricts their coRegulation’s stated objectives [6]. Such limitationB7-620 projects by other financing instruments. involvement of the partner countries’ national sectfrom being continued beyond the phase of Commiss

XIX. Co-funding warrants critical reflection in regefficiency {ABL} In principle co-funding contributes to efficient impleEC can expect to implement more projects unindividual projects may extend the scope and size oof co-funding depends upon transparent documeminimum 20 % contribution either translates into exin the EC’s contribution. During the field phase it wacases (owing mostly to the weak formulation of co-funding arrangements’ positive contribution to pr Furthermore, beneficiaries mostly perceive the coadds to their administrative burden, and they comco-funding rules by the Commission Services (e.g. kind) [32].

XX. (i) Interpretation and practical enforcemendisadvantages for certain applicants result in los In past issues of the Guidelines for Applicants, certo the Regulations, have been excluded [25]. This the highly demanding and time consuming Carequirements, together results in situations where menvironmental NGOs. Consequently, NGOs based in the partner countribeneficiaries. However, as has been observed durinasymmetric, i.e. the northern partner occupies a dsouthern NGOs rarely takes place in practice. ThAfrica, whereas northern and southern NGOs cAmerican countries.

Examples of indicators on project purpose levelbeing not SMART (specific, measurable,attainable, relevant and trackable) in their entirety - Economic benefit to the participating Community

(45) - Proper management practices enacted (80) - A halt in the decrease of critical forest resources

(96) - Increased participation by women in decision-

making processes (60) - Impact of the improvements on the sustainable

management of the resources (36)

04 65

des. Funding periods for forest projects under tal duration of forest projects (with successive only limits the projects’ individual achievement eless mostly achieve the results for the current ntributions to the achievement of the Forest

s are not compensated for by a take-over of Lack of donor coordination and insufficient

or administrations [11] further prevent projects ion support [36].

ard of its contribution to the Budget Line’s

mentation of measures in two ways. First, the der the same financial framework. Second, f their activities. Either way, the positive effect ntation of how co-funding in excess of the

tra activities, or into a commensurate decrease s found that transparency was lacking in some OVIs) and that, in consequence, the existing oject efficiency could not be gauged [7].

-funding requirement as burdensome since it plained about the inconsistent application of

in respect of the acceptance of contributions in

t of eligibility criteria and (ii) structural t opportunities {ABL}

tain categories of applicants, eligible according a priori exclusion of applicants, combined with ll for Proposals procedure and co-funding ost beneficiaries are northern or international

es participate mainly in partnership with these g the field phase, such partnerships tend to be

ominant position, and capacity building for the is observation applies mainly to sub-Saharan ooperate more equitably in Asian and Latin



The fact that private sector operators are effectively excluded from participation as grant beneficiaries – combined with the structural preference for NGOs with a predominantly environmental focus – results in a systematic under-representation of the economic dimension of sustainable development. Finally, exclusion of state (sector) administrations as grant beneficiaries [17] limits the Regulations’ effectiveness as regards national policy formulation and legal reform processes.

5.3.1.3 Efficiency

XXI. The first Call for Proposals resulted in delays, and waste of resources {ABL} In the Guidelines for Applicants under the first Call for Proposals, the Commission stated its intention to award no more than 35 grants. About 20 times as many proposals were received [20]. This overwhelming response to the first Call for Proposals derailed the complex sequence of Annual Programming and Strategic Orientation. Project selection and contracting were delayed, which must be considered a logical consequence given the small number of task managers concerned with Budget Line administration in AIDCO F4. Outsourcing of the first screening of applications to external consultants cannot compensate for the obvious lack of (human) resources assigned to the management of the Environment and Forests Regulations. The Commission Services subsequently perceived the need to continue selection of projects from the first batch of proposals received (instead of launching a subsequent Call for Proposals in 2002) [21]. Furthermore, AIDCO wished to increase the likelihood of success for the applicants. From the applicants’ viewpoint, the process was nevertheless wasteful, and the likelihood of success perceived as low [20].

XXII. Lack of standardised M&E and reporting instruments impedes efficient control {ABL/PI} So far, existing internal and external monitoring systems have not been streamlined into an integrated monitoring system. Internal monitoring is conducted according to the individual contractor’s own commitment as defined in the proposal (project description). External monitoring, on the other hand, is limited to a random selection of individual projects (non-specific to individual financing instruments) within one country or region (ROM). Both Regulations require for the Commission to regularly evaluate B7-620 projects151. However, ROM does not provide an instrument for meeting the Regulations’ requirement for external evaluation of the projects. Because there is no standardised (indicator-based) progress-reporting system or format, reporting duties are perceived as time-consuming and burdensome by both the project implementers and the task managers involved. Consequently, the Commission Services do not adequately benefit from the experience and accumulated knowledge from the projects [29] . No encoding system152 has been put in place (e.g. in the CRIS database) which would allow quick classification of individual projects (and indeed the entire project portfolio) according to their focal tasks and contribution to cross-cutting issues.

5.3.1.4 Impact /Sustainability

XXIII. Lack of mechanisms and arrangements for feeding B7-620 projects into geographical programming hinders achievement of impact and sustainability of pilot measures {ABL} Both Regulations aim at innovative or “pilot” measures which, by definition, require continuation to achieve their full effect. Owing to the fact that EC Delegations have not represented B7-620 projects so far in donor coordination fora, the taking over by other donors of particularly promising pilot projects does not take place. Furthermore, positive experiences have not been fed back into geographical programming in a systematic manner, which (among other reasons, e.g. inadequate

151 Art. 10 [2] ENV and Art. 11 [2] FOR 152 For example, German Development Cooperation has long since adopted an elaborate system of “gender codes”,

“environment codes”, “poverty codes” etc., which are uniformly presented on each project sheet



presentation of B7-620 projects in donor coordination fora) narrows down the options for continued assistance by the Commission [36],[45].

XXIV. The Commission Services do not capitalize on lessons learnt and experiences, and hence fail to pass on opportunities for institutionalised learning {ABL/PI} Institutional learning processes on the Commission’s part are constrained since feedback from individual projects is still not synthesised in a systematic manner [29]. No systematic compilation of project experience is available apart from fragmentary documentation, which is produced mostly through the individual projects’ own initiative. Where projects communicated with the Commission Services, the focus has so far been on administrative issues. Regular subject-matter dialogue has only recently been institutionalised through the newly-introduced practice of calling regional information-exchange meetings for B7-620 project staff. 26 representatives of projects in Africa convened in Addis Ababa in March 2004, the first meeting of this kind. A similar exercise has by now been conducted in Lima (Peru), and will soon take place in New Delhi. In India, the EU is a member of a Joint Working Group on environment – but its role is more reactive than pro-active, as policy issues still tend to be handled in Brussels. Exercises of this kind contribute, through discussion of lessons learnt and procedural aspects, to the aforementioned “aggregate coherence”, and promote consultation and coordination at the same time. Similarly beneficial effects can be expected with respect to both the quality of project implementation and sustainable impact.

5.3.1.5 Coordination, Coherence and Complementarity

XXV. EC services at HQ-level coordinate their activities internally, as well as vis-à-vis EC Member States and other donors {ABL} Various legal norms oblige the Commission to proactively seek coordination internally, as well as with Member States and other donors [34], [35]. Furthermore, Strategic Orientation and Annual Programming, as well as project selection, is done through an iterative process of inter-Service consultation, including the EU Member States.

XXVI. In practice, other donors do not benefit from experience and lessons learnt from B7-620 projects {ABL/PI} The fact that, contrary to what is required by the Regulations, there is no systematic subject-related reporting on measures funded under B7-620, prevents other donors, other EC Services and other stakeholders from capitalizing on information about the implementation progress [37].

XXVII. So far, coordination (and, consequently, also coherence and complementarity) are not ensured at partner-country level {ABL} Prior to deconcentration EC Delegations had no mandate to engage proactively in the administration of B7-620 projects. For this reason, coordination with environment and forest projects under other financing instruments was largely coincidental, and left up to the individual EC Delegation’s initiative. The same holds true for donor coordination fora, where EC Delegations represented geographical financing instruments but not necessarily horizontal instruments [36].

5.3.2 Recommendations concerning both Regulations


PRIORITY: LEVEL: P Improve efficiency of CfP procedures. 1ST ABL

RATIONALE: While the Regulations provide for broad thematic scope, Strategic Orientation serves to define operational thematic clusters. This selection of priorities should reflect regional needs and priorities more clearly. Further, applicants should be required to demonstrate more clearly in



their proposals the innovative character of their proposed projects. In combination, these requirements would help prevent submission of excessive numbers of “blue-print” proposals. (XXI) Initial selection of proposals on the basis of short, concise concept notes could help shorten the overall selection procedure, and reduce the related costs and effort on the applicants’ part. Other donors’ experience with this approach suggests that it warrants careful consideration. The Commission should investigate the potential advantages of a two-step concept-paper approach. (conclusion XXI) OPERATIONAL APPLICATION: A) Define more clear-cut, regional priorities so as to limit the number of ; B) investigate the possible impacts of a two-stage approach on (i) the range of stakeholders, (ii) the innovativeness of proposals, (iii) saving of time & resources, and (iv) better reflection of priorities, and subsequently document experiences and lessons learnt in a systematic manner.

PRIORITY: LEVEL: Q Provide applicants with timely information on the CfP

procedures 1ST ABL RATIONALE: All applicants should be informed from the very beginning of the process by which they may expect a decision on their proposals. Where the time-span between publication of the CfP and the initial disbursement of funds threatens to exceed the prescribed deadlines, information is crucial in order to avoid lingering discontent on both the contractors’ and partner-country stakeholders’ part. (conclusion XXI) OPERATIONAL APPLICATION: Define, shorten, publish, and monitor deadlines & milestones of CfP procedure in a more proactive manner. Furthermore the CfP should include information on the approximate number of projects to be selected.

PRIORITY: LEVEL: R Drop the practice of excluding certain potential applicants 1ST ABL

RATIONALE: The Commission should design eligibility criteria in the Guidelines for Applicants in accordance with the rules set out in the Regulations (conclusion XX) . Various stakeholders can contribute to B7-620 measures with their specific capacities and experiences. For maximum impact and flexibility in different fields of action - for example strategic policy contributions and support of sector-reform processes in partner countries - the scope of eligible actors should be kept as broad as possible. OPERATIONAL APPLICATION: When drafting Guidelines for Applicants, adhere to eligibility criteria for applicants as stipulated in the Regulations.

PRIORITY: LEVEL: S Improve role of southern partners 2ND ABL

RATIONALE: Where eligible northern and southern partners join forces as a partnership, roles and responsibilities (e.g. in respect of the allocation of funds) should be strengthened by the Commission. This would promote the Commission’s aim of building southern partners’ capacities and institutional strength. (XIX, XX) OPERATIONAL APPLICATION: Define criteria for more equitable partnerships between northern and southern partners, and include them in the Guidelines for Applicants.

PRIORITY: LEVEL: T Provide EC Delegations with adequate resources for the budget line’s administration 1ST ABL/R-WF

RATIONALE: Following deconcentration, the Delegations’ mandate for the administration of B7-620 comprises: (a) involvement in project selection, (b) supervision and management of M&E, (c) representation of B7-620 projects vis-à-vis other financing instruments and in donor fora, (d) involvement during field visits and (e) liaison with national sector administrations. Such a broad mandate will enable Delegations to promote replication of promising measures on the basis of effective monitoring. Commission Services will thus be better placed to undertake proactive policy advocacy. For these purposes EC Delegations need adequate financial as well as human resources. (conclusion XVII) OPERATIONAL APPLICATION: Enhance the resources available to EC Delegations within their extended mandate for budget-line administration.

PRIORITY: LEVEL: U Improve Delegations’ technical capacities 2ND ABL

RATIONALE: Given their regional knowledge, Delegations are in a particularly favourable position



to facilitate enhanced subject-matter dialogue and provide lead points for regional networks of forest and environment projects across all financing instruments. Therefore, they should have the mandate to influence Strategic Orientation and CfP preparation from a regional perspective. Because EC Delegations cannot be expected to employ experts for either forest sector development or environmental issues, the creation of regional focal points with specific expertise would provide professional support in a cost-effective manner. This would entail feedback of needs and priorities upwards from EC Delegation level, with the regional focal points exercising a relay function vis-à-vis HQ. (conclusion XXIV, XXVI, XXVII) OPERATIONAL APPLICATION: Establish regional focal points, with adequate sectoral expertise, to provide information services to several EC Delegations. Regional focal points should be designed in consideration of geographical financing instruments (ACP, ALA, MEDA, TACIS). They should act in a relay role, so as to document, capitalise on, and disseminate lessons learnt through regional workshops, publications, and field visits. They would further integrate all Commission-supported forest or environment projects into thematic networks.

5.3.2.2 Recommendations at project implementation level

PRIORITY: LEVEL: V Improve the quality of the OVIs 1ST PI

RATIONALE: Assessment of project effectiveness, efficiency, and impact is considerably impeded by the fact that in most cases OVIs are inadequately identified. Furthermore, failure to adapt and update OVIs after a project’s inception leads to their not being used as control instruments in PCM. It is also clear that more emphasis needs to be put on (impact-oriented) OVIs which should be used during M&E, as well as during other PCM stages as appropriate. (conclusion XVIII, XXII) OPERATIONAL APPLICATION: A) Make further disbursement conditional on revision and approval of OVIs. Following their inception, projects should be allowed a period of about three months to review, and if necessary adapt, tentative OVIs from the project proposal, based on participatory validation with major stakeholders during planning of operations. B) To improve the quality of the OVIs, the Commission should consider the drafting of a sector specific manual giving guidance on the formulation of forest- or environment-related OVIs.

PRIORITY: LEVEL: W Make EC contributions more transparent and visible 2ND PI/ ABL

RATIONALE: To gauge the Budget Line’s efficiency, it is necessary to attribute measurable achievements at the field level to EC funding. This requires transparency of co-funding support from other donors and sources, especially in those cases, where additional funding exceeds the relative share stipulated in a given project’s contract. As a general rule, all additional funding aside from that stipulated in the contract must be disclosed throughout project implementation. (conclusion XIX) OPERATIONAL APPLICATION: Add a clause to the standard contract form such as: “The Commission Services must be notified by the contractor of any additional funding received from third parties in excess of the fixed co-funding ratio”. This will necessitate adoption and use of OVIs which allow the Commission to verify quantitative as well as qualitative achievements in respect of the disbursed EC funds.

PRIORITY: LEVEL: X Harmonise and improve existing M&E instruments 1ST PI /ABL

RATIONALE: For the sake of enhanced transparency and comparability of the projects’ progress, monitoring done on the contractors’ initiative should be standardised and based upon OVIs. Furthermore, the Regulations require evaluations initiated by the Commission Services, an obligation which so far has not been sufficiently met. This is the basis for institutional learning and for providing information to all stakeholders (e.g. donors). (conclusion XXIV)

OPERATIONAL APPLICATION: A) Add to the contracts a specific clause to link monitoring to OVI, and prescribe standardised formats. B) Adhere to evaluation requirements more uniformly in Budget-Line administration.

PRIORITY: LEVEL: Y Capitalize on experiences and lessons learnt 1ST PI/ABL

RATIONALE: Reporting formats need to reflect the logical framework in their structure, and build on systematic assessment of OVIs. Reports should be exhaustive in that they provide task managers



with all the information required for a comparison of planned versus achieved outcomes at the levels of purpose, results and activities, and drawing attention to (i) changes in risks and assumptions, (ii) any need for adaptation of planning, (iii) contributions by third parties, and (iv) recommendations for action to be taken by the EC Delegation. Standardised reporting is a precondition for quick reference by the task managers involved and for institutional learning by all the concerned Commission Services. (conclusion XVIII, XXII, XXIV) OPERATIONAL APPLICATION: Introduce standardised reporting formats, drawing on impact-oriented OVIs. A given project’s specific characteristics (e.g. in respect of cross-cutting issues) should be captured through an encoding system (reflecting the key objectives mentioned in the Regulations).