annual work programme 2013 - europaenrd.ec.europa.eu/enrd-static/fms/pdf/b6b48b03-0bad-ed37...jan...

22
ANNUAL WORK PROGRAMME 2013

Upload: others

Post on 22-Mar-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

ANNUAL WORK PROGRAMME 2013

Page 2: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

2

Page 3: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

3

Table of Contents

INTRODUCTION ..................................................................................................................................... 5

WORK PROGRAMME ............................................................................................................................. 7

1. TO IMPROVE METHODS AND TOOLS IN EVALUATING RURAL DEVELOPMENT PROGRAMMES (CONTENT DEVELOPMENT FUNCTION) ......................................................... 7

1.1 To improve evaluation methodology and processes of RD Programmes ........................... 7 1.1.1 Thematic Working Group ................................................................................................................ 7 1.1.2 Develop further topics from the ‘Thematic Pool’ ............................................................................. 8

1.2 To improve the common understanding of evaluation methods and tools ......................... 8 1.2.1 Develop a Glossary ........................................................................................................................ 8 1.2.2 Develop Frequently Asked Questions ............................................................................................ 9

1.3 To improve and update guidance related to the CMEF in the current and future programming period ................................................................................................................. 9

1.3.1 Update guidance related to the CMEF (current period) .................................................................. 9 1.3.2 Support the CMEF review (future period) ....................................................................................... 9

1.4 To improve the response to the needs in terms of RD evaluation ..................................... 10 1.4.1 Update the assessments of needs carried out in 2012 ................................................................. 10 1.4.2 Prepare the 2014 Annual Work Programme................................................................................. 10

2. TO INCREASE CAPACITY IN IMPLEMENTING RD EVALUATION PROCESSES (TECHNICAL SUPPORT FUNCTION) ............................................................................................................... 12

2.1 To improve the assessment capacity in the field of RD evaluation .................................... 12 2.1.1 Support the assessment of the Annual Progress Reports for Rural Development Programmes

submitted in 2013 ......................................................................................................................... 12 2.1.2 Training of EC Desk Officers on Monitoring and Evaluation ......................................................... 12

2.2 To support the implementation of the ongoing evaluation ................................................. 13 2.2.1 Support the capacity-building on evaluation in the Member States through targeted actions ...... 13 2.2.2 Collect best practices in implementing the CMEF methodology ................................................... 13

3. TO SHARE AND DISSEMINATE KNOWLEDGE IN THE EVALUATION OF RURAL DEVELOPMENT PROGRAMMES (DISSEMINATION & NETWORKING FUNCTION) .............. 15

3.1 To ensure effective dissemination of information regarding evaluation methodology and processes ................................................................................................................................ 15

3.1.1 Establish the presence of the Evaluation Expert Network and its Helpdesk ................................. 15 3.1.2 Manage a library of relevant studies and other relevant information ............................................ 16 3.1.3 Run the Evaluation Expert Network’s website .............................................................................. 16 3.1.4 Produce newsletters ..................................................................................................................... 16 3.1.5 Manage a mailing and distribution list database ........................................................................... 17

3.2 To improve collaborations and foster synergies between stakeholders ........................... 17

Page 4: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

3.2.1 Run the Evaluation Helpdesk ....................................................................................................... 17 3.2.2 Follow up on the activities of the Evaluation Helpdesk ................................................................. 19 3.2.3 Collaborate with the Expert Committee on Evaluation of Rural Development Programmes ........ 20 3.2.4 Liaise with Member States ........................................................................................................... 21 3.2.5 Collaborate with the European Network for Rural Development .................................................. 21

Page 5: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

5

Introduction

Council Regulation (EC) No 1698/2005 of 20 September 2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) foresees in Article 67, as a part of technical assistance for rural development policies, a European Network for Rural Development to be established at Community level. This Article on the European Network for Rural Development stipulates different tasks, including to “set up and run expert networks with a view to facilitating an exchange of expertise and supporting implementation and evaluation of the rural development policy” (Article 67 (e)). Against this background the European Commission has set up, as a part of the European Network for Rural Development, a European Evaluation Network for Rural Development (hereinafter referred to as "Evaluation Expert Network") that fulfils the evaluation-related functions foreseen by the aforementioned Article. Organisation The Evaluation Expert Network works under the responsibility of the evaluation function of the Directorate-General for Agriculture and Rural Development. The Evaluation Expert Network is supported and supervised by a Steering Group that involves the competent units of the Directorate-General for Agriculture and Rural Development, as well as other Commission services. The work of the Evaluation Expert Network related to the exchange of expertise and the establishment of best practice on evaluation of the rural development policy is followed by the Expert Committee on Evaluation of Rural Development Programmes (Evaluation Expert Committee). This Committee was established by Commission Decision of 20 February 2008 setting up the organisational structure for the European Network for Rural Development (2008/168/EC). It is composed of two representatives from each national competent authority and chaired by a representative of the Commission. Actors at the level of Member States, as well as at programme level (administrations, evaluators, academics, stakeholders) will be involved via seminars, workshops, discussion of thematic studies, and the dissemination of a newsletter. The technical task of the Evaluation Expert Network (the Evaluation Helpdesk) is carried out by an external contractor, EEIG RurEval. Evaluation Helpdesk The Evaluation Helpdesk serves as central information point concerning the evaluation of Rural Development Programmes and assists in the establishment and the management of the Evaluation Expert Network. Moreover, the Helpdesk provides expertise and guidance on methodological issues, such as evaluation practices and data collection, and provides support to the Commission and to the Member States in dealing with evaluation reports. A dedicated set of technical support services and tools is provided by the Helpdesk. These include a trilingual website (English, French and German), an electronic newsletter, a question and answer service, a glossary of terms, best practice examples and access to key literature. The Helpdesk is composed of a permanent team of staff in Brussels and is supported by around twenty experts from across the 27 EU countries with knowledge in the field of evaluating Rural Development Programmes. Thematic working groups are set up to analyse and draw conclusions on key themes such as assessment of the socio-economic and environmental impacts, assessment of the Leader approach, etc.

Page 6: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

6

Objectives The overall objective of the Evaluation Expert Network is to increase the usefulness of monitoring and evaluation as tools for improving the formulation and implementation of rural development policies. This will be achieved by:

improving methods and tools in the evaluation of Rural Development Programmes,

increasing capacity in implementing the evaluation process, and by

sharing knowledge in the evaluation of Rural Development Programmes. The graph below presents the three specific objectives mentioned above, broken down into operational objectives:

Specific Objectives Operational Objectives

To improve methods and tools in evaluating Rural Development Programmes

To increase capacity in implementing RD evaluation processes

To share knowledge in the evaluation of Rural Development Programmes

To improve evaluation methodology and processes

To improve the common understanding of evaluation methods and tools

To improve and update guidance related to the CMEF in the current and future programming period

To improve the assessment capacity

To support the implementation of the ongoing evaluation systems

To improve quality standards of RD evaluations

To improve the response to the needs

To ensure effective dissemination of information regarding evaluation

To improve collaboration and foster synergies between evaluation stakeholders

Page 7: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

7

Work Programme

1. To improve methods and tools in evaluating Rural Development

Programmes (content development function)

1.1 To improve evaluation methodology and processes of RD Programmes

The Evaluation Expert Network adopts a thematic approach in order to support the development and improvement of evaluation methods, with a view to contributing to the increase of capacity in the field of evaluation of Rural Development Programmes.

1.1.1 Thematic Working Group

The following tentative topics could be covered by a Thematic Working Group:

Guidelines for the Evaluation Plan of RDPs 2014-2020

Guidelines for the RDP ex post evaluation 2007-2013 The objective of the Thematic Working Group is to prepare, based on the experiences of the current programming period, relevant background documents that can be used to improve the guidance for the current and next programming period. This working group is designed as a think-tank concerning RD evaluation methods and is composed of evaluation experts and practitioners from the Member States. Workshops will be the preferred working method of the group. The analytic results of the working group feed into the CMEF review process. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Potential Outputs:

Thematic working paper(s), guidance documents (exact output and timeline to be defined in Concept Note)

Page 8: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

8

1.1.2 Develop further topics from the ‘Thematic Pool’

A number of topics from the needs assessment have been included in the “Thematic Pool” for the

further content work of the Helpdesk. E.g. in 2010, 2011, 2012 the Helpdesk provided support for the assessment of National Rural Network Programmes. In 2012 “approaches of organising ongoing evaluation” and “capacity building” have been followed up. In an interactive session with the Evaluation Expert Committee on potential topics for further support should be developed discussed. The following topics could be further developed in 2013:

It is suggested that the topics of the “Thematic Pool” follow the Good Practice workshops: Based on the topic of the respective good practice workshop short input papers (briefing notes) should be drafted by thematic experts.

The presented list is not exhaustive and can be amended according to the most urgent needs. These topics can be tackled by thematic working groups, ad-hoc workshops, quick-surveys, investigations, secondary literature research, collection of good practices, etc. The extent of coverage and the most appropriate output will be decided on a case by case basis. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Potential Outputs

e.g. thematic input papers (briefing notes), support documents, workshop-minutes, explanatory documents

1.2 To improve the common understanding of evaluation methods and tools

In 2012 the Evaluation Helpdesk published the trilingual glossary of terms related to the evaluation of rural development along with the set of FAQs on its website. Work will continue in 2013 to further develop and strengthen the contents. The dissemination of these tools will be organised via the Network’s website – cf. Activity 3.1.3 - and the Network’s newsletter – cf. Activity 3.1.4 mainly.

1.2.1 Develop a Glossary

This activity will chiefly consist of a single ongoing activity:

1.2.1.1 Further develop a glossary of terms on evaluation of rural development

The development of the glossary of terms on evaluation of the Rural Development Programmes will be continued. New terms to be included in the glossary will be identified in relation to the other activities performed by the Evaluation Expert Network, in particular the theme-based activities aiming to improve evaluation methodology. Definitions of new terms on the website will be made available in English, French and German. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Additional set of terms included in the glossary

Updated glossary on website (in English, French and German)

Page 9: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

9

1.2.2 Develop Frequently Asked Questions

This activity will consist of a single ongoing activity:

1.2.2.1 Compile Frequently Asked Questions

Additional FAQs will be compiled on the basis of the requests for information received and dealt with by the Evaluation Helpdesk. Replies to requests for information will be prepared by the Evaluation Helpdesk and agreed with the European Commission before being added to a list of potential FAQs. An extended list of FAQs to be disseminated and published on the website will be agreed within the framework of the regular meetings between the Commission and the Evaluation Helpdesk. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Additional set of Frequently Asked Questions and answers

Updated FAQ section on the website in English, French and German

1.3 To improve and update guidance related to the CMEF in the current and future programming period

The results of the thematic working group(s) – cf. objective 1.1. above – will complement the Handbook on the CMEF and further detail the guidance given by the Handbook.

1.3.1 Update guidance related to the CMEF (current period)

The aim of the activity in 2013 will be to complement the respective common CMEF guidance on the basis of:

the outcomes of the work of Thematic Working Groups

further analysed experiences from the mid-term evaluation

approved answers to technical questions. An ongoing exchange with the Member States concerning the work related to this activity will be ensured through the meetings of the Evaluation Expert Committee, workshops in Member States, etc. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Complementary papers

1.3.2 Support the CMEF review (future period)

In view of the next programming period, the CMEF needs to be reviewed in order to respond more closely to the objectives and priorities of the post-2013 policy. Within DG Agriculture therefore a 'Working Group on the review of the CMEF' has been set up with the aim to develop proposals for improvement of the monitoring and evaluation system for the future programming period. In autumn 2012 further technical working groups have been installed to prepare the future framework for monitoring and evaluation post-2013.

Page 10: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

10

The working group and selected sub-working groups are supported by the Helpdesk of the European Evaluation Network for Rural Development by:

Participation of Helpdesk representative(s) in working group meetings

Providing thematic input for the working group based on relevant Helpdesk outputs

Providing relevant information sources (databases, SWOT, working papers, etc.)

Ensuring interaction with other Helpdesk activities (Thematic Working Groups, etc.)

Preparation of contributions for a Monitoring and Evaluation Handbook for 2014-2020

Work from the Thematic Working Group (activity 1.1.1) may feed into this activity. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Input papers, presentations, handbook etc.

1.4 To improve the response to the needs in terms of RD evaluation

The assessment of needs organised in 2010, 2011 and 2012 aimed to help priority setting of the Evaluation Expert Network as well as to improve the outlook to the post-2013 phase. The results of this exercise will be used as a valid basis for the planning of the Network’s activities beyond 2013.

1.4.1 Update the assessments of needs carried out in 2012

The paper on the assessment of needs performed in 2011 highlights needs and issues to be considered when preparing the successive Annual Work Programmes. Due to the evolving nature of the topic, the needs of the evaluation community are examined yearly by drawing on all relevant information sources (minutes from Member State missions, conferences, technical information requests, etc.) and Focus Groups organized in the Member States. In 2013 the Focus Groups in the Member States will be prepared in close collaboration with the EC Desk Officers (e.g. selection of topic, definition of Focus Group method, participation). In 2013 an updated paper will be made available to the Member States and the evaluation community at large. The needs will later be presented to and discussed with the Evaluation Expert Committee and constitute an important input for the formulation of the Annual Work Programme(s) of the Evaluation Expert Network. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Updated paper on the assessment of needs

1.4.2 Prepare the 2014 Annual Work Programme

The aim of this activity is to ensure that the needs of the evaluation stakeholders are addressed in an adequate and consistent manner and encapsulated in a well planned and focused activity schedule. The following actions are envisaged, which will be run simultaneously:

Page 11: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

11

1.4.2.1 Draft the 2014 Annual Work Programme

The Annual Work Programme will be put together by the Evaluation Helpdesk on the basis of a needs assessment and in close collaboration with the European Commission. This action will comprise three key milestones:

1st draft AWP submitted to the European Commission in July 2013

2nd draft AWP updated and submitted to the European Commission in October 2013

AWP for 2014 finalised and submitted to the European Commission in December 2013. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

1st

draft 2nd draft Final

1.4.2.2 Finalize the 2014 Annual Work Programme

The draft work programme will be presented at the meetings of the Evaluation Expert Committee on Rural Development Programmes planned in 2013. An advanced draft version of the Annual Work Programme will have been forwarded to the representatives of the Member States beforehand, thus allowing a thorough discussion during the meeting on the contents proposed. Input of the Evaluation Expert Committee will be used to finalize the Annual Work Programme. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Annual Work Programme for 2014

Page 12: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

12

2. To increase capacity in implementing RD evaluation processes

(technical support function)

2.1 To improve the assessment capacity in the field of RD evaluation

Activities will be planned to help the European Commission and Member States increase their assessment capacity in relation to evaluation results.

2.1.1 Support the assessment of the Annual Progress Reports for Rural Development Programmes submitted in 2013

In the previous years the evaluation sections in the Annual Progress Reports for 2008, 2009, 2010 and 2011 were assessed by the Evaluation Helpdesk with the support of its geographic experts. On the basis of an analysis grid the content was analysed and recommendations were prepared.

2.1.1.1 Update the assessment tool and guidelines

The synthesis grid and guidelines created for the assessment of Annual Progress Reports for 2011 will be updated and adapted for the Annual Progress Reports for 2012.

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

2.1.1.2 Assess the functioning of the ongoing evaluation systems in Member States

The evaluation sections of a well-balanced set of Annual Progress Reports for 2012 will be assessed, in particular the functioning of the structures for ongoing evaluation. Findings and recommendations will be synthesised in a working paper. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Updated assessment tool

Synthesis of Annual Progress Reports for 2012

2.1.2 Training of EC Desk Officers on Monitoring and Evaluation

It is suggested to prepare a training of EC Desk Officers on the new monitoring and evaluation framework. (Further details to be specified in concept note) Timeline (to be defined)

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Page 13: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

13

2.2 To support the implementation of the ongoing evaluation

While between 2007 and 2013 the organisation of evaluation activities on an ongoing basis aims to ensure better preparation for formal mid-term and ex post evaluation, this approach is fairly new to the majority of Member States who will have to adequately adopt and implement it: specific support is therefore required to ensure efficient delivery of these processes.

2.2.1 Support the capacity-building on evaluation in the Member States through targeted actions

The needs assessments in previous years revealed that capacity-building in the field of evaluation of Rural Development Programmes is a key challenge for most Member States. The preparation of supportive material and the implementation of targeted actions by the Helpdesk will create synergies across the EU and effectively help improve the evaluation capacity. .

2.2.1.1 Prepare a targeted support for capacity-building activities for stakeholders involved in the ongoing evaluation of Rural Development Programmes in the Member States

The Evaluation Helpdesk will proactively support activities of capacity building in the Rural Development programmes through ongoing involvement of evaluation stakeholders into the content work of the Helpdesk. Moreover, the information material (working papers, Power Point Presentations, etc.) prepared by the Evaluation Helpdesk will be made available for use in the Member States. The content of the support material will be agreed with the European Commission. In addition, the support material will also be made available on the Internet site. The activity could potentially be carried out in form of capacity building workshops on selected topics.

2.2.1.2 Run targeted activities for supporting capacity-building of stakeholders involved in the ongoing evaluation of Rural Development Programmes in Member States

Targeted activities to support capacity-building in the Member States will be included into the regular missions to the Member States run by the Evaluation Helpdesk – cf. Action 3.2.4.1. These activities can be designed e.g. as interactive working sessions of good practice sharing; presentations on specifically challenging issues; questions and answer sessions with evaluation stakeholders, etc.

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Material to support capacity building in the Member States (e.g. presentations, etc.)

2.2.2 Collect best practices in implementing the CMEF methodology

This ongoing activity aims to provide the European Commission, the Member States and the evaluation community at large with examples of good practice worth disseminating and sharing at EU level. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Page 14: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

14

2.2.2.1 Good Practice Workshops

Also in 2013 the collection of good practices will be primarily realized through a series of good practice workshops to be organized in collaboration with Member States authorities. The topics of the good practices will suggested in a bottom-up way by evaluation stakeholders in the Member States and may include, for instance:

The assessment of the future RD impact indicators (in regionalized Member States)

The assessment of climate change measures in RDPs

Econometric modelling in the context of RDP evaluation

Use of context indicators in preparation of RDPs, and progress with ex ante evaluation Further examples of good practice are selected according to defined criteria and according to the most urgent needs of the evaluation community.

2.2.2.2 Update section on “Good practices in terms of evaluation methodology and processes” on website

Based on the collected examples of good practice the set of full descriptions should be further prepared and made accessible on the website of the European Evaluation Network for Rural Development on a continuous basis. The dissemination of these examples of good practice will also be part of the operational objective 3.1. ‘To ensure effective dissemination of information related to evaluation methodology and activities’. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Set of good practice examples on evaluation methods and processes (Good Practice Newsletter)

Updated Good Practice section on the website

Page 15: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

15

3. To share and disseminate knowledge in the evaluation of Rural

Development Programmes (dissemination & networking

function)

3.1 To ensure effective dissemination of information regarding evaluation methodology and processes

The year 2008 was dedicated to the setting-up of the communication and dissemination tools. In 2009, 2010, 2011 and 2012 these tools were further tested with a view to further strengthening the presence of the Evaluation Expert Network and the Evaluation Helpdesk amongst the Evaluation Community.

3.1.1 Establish the presence of the Evaluation Expert Network and its Helpdesk

In previous years contacts were established with Member States as part of specific missions aiming to promote the role of the Evaluation Expert Network and the services provided by the Evaluation Helpdesk. Representatives of the Evaluation Helpdesk also attended a number of events. The objective for the year 2013 will be to continue strengthening the presence of the Evaluation Expert Network and its Helpdesk in the activities organised at EU level in relation to evaluation of Rural Development Programmes.

3.1.1.1 Update presentation files

An initial set of generic presentation files – e.g. role and functions of the Evaluation Helpdesk, etc. – was created in 2008. These were updated and refined in 2009, 2010, 2011 and 2012. Also in 2013 they will continue to be regularly updated, while additional presentation files will be created to reflect the achievements of the Evaluation Expert Network in specific fields. The files will be used at events or meetings in Member States – which may require some degree of adaptation according to the audience – and will also be partly available on the Network’s internet site. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.1.1.2 Attend events and meetings in Member States

As was the case in previous years, also in 2013 the Evaluation Helpdesk contributes to events and meetings in the Member States. The events to be attended by the Evaluation Helpdesk will be agreed on a case-by-case basis with the European Commission. An indicative plan for the events to be attended for the 1

st half of 2013 is to be submitted to the EC by the end of December 2012, and for the

2nd

half of 2013 by the end of June 2013. The Helpdesk may be represented by Members of its permanent or non-permanent team. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Presentation files

Reports on events attended

Conference papers

Page 16: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

16

3.1.2 Manage a library of relevant studies and other relevant information

The information collection for evaluation literature and procedures were established and tested in 2008. From 2009 to 2012 the information base was expanded. This activity will continue in 2013 on an ongoing basis. Two interrelated actions are envisaged as part of this activity:

3.1.2.1 Collect and collate information on publications and events

Key documents on evaluation – either in hard copies or in electronic format - will be collected and abstracts be drawn up in English. Also official EC literature as well as EC guidance / working documents will be made available as part of this library. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.1.2.2 Disseminate information on publications and events

Abstracts on the information collected and any key relevant documents will be disseminated via the Evaluation Expert Network’s website and Newsletters. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Library of documents related to evaluation of rural development

Abstracts on publications and events

3.1.3 Run the Evaluation Expert Network’s website

A provisional website, containing basic information on the Evaluation Expert Network, was set up in 2008 on Europa, the European Union website portal. Since 2010 the final website has been online, though particular sections are continuously expanded. Publishing content on the Evaluation Expert Network’s website is an ongoing activity. It will entail updating or creating new pages for the internet site as well as uploading relevant files – e.g. abstracts on publications, working papers, FAQs, etc. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Ready-to-publish material, uploaded on the Evaluation Expert Network’s website in EN, DE, FR

3.1.4 Produce newsletters

Depending on the yearly outputs of the Evaluation Expert Network up to 2 newsletters will be produced in 2013. These newsletters will aim at promoting the results achieved by the Evaluation Expert Network. All newsletters will be disseminated in the usual electronic format in three languages – EN, DE, FR.

Page 17: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

17

3.1.4.1 Produce and disseminate Newsletter

The specific issues of the Evaluation Expert Network’s Newsletter will be disseminated depending on the outputs. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Issues of the Evaluation Expert Network’s electronic newsletter in EN, DE, FR

3.1.5 Manage a mailing and distribution list database

A mailing and distribution list database was set up in 2008, which aimed to disseminate the work undertaken by the Evaluation Expert Network, including newsletters, to different audiences. It was further updated in the following years.

3.1.5.1 Update and expand the mailing and distribution list database

The mailing and distribution list database will be updated and expanded on a regular basis. The updated lists will be submitted to the European Commission in June and December for approval. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

to EC to EC

Expected Outputs:

Mailing and distribution list

3.2 To improve collaborations and foster synergies between stakeholders

Sharing good practice on evaluation of rural development will be achieved by establishing and maintaining collaborations with the relevant stakeholders and making use of potential synergies emanating from the links created.

3.2.1 Run the Evaluation Helpdesk

The Helpdesk was established in 2008 with the aim to act as a hub for information related to evaluation of Rural Development Programmes in the EU. The objective for the year 2013 will be to continue strengthening the role and functions of the Evaluation Helpdesk as a robust reference tool for all those with an interest in the field of evaluation of rural development. The following ongoing actions are planned:

3.2.1.1 Coordinate requests for information

The Evaluation Helpdesk regularly receives requests for information from Member States in relation to the evaluation of Rural Development Programmes. The Evaluation Helpdesk prepares replies to these

Page 18: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

18

requests and sends them to the European Commission for approval – as part of Technical Reports to be forwarded every two weeks on average - before being forwarded to Member States. The specific focus for 2013 will be laid on: - the provision of relevant answers to Member States’ questions - the publication of answers as part of FAQs Steps for dealing with requests for information are described in the Evaluation Helpdesk’s Manual of Procedures. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.1.2 Host visitors to the Evaluation Helpdesk

The Evaluation Helpdesk will host visitors on the basis of appointments. Short reports on those visits will be drawn up and forwarded to the European Commission, along with the regular Technical Reports – cf. 3.2.1.1. above. Steps for dealing with visits are described in the Evaluation Helpdesk’s Manual of Procedures – cf. 3.2.1.4 below. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.1.3 Coordinate the permanent and non permanent Evaluation Helpdesk teams

Managing the permanent and non-permanent Team of the Evaluation Helpdesk will be the responsibility of the Team Leader. The coordination and flow of information with the non-permanent teams, i.e. the Core Team of Experts and the Geographic Experts, will be ensured through regular circulars and other dissemination forms (e.g. direct contacts, direct e-mails, etc.). If appropriate, meetings will be organised with the team of experts. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.1.4 Maintain the Evaluation Helpdesk’s Manual of Procedures

The Manual of Procedures aims to clarify roles and the structure for various types of collaborations and tasks. It will be updated when needed, amended and possibly expanded according to needs in terms of activity coordination. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Page 19: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

19

3.2.1.5 Maintain the Evaluation Helpdesk’s filing system

The filing system of the Evaluation Helpdesk, including its coding system, was established in 2008. It will be updated in 2013, if needed. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Technical reports including requests for information and proposed replies

Approved replies to requests for information

Reports on visits to the Evaluation Helpdesk

Minutes of meetings of the Core Team of Experts

Updated Manual of Procedures

Updated filing system

Circulars for Geographic Experts

3.2.2 Follow up on the activities of the Evaluation Helpdesk

This activity will aim to secure an efficient flow of information between the consultants in charge of managing the Evaluation Helpdesk and the European Commission and in particular to ensure that planned activities meet expectations. The following actions will be run:

3.2.2.1 Hold regular meetings between the consultants and the European Commission

Meetings will be held regularly to discuss the implementation of the various activities planned in the work programme as well as the technical questions received. If appropriate, minutes will be drafted. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.2.2 Participate in meetings with the Steering Group

Periodic meetings with the Steering Group will aim to keep the members of the Steering Group informed about the planned activities and the work achieved – cf. quarterly and annual reports – as well as the problems encountered and the solutions found. These meetings are also a forum for discussing the drafts of the Annual Work Programmes. Timeline: dates of Steering Group meetings to be agreed

Page 20: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

20

3.2.2.3 Report about activities and achievements

Quarterly and annual reports will be submitted to the European Commission, highlighting the activities carried out and the results achieved, as well as the problems encountered and the solutions found. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Q4 Q1 Q2 AT

annual + Q2 MT Q3

Expected Outputs:

Meeting minutes

Four quarterly reports on main tasks

Quarterly reports on additional tasks as completed

One annual report

3.2.3 Collaborate with the Expert Committee on Evaluation of Rural Development Programmes

The Expert Committee on Evaluation of Rural Development Programmes – Evaluation Expert Committee - was formally set up in 2008. Meetings of this Committee are held in Brussels.

3.2.3.1 Prepare and attend meetings of the Evaluation Expert Committee

The meetings of the Evaluation Expert Committee or similar groups will be held at least 2 times a year. The dates of the meetings will be agreed well in advance. The Evaluation Helpdesk will assist the European Commission in preparing these meetings. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.3.2 Coordinate the follow-up of Evaluation Expert Committee meetings

The Evaluation Helpdesk will be in charge of drawing up the minutes of meetings of the Evaluation Expert Committee or similar groups, while the Commission will be responsible for disseminating them. The Evaluation Helpdesk will ensure that the outcome of the discussion feeds into the existing and future Annual Work Programmes, as well as the activities of the various thematic working groups. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Presentation materials

Evaluation Expert Committee meeting minutes

Page 21: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

21

3.2.4 Liaise with Member States

3.2.4.1 Organise missions to Member States

Specific missions to Member States will be organised. The main target group are the evaluation stakeholders (Managing Authorities, evaluators, national networks, etc.) in the Member States. Regarding specific missions to the Member States – including direct contacts with the Managing Authorities, those countries not visited from 2008 to 2012 will be given a priority. The thematic orientation of presentations will take into account the current themes and issues of the Helpdesk’s work. The other key objective of these missions will be to identify and disseminate best practice and contribute to building capacity for the evaluation of Rural Development Programmes – cf. Activity 2.2.1. A detailed mission plan for the 1

st half of 2013 is to be submitted to the EC by the end of December

2012, and for the 2nd

half of 2013 by the end of June 2013. This action will be run jointly with Action 3.1.1.2. ‘Attend events and meetings in Member States’. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.4.2 Manage direct contacts and exchange of information with evaluation stakeholders in the Member States

The Evaluation Helpdesk will coordinate requests for information from evaluation stakeholders (bodies concerned with the evaluation of RDPs, evaluators, etc.) in line with Action 3.2.1.1. ‘Coordinate requests for information’. Any other information received or message exchanged between the Evaluation Helpdesk and evaluation stakeholders will be registered and the European Commission informed about the contents of these exchanges. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.4.3 Provide support to Member States for evaluation-related conferences and workshops

The Evaluation Helpdesk will provide methodological support to Member State authorities which organise in cooperation with DG Agriculture evaluation-related conferences and seminars. This support may consist in methodological input and assistance for information and dissemination (e.g. provision of contact details, announcement on website, dissemination of outputs through network, etc.). Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

3.2.5 Collaborate with the European Network for Rural Development

This activity aims at maintaining links with the European Network for Rural Development, thus ensuring consistency and creating synergies between the activities of the two Networks. The European Network for Rural Development (EN RD) was set up in 2008. Both Networks are sharing the same website, while the consultants in charge of the wider European Network for Rural Development

Page 22: ANNUAL WORK PROGRAMME 2013 - Europaenrd.ec.europa.eu/enrd-static/fms/pdf/B6B48B03-0BAD-ED37...JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Potential Outputs e.g. thematic input

22

will be responsible for the overall management of it. Regular exchanges on activities and schedules between the two networks are also foreseen. While the entities mentioned below are the most relevant ones this list is however not exhaustive: information exchange and coordination with other bodies, such as the Leader Network, the Leader sub-committee, the evaluation network of DG REGIO, etc. is considered highly useful and will be intensified for specific themes and purposes. The following actions are envisaged:

3.2.5.1 Liaise with the Coordination Committee of the ENRD

The Coordination Committee is part of the European Network for Rural Development. Representatives of the Evaluation Helpdesk will attend meetings of the Committee and relevant sub-committees.

3.2.5.2 Liaise with the Contact Point of the ENRD

It is agreed that joint meetings between the consultants in charge of both Networks will be held. The European Commission will be invited to those meetings and decide about its participation on the basis of the proposed agenda. One of the main tasks shared between the Helpdesk and the Contact Point relates to the website of the Evaluation Expert Network, which is an integral part of the website of the wider European Network for Rural Development. Other tasks may include sharing contact databases, information in relation to indicators, etc. Short minutes will be drawn up for these meetings and forwarded to the European Commission. Timeline:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Expected Outputs:

Minutes of the meetings between consultants in charge of both Networks