Evaluation Arrangements for 2007-2013:

A Decentralized Approach

Laura TagleEvaluation Unit – Department for Development Policies – Ministry for

Development

Rossella TarantinoEvaluation Unit – Basilicata Region

Bruxelles 26-27 June 2008

Presentation Outline

• Presenters’ background: collective evaluation capacity building

• Overall approach to evaluation: decentralization, responsibility and Evaluation plans

• Main issues:– Evaluation questions– Involvement of social and economic partners

• A regional perspective on a “central” endeavour

National Evaluation System

• Work group: collective work for evaluation capacity development

• Individuals from:– Evaluation Unit of Ministry for Development

(responsible for work group coordination)– ISFOL (Agency responsible for evaluation of ESF)– INEA (Agency responsible for evaluation of Rural

Development)– Regional Evaluation Units

Decentralization and responsibility

• Decentralization: – each Regional government responsible for evaluating its

interventions– Coordination ministry responsible for evaluating from

“central” point of view– Regional ownership: each Region asks the evaluations it

needs and times it for when it needs the results• No common evaluation questions• No prescribed timing

• Responsibility:– No sanction– Evaluation plans and evaluations are public– National Evaluation System guarantees guidance,

training, support, systematic observation and reputational mechanisms

Evaluate the effects of regional policy

• Until the end of 2009, mainly ex post evaluations of past interventions

• Evaluate EU- and nationally funded interventions• Evaluate effects of joint interventions on areas/groups• Evaluation questions on specific issues, groups, areas,

NOT generic “program”

Steering Groups

• Ensure involvement of socio-economic partners: Steering Group at Plan level

• Ensuring qualityand independence of each evaluation: Steering Group (+ other methods) at each evaluation level. No MA.

Risks:• creating a new profession: “Steering Group

member”• Duplicating the problems of Monitoring

Committees

Evaluation plans

• One plan per Region/central ministry regarding all programs (past and present) (unitary approach)

• Need to build a new internal organization• 30 programs• Only two Regions are still developing the plan (out

of 21)

Evaluations planned for 2008 e 2009

• Human Resources: 5• Education: 1• Urban policies: 1• Transport: 2• Implementation analysis:

13• Program evaluation: 18

• Research & innovation : 6• Enterprises: 3• Environment: 3• Cultural heritage: 1• Gender: 3• Area-based: 9• ICT: 2

Satisfaction at innovation spread, but

• Out of 67 evaluations planned for 2008-2009, 31 (+ all the regional development evaluations) still have a generic “Structural Fund program” focus:– Inertia in accepting innovations from regulations and

from NSF– Need for general overview– Incomplete or contradictory support from EC

• SNV: – strengthening our common work on ECD with EC– Increase capacity of national coordination agencies

• Further work needs to be done to involve social & economic partners

• Is a Plan Steering Group the right way to go about it?

The National Evaluation System and the evaluation plans

• “Guidelines for the organisation of the evaluation activities of the regional policies: the evaluation plans for 2007-13”;

• Working groups that went through the evaluation plans drafted by Regions and Ministers according to a peer group approach. These groups were created on volunteer basis and composed of both regional and national representatives.

This cross-regional and national scrutiny of the evaluation plans enabled a pooling of experiences and know-how. for instance, after going through the evaluation plan of the Liguria Region I changed/improved the evalution plan of my Region

The evaluation plan: the unitary approach

• Each Region has a unitary strategy implemented through OP’s

• One Evaluation Plan for Each Region that assess:• ERDF OP• ESF OP• EARDF OP• OP funded by National Fund for deprived areas• Interventions financed by regional resources

The evaluation plan: the unitary approach

to assess all the development policies implemented by the Region

–independently by the financial source – in order to have a global overview

of the effects that these policies produce.

Obviously in the respect of:

– Specific rules concerning each Fund;– The need of the European Commission and of the

single Managing Authorities to underline the specific community added value.

• To coordinate the evaluations and avoid overlapping and duplication;

• To build a global overview of the individual evaluations and thus have a big picture of the effects of the regional development strategy;

• To assess the results produced jointly by different actions on the same area or target group

• To have a shared knowlegde to improve the quality of programs and of their implementation

The unitary approach- why?

Specific evaluation activities focus on themes/areas/target groups and take into account results produced by different interventions

An ex-post evaluation in my region concerns the information society and the contribution to:

• The quality of the public services (i.e; health)

• The creation of community networks (i.e. Schools)

• The territorial and social inclusion

• The competitive capacity of SMEs

How to deal with organizational and financial issues?

• Pooling of funds (liguria)

• Steering Group with the participation of all Managing Authorities and departments involved in these policies (Basilicata)

• To address specific national resources (Emilia oRmagna)

The unitary approach-how

• In some regional plans there is a divide between the ERDF and ESF evaluations

• In the implementation phase, the unitary approach is even more difficult

Therefore, for us (Regions) it is important:• EC common approach• EC Common guidance• EC (DG Regio and DG Employment + DG Agri) joint

meetings and analysis with National Evaluation Systems

The unitary approach: not easy

EC support and guidance

• The joint analysis on the mid-term evaluation conducted on 2004 was very useful and gave important hints to improve the quality and awareness of evaluations: – focus on specific questions, stakeholders

involvement in the definition of questions and in the dissemination of evaluations results, emphasis on effects, etc

• Guidance

• Participation in our (SNV) ECD activities