(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 1

EJ-

2007

EJ-

2007

Key factors of a quantitative indicator system

Emmanuel JAECK

Prefecture of the Midi-Pyrénées region / France

Warsaw 2009

EJ\Warsaw-2009.ppt

Sixth European Conference on Evaluation of Cohesion Policy

“New Methods for Cohesion Policy Evaluation:

Promoting Accountability and Learning”

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 2

EJ-

2007

EJ-

2007

Emmanuel JAECK

Head of the Evaluation Unit

Prefecture of the Midi-Pyrénées region

(Managing Authority)

FRANCE

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 3

EJ-

2007

EJ-

2007

Midi-Pyrénées?

Source : http://www.midipyreneeseurope.be/05_territoire/carte_00.html

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 4

EJ-

2007

EJ-

2007

A consciously instrumental approach

=>

Looking for causal links

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 5

EJ-

2007

EJ-

2007

1 – Multi-Level System1 – Multi-Level System

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 6

EJ-

2007

EJ-

2007

Single Programming Document 2000-2006 Midi-Pyrénées

• 7 Priorities – 23 measures

•75 actions

• On 31 December 2006On 31 December 2006

• 15 760 operations entered15 760 operations entered• 11 800 operations planned11 800 operations planned

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 7

EJ-

2007

EJ-

2007

But in reality we have:

•128 individual actions

likeExample of a heterogeneous action (territorial approach)

Axe 2 ………..……. = Promouvoir les territoires de projetsMeasure 2.7…..... = Accompagner le développement

concerté et durableAction 2.7.2…….. = Soutien aux projets intégrés

=> Support to– Actions culturelles– Patrimoine rural– Insertion pas l’économie– Fiche industrielle et urbaine

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 8

EJ-

2007

EJ-

2007

For 11,800 planned operations

• 202 output and result indicators• 18 indicators for performance reserve• 6 impact indicators

And on 30th October 2009

3.8 % of indicators not correctly registered (incomplete or incoherent)

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 9

EJ-

2007

EJ-

2007

Key factors of:

1. 2. Collection System

3. Management of indicator data

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 10

EJ-

2007

EJ-

2007

2 – Collection system2 – Collection system

A network of officers trained

An information systemA collection tool

A procedure + A practical guide

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 11

EJ-

2007

EJ-

2007

3 – Management of Indicator Data3 – Management of Indicator Data

Contact with the

promoter

Control by examining department

Quality control

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 12

EJ-

2007

EJ-

2007

Sensitive points:

• System Design

• Implementation

• Use of Indicators

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 13

EJ-

2007

EJ-

2007

Sensitive points : System Design

• The system cannot reflect everything. Striking the balance between monitoring and evaluation is a must.

• Un système qui répond à tous les enjeux de pilotage et aussi de gestion.

• The objectives of the actions are sometimes vague and choosing the indicators often follows later

• Higher level indicators to be precisely defined very early

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 14

EJ-

2007

EJ-

2007

Sensitive points: Implementation

• Bear in mind that the system cannot quickly answer to questions on effects

• Train and support the persons responsible for the management of indicator data throughout the period

• Keep track of changes of reference documents (PO et Domo)

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 15

EJ-

2007

EJ-

2007

Sensitive points: Use of Indicators

• Indicators should not become final objectives

• Integrate the ‘natural’ dynamics of each action in target setting

• Take into account the internal factors influencing the progress of indicator values

• Communiquer avec rigueur et prudence à l’aide d’indicateurs notamment si les précautions méthodologique ne sont pas entendables

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 16

EJ-

2007

EJ-

2007

Pragmatic System under Control

• Legitimate• Real Obligation• Control from Supervising

Authority• Potential Sanctions

(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 17

EJ-

2007

EJ-

2007

Thank you

for

your attention