(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 1
EJ-
2007
EJ-
2007
Key factors of a quantitative indicator system
Emmanuel JAECK
Prefecture of the Midi-Pyrénées region / France
Warsaw 2009
EJ\Warsaw-2009.ppt
Sixth European Conference on Evaluation of Cohesion Policy
“New Methods for Cohesion Policy Evaluation:
Promoting Accountability and Learning”
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 2
EJ-
2007
EJ-
2007
Emmanuel JAECK
Head of the Evaluation Unit
Prefecture of the Midi-Pyrénées region
(Managing Authority)
FRANCE
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 3
EJ-
2007
EJ-
2007
Midi-Pyrénées?
Source : http://www.midipyreneeseurope.be/05_territoire/carte_00.html
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 4
EJ-
2007
EJ-
2007
A consciously instrumental approach
=>
Looking for causal links
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 5
EJ-
2007
EJ-
2007
1 – Multi-Level System1 – Multi-Level System
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 6
EJ-
2007
EJ-
2007
Single Programming Document 2000-2006 Midi-Pyrénées
• 7 Priorities – 23 measures
•75 actions
• On 31 December 2006On 31 December 2006
• 15 760 operations entered15 760 operations entered• 11 800 operations planned11 800 operations planned
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 7
EJ-
2007
EJ-
2007
But in reality we have:
•128 individual actions
likeExample of a heterogeneous action (territorial approach)
Axe 2 ………..……. = Promouvoir les territoires de projetsMeasure 2.7…..... = Accompagner le développement
concerté et durableAction 2.7.2…….. = Soutien aux projets intégrés
=> Support to– Actions culturelles– Patrimoine rural– Insertion pas l’économie– Fiche industrielle et urbaine
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 8
EJ-
2007
EJ-
2007
For 11,800 planned operations
• 202 output and result indicators• 18 indicators for performance reserve• 6 impact indicators
And on 30th October 2009
3.8 % of indicators not correctly registered (incomplete or incoherent)
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 9
EJ-
2007
EJ-
2007
Key factors of:
1. 2. Collection System
3. Management of indicator data
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 10
EJ-
2007
EJ-
2007
2 – Collection system2 – Collection system
A network of officers trained
An information systemA collection tool
A procedure + A practical guide
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 11
EJ-
2007
EJ-
2007
3 – Management of Indicator Data3 – Management of Indicator Data
Contact with the
promoter
Control by examining department
Quality control
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 12
EJ-
2007
EJ-
2007
Sensitive points:
• System Design
• Implementation
• Use of Indicators
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 13
EJ-
2007
EJ-
2007
Sensitive points : System Design
• The system cannot reflect everything. Striking the balance between monitoring and evaluation is a must.
• Un système qui répond à tous les enjeux de pilotage et aussi de gestion.
• The objectives of the actions are sometimes vague and choosing the indicators often follows later
• Higher level indicators to be precisely defined very early
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 14
EJ-
2007
EJ-
2007
Sensitive points: Implementation
• Bear in mind that the system cannot quickly answer to questions on effects
• Train and support the persons responsible for the management of indicator data throughout the period
• Keep track of changes of reference documents (PO et Domo)
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 15
EJ-
2007
EJ-
2007
Sensitive points: Use of Indicators
• Indicators should not become final objectives
• Integrate the ‘natural’ dynamics of each action in target setting
• Take into account the internal factors influencing the progress of indicator values
• Communiquer avec rigueur et prudence à l’aide d’indicateurs notamment si les précautions méthodologique ne sont pas entendables
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 16
EJ-
2007
EJ-
2007
Pragmatic System under Control
• Legitimate• Real Obligation• Control from Supervising
Authority• Potential Sanctions
(C) Emmanuel JAECK Warsaw 2009Key factors of a quantitative indicator system 17
EJ-
2007
EJ-
2007
Thank you
for
your attention