presentation 3: “practical experiences gained from carrying out case studies in relation to...

Presentation 3:

“Practical experiences gained from carrying out case studies in relation to different policy themes and/or overall context settings”

Dr. Thomas Stumm Managing Director of EureConsult S.A.

(Echternach, G.D. of Luxembourg)

Evaluating Structural Funds interventions in one Member State:Telling the story from a country-specific perspective

My experience confirmed to me that in-depth case studies have great potentials for unveiling a number of issues which are normally not perceivable at a first glance by any horizontal evaluation.

The precise but often also diverse nature of impacts resulting from a policy intervention in a given region (e.g. “What has been achieved in terms of direct/indirect effects and intended/unintended effects?).

The way in which a policy intervention is implemented and working in practice in the particular region being examined (e.g. “How has a policy intervention achieved the impact?).

The influence of contextual factors or overall developments in order to locate the effect of a particular intervention or the way how a policy intervention has been realised within these (“Why” is the “what” and “how” of a policy intervention like it is?).


If multi-site case studies are realised for a specific country, an appropriate presentation format needs to be developed when elaborating a “general framework” for carrying out the case study work ( core teams of the main contractor).

It has proved very useful presenting the overall outcome of multi-site case studies realised for a single Member State in a wider format (e.g. in form of a “country report”).

General context analysis preceding the actual case study part.

Subsequent in-depth analysis of the cases under review.

Final synthesis relating in-depth analyses to initial working hypotheses / the general patterns of the country context.


A sound initial briefing of the field teams carrying out case study work at a later stage should be realised, e.g. in form of a one-day “kick-off workshop” bringing together all parties involved in an evaluation.

The field teams / local experts need to get a clear understanding about the wider purpose and strategic objectives of the overall evaluation ( task of the client).

The field teams / local experts must be made fully aware of the central purpose of the case study within the wider evaluation ( task of the core team).

The field teams / local experts must become fully familiar with all aspects relating to the practical realisation of case studies ( task of the core team).

Due to my positive experiences made with such briefings, I recommend this to be a mandatory first step in any evaluation involving extensive case study work.


Field evaluators should realise a case study-based in-depth analysis with a certain degree of “distance” and “flexibility”.

As the preparation / elaboration of a good case study needs time, try to take some distance with respect to the inevitably occurring time-pressure & keep up professional standards.

Be flexible in the actual preparation & realisation of stakeholder interviews (e.g. drop questions or add new ones; carry out single-person interviews and structured group discussions)

When drawing up the case study report, try taking some distance with respect to an initially defined reporting template. Intuitively “co-determine” the importance given to a pre-defined item on ground of your actual knowledge & with respect to the actual relevance of the topic within the wider story to be told about a case.

Show intuitive flexibility while elaborating an informative and analytical but still appealing narrative which tells the story about a specific case (e.g. by focussing the story on a number of “critical incidents”).

Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective

The use of the case study method is particularly recommended for evaluating European Territorial Co-operation (ETC).

This is due to the fact that the complexity of real-life interactions within co-operation processes is considerably higher if compared to those taking place in interventions carried out in a single-country perspective.

Case studies can - in principle - be used at three different levels: (1) When evaluating Community-level funding schemes such as the “old” INTERREG Community Initiative or the new ETC-Objective. (2) When evaluating an individual cross-border, transnational or inter-regional co-operation programme.

(3) When evaluating a specific co-operation project.

Evaluating territorial co-operation programmes & projects:

Telling the story from a multi-country perspective

According to my practical experience, however, case studies on ETC are also a very demanding and particularly challenging task for each evaluator involved.

The evaluator has to be familiar with more than only one country-specific context (e.g. basic political-administrative settings, different legal & regulatory systems, different cultures & ways of “doing” and “thinking”).

The evaluator needs to have well-developed and appropriate language capacities (esp. in the context of cross-border co-operation).

The evaluator needs to have a well developed understanding about the previous history of co-operation (i.e. long-term development & core activities carried out in the past).

The the activities for preparing and realising an ETC-case study are much more complex as a greater number of key stakeholders originating from different countries and from different levels of co-operation need to be identified / contacted for telling a story which is comprehensive and well-balanced.



(1) An evaluation of Community-level funding schemes

Compared to the previous INTERREG II ex-post evaluation, the current ex-post evaluation of the INTERREG III Community Initiative (2000-2006) has adopted a very different approach: Case studies play an important role in the entire evaluation process.

Based upon an extensive and aggregated analysis realised previously across all programmes, a typology of INTERREG III programmes was elaborated on ground of which a case-study sample was selected which well-reflected the diversity and types of existing programmes.

These 16 programme case studies, with each of them also involving the realisation of 5 shorter project-case studies, have shed more light on specific aspects which have already been analysed at a more aggregated level across all programmes.



(2) An evaluation of individual ETC-programmes

Programme-level evaluations (e.g. the ongoing evaluations for the 2007-2013 period) should make more use of the case study method especially when it comes to assessing their project-level implementation.

This can provide the strategic “programme owners” with a much better understanding about the complex reality of the project-based implementation of their programme and about the real scope and nature of the effects generated by the approved projects.

The outcome of such project-level case studies is also of a long-term relevance, as it helps building up a permanent & precious “historical memory” about the client dimension of a programme.

Another interesting perspective is the use of case studies in the context of future ex-ante evaluations for ETC-programmes after 2013.

Case studies can relate to various aspects in the process (i.e. problem assessment & strategy building and design of the programme-wide decision-making/management and implementations structures).

Case studies can adopt both a programme-internal focus (e.g. Learning from the past for the future) or a programme-external focus programme (e.g. Learning from others how they have dealt with a certain issue?).



(3) An evaluation of individual ETC-projects

I am strongly convinced that the use of the case study method can also make an important contribution to raise the overall quality & depth of an individual project evaluation (if those are realised).

Project-focused evaluations are not yet realised very frequently: Reasons might be a not yet very broadly developed awareness about the associated benefits & the still rather weakly developed methods or techniques for evaluating ETC-projects.

An external evaluation should be carried out if a project has (1) a certain budget size (e.g. > € 2 million) and/or (2) if it has a rather complex implementation structure and realises particularly experimental actions bearing a certain risk of failure.

The most appropriate format to be adopted by a project evaluation should either be that of an ongoing evaluation or that of a “classical” mid-term evaluation.

The focus of an evaluation must serve the project & its (financial) scale must be reasonable.

Individual project evaluations can elaborate “mini-case studies” which analyse all or certain operational elements of a project (e.g. an entire work package, a work process) or focus on issues of a particular common interest (e.g. depth & quality of project-level co-operation; emerging direct effects & unexpected direct/indirect effects etc.).

Thank you

for your attention!

presentation 3: “practical experiences gained from carrying out case studies in relation to...

Documents