evimp-2of ec funded projects under fp5 analysing the … · collecting, processing and analysing...

64
ANALYSING THE OUTCOMES OF EC FUNDED PROJECTS UNDER FP5 EVIMP-2

Upload: others

Post on 15-Oct-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

ANALYSING THE OUTCOMES OF EC FUNDED PROJECTS UNDER FP5 EVIMP-2

Page 2: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

ANALYSING THE OUTCOMES OF EC FUNDED PROJECTS UNDER FP5 EVIMP-2

Interested in European research?

Research*eu is our monthly magazine keeping you in touch with main developments (results, programmes, events, etc.).It is available in English, French, German and Spanish. A free sample copy or free subscription can be obtained from:

European Commission Directorate-General for ResearchCommunication UnitB-1049 BrusselsFax (32-2) 29-58220E-mail: [email protected]: http://ec.europa.eu/research/research-eu

EUROPEAN COMMISSION

Directorate-General for ResearchDirectorate G — Industrial TechnologiesUnit G1 — Horizontal aspects and coordination

E-mail: [email protected]

Contact: Sandra Peeters

European CommissionOffice CDMA 6/113B-1049 Brussels

Tel. (32-2) 29-55308Fax (32-2) 29-67023E-mail: [email protected]

Page 3: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

ANALYSING THE OUTCOMES OF EC FUNDED PROJECTS UNDER FP5 EVIMP-2

EUROPEAN COMMISSION

Directorate-General for Research2009 Directorate G - Industrial Technologies EUR 24054 EN

Page 4: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

EUROPE DIRECT is a service to help you find answers to your questions about the European Union

Freephone number (*):

00 800 6 7 8 9 10 11(*) Certain mobile telephone operators do not allow access to 00 800 numbers or these calls may be billed

LEGAL NOTICE

Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of the following information.

The views expressed in this publication are the sole responsibility of the author and do not necessarily reflect the views of the European Commission.

A great deal of additional information on the European Union is available on the Internet.It can be accessed through the Europa server (http://europa.eu).

Cataloguing data can be found at the end of this publication.

Luxembourg: Office for Official Publications of the European Communities, 2009

ISBN 978-92-79-13385-5doi 10.2777/50716

© European Communities, 2009Reproduction is authorised provided the source is acknowledged.

Pictures © Shutterstock, 2009

Printed in Belgium

Printed on white chlorine-free PaPer

Page 5: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Table of Contents1. INTRODUCTION 7

2. EVIMP-2 AND THE HISTORY OF EVALUATION 11

3. EVIMP-2 METHODOLOGY: THE NEXT FRONTIER IN EVALUATION 15 3.1 A more thorough approach in data gathering 19 3.2 Key reports for comprehensive results 19

4. PROFILE OF PROJECTS: A SOLID REFLECTION OF FP5 21 4.1 Clusters developed for the analysis of Evimp-2 results 23

5. OVERALL RESULTS 25 5.1 Benefits for industrial partners: More exploitation, fewer risks and costs 28 5.2 Benefits for research partners: Furthering research and optimising funding 28 5.3 Contribution to the environment 29 5.4 Promising exploitation opportunities 29 5.5 Achieving technical and other objectives 30 5.6 Management experience and consortium structure 30 5.7 Project success and failure factors 30 5.8 RTD vs. CRAFT analysis 31

6. A CLOSER LOOK AT INDIVIDUAL STUDIES 33 6.1 Matrix Knowledge Group 35 6.2 VDI/VDE Innovation + Technik 36 6.3 Technomar GmbH 37 6.4 Vekinis Report 37

7. CONCLUSIONS AND RECOMMENDATIONS 39 7.1 Selection recommendations 41 7.2 Evaluation recommendations 41

SUCCESS STORIES 43

Page 6: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations
Page 7: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

1. Introduction

Page 8: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

8_Evimp-2

Page 9: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_9

What is Evimp-2?

Evimp-2 (Evaluation and long-term impact assessment of industrial research) is the most recent evaluation initiative within the European Commission (EC) of completed projects conducted under various industrial and technology programmes within the European Union (EU) since the early 1990s.

The main goal of Evimp-2 is to identify if the Growth Programme under the Fifth European Community Framework Programme for Research (FP5) has produced the results and impact expected from it. Evimp-2 also examines if the Programme was worth the investment and what could be improved in later Research and Technological Development (RTD) funding programmes.

Evimp-2 provides an overall picture of the output of the projects in line with the objectives of the Growth Programme. This is achieved by examining the results from different angles, such as determining success and failure factors of projects, studying exploitation potential and providing recommendations that further future R&D programme design and implementation.

Why is Evimp-2 important?

The work of Evimp-2 is critical in assessing projects after completion for several reasons. In general terms, research and exploitation of results benefit the economy and encourage it to grow. Evimp-2, for example, can identify why there is a large gap between technically successful projects and those that exploit the results (less than 50 %). Assessing the projects’ risk of failure – such as not using results or not implementing new technology according to the Technical Implementation Plan (TIP) – is also part of Evimp’s mission. In addition, Evimp-2 has examined patterns, trends, causes and major obstacles that were present at the start-up of these projects. It has also identified those who want to publish their results and/or exploit them.

Within the European Commission itself, the EVIMP-2 study offers undoubtedly momentous insight as it can identify the benefits of the Commission’s activities and enable it to design future programmes as well as future proposal evaluations. In more detail, Evimp-2 has enabled the Commission to assess whether it is reaching its goals and outcomes with respect to funding and furthering SME activity, networking, innovation, exploitation and even environmental impact in many instances.

In its nature as an assessment mechanism, Evimp-2 can improve the effectiveness of RTD and recommend actions to promote dissemination and exploitation of RTD results, aiming at market uptake. It has already identified factors for success or failure of RTD projects resulting in suggestions on how these factors can be influenced. Finally, Evimp-2 improves the visibility of the success of RTD projects and programmes, underlining the importance of the work of the European Commission.

Overall, the EVIMP-2 study has successfully assessed over 700 projects that were funded by the European Commission. This includes RTD projects, Cooperative Research projects (CRAFT) and Thematic Network projects under the FP5 Growth Programme. The evaluation was undertaken by contracting several consulting companies who have examined different lots at different stages during FP5. A more detailed rundown of contractors and number of projects can be found in section three under Methodology.

What is the purpose of this report?

The purpose of this report is to explain Evimp-2 in more detail and reveal the most important findings of Evimp-2. The report combines the description of the process with the results and recommendations of different consulting firms solicited to carry out the Evimp-2 study and analyse them within one document. It begins by introducing Evimp and its history, outlining as well the benefits to the EC and to society, then presents the methodology used to arrive at the results and conclusions, followed by the profile of companies. In more detail, the report introduces the main consulting companies involved in the process, in addition to presenting the main findings, conclusions and recommendations that have arisen from Evimp-2.

1. Introduction

Page 10: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

10_Evimp-2

Page 11: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_11

2. EVIMP-2 and the history of evaluation

Page 12: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

12_Evimp-2

Page 13: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_13

There has always been a need for the European Commission, as with any reputable organisation or government body, to assess the impact of its efforts, initiatives and contributions. The EC has strived to perfect a system for evaluating the impact of projects it has funded throughout the years, an initiative which has finally been streamlined and taken shape in form of an exercise called Evimp-2.

Over the years and culminating with Evimp-2, the quality of evaluations has progressed significantly and the evaluation process has evolved, resulting in valuable information on success and failure factors, recommendations for selection and evaluation of projects, and a better understanding of how to exploit results.

Evimp-2 has certainly come a long way from the early days of evaluating projects funded by the EC. The subject of evaluation has met many challenges along the way, a process which began as far back as 1990 with the evaluation of Brite-Euram projects – the framework under which the EC has funded research in industrial and material technologies. Since then, independent contracting companies have helped evaluate EC projects, contributing significantly to unbiased and accurate information.

All these initiatives and earlier evaluation effor ts of the Evimp-2 exercise slowly evolved with the needs and requirements of the EC, preparing the stage for a more comprehensive and objective ex-post evaluation system of funded projects. The initiatives provided valuable insight regarding what made a good project and led to eligibility criteria for projects that sought EC funding.

In 1999, the Commission took the decision to assess all RTD, CRAFT and thematic network projects under Brite-Euram, industrial and materials technologies (IMT) and transport programmes, as well as programmes that fell under Standards, Measurements and Testing (SMT). This turning point signalled the launch of the Evimp exercise. Novel ideas were introduced at this point with respect to evaluation techniques, such as migrating from a quantitative approach to a more qualitative one.

By 2002 the Commission had identified a strong need to assess all RTD, CRAFT and thematic network projects of the Competitive and Sustainable Growth Programme in a more comprehensive manner, launching the EVIMP-2 study for the assessment of 1 500 projects, of which 726

2. EVIMP-2 and the history of evaluation

have been considered in this report. After overcoming a challenging procurement procedure and reorganising the process, Evimp-2 was formally launched in 2004.

Page 14: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

14_Evimp-2

Page 15: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_15

3. EVIMP-2 methodology: the next frontier in evaluation

Page 16: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

16_Evimp-2

Page 17: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_17

A powerful team of evaluators and consulting firms were involved in collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations. The complete evaluation of the Evimp-2 projects was initially divided among different contractors into different lots from 1-7, each covering a different field as outlined in the table below:

Lot nr Intended objective

1Evaluation of the completed projects in the field of Innovative Products, Processes and Organisation

2Evaluation of the completed projects in the field of Sustainable Mobility and Intermodality

3Evaluation of the completed projects in the field of Land Transport and Marine Technologies

4Evaluation of the completed projects in the field of New Perspectives in Aeronautics

5

Evaluation of the completed projects in the field of Materials and their Technologies for Production and Transformation and New and Improved Materials and Production Technologies in the Steel Field

6Evaluation of the completed projects in the field of Measurements and Testing and in the field of Support for Research Infrastructures

7

Support services (to be provided in coordination with lots 1 to 6), exploitation of the resulting database (including statistical analysis), contribution to the evaluation of the programme and its key actions and generic activities, design of a possible follow-up assessment (including testing of the methodology), reports and presentations

Lots 1-6 were assigned to evaluate specific projects within different areas of the Growth Programme, while Lot 7 was assigned to coordinate the different lots and carry out an overall evaluation of the GROWTH programme. The breakdown of contracted companies for each lot is as follows:

Lot 1: VDI/VDE-IT, Berlin, GermanyLot 2: (cancelled)Lot 3: Deloitte, Diegem, BelgiumLot 4: Deloitte, Diegem, BelgiumLot 5: Technomar GmbH, Munich, GermanyLot 6: Technomar GmbH, Munich, GermanyLot 7: Deloitte, Diegem/Belgium, until end of 2006. Matrix as of April

2008, and PubliResearch in 2009

Note that Lot 2 was cancelled in the course of the procurement procedure. In addition, the following tasks of Lot 7 were necessary in the second part of the study and were contracted via ad hoc procedures:web hosting and maintenance of the online database; statistical and qualitative analysis of the collected data;overall reports and publications.

3. EVIMP-2 methodology: the next frontier in evaluation

Page 18: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

18_Evimp-2

Page 19: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_19

3.1. A more thorough approach in data gathering

While to a certain extent Evimp-2 relied on the well established methodology of data gathering that was articulated in previous evaluation systems, the output of the data gathering shifted from a mainly quantitative one to a substantially qualitative analysis. For example, the contractors tested their hypotheses regarding success and failure factors both quantitatively and qualitatively, and verified where the two approaches agreed in their results.

Although this approach may require more time to collect and analyse data, the results are significantly more accurate and reflect the reality of a project’s successes and failures in a manner that previous indicator approaches could not accomplish. This, among other factors such as a dedicated management team of the EVIMP-2 study has led to a substantially more accurate method of evaluating impact and funding of projects.

The independent evaluators collected information on the actual achievements for each project, including RTD results, intended exploitation, socioeconomic impact, policy support, future impact and success or failure factors. The data were gathered from project reports, followed by a web-based search and investigation to determine the extent of publication, exploitation and progress. An interview with the project coordinator ensued to obtain additional information on the project. This resulted in a holistic approach that reveals the complete picture about the project.

All the source data were gathered by German-based consulting firms VDI/VDE-IT and Technomar. The data even included evaluator comments that outlined unique and revelatory points in each project, a procedure that allows a more in-depth look and unique situations. To ensure a consolidated and coordinated approach that would yield firm results, the evaluators of both VDI/VDE-IT and Technomar undertook briefing,

training and calibration sessions, in addition to an inter-calibration exercise between both firms.

Armed with knowledge and data that are required to be current and accurate, the evaluators completed a detailed questionnaire that assisted experts from other consulting firms (Matrix Knowledge Group and PubliResearch) to analyse the data, identify trends and report on the results in a useful format.

While VDI/VDE-IT and Technomar supplied their impressions and conclusions after collecting the data, Matrix provided a more sound analysis of the data. Overall, the contractors arrived at more or less similar and/or complimentary conclusions regarding the projects’ impacts, as well as significant observations and recommendations that would support any assessment of these projects. To illustrate, the results for variables such as success rate and exploitation rate are quite similar.

3.2. Key reports for comprehensive results

In this vein, the three main reports that ensued – in addition to this one – are:

Statistical and Qualitative Analysis of the Evimp-2 Results, by Matrix »Knowledge Group

Evaluation of the Results and Anticipated Socioeconomic Impact of »1 500 Completed Projects of the Growth Programme – Lot 1, by VDI/VDE Innovation + Technik GmbH

Evaluation of the Results and Anticipated Socioeconomic Impact of »1 500 Completed Projects of the Growth Programme – Lot 5, by Technomar GmbH

Exploitation of the EVIMP-2 Evaluations to Ascertain the Reasons for »Weak or Non Exploitation of RD Results, by Dr. George Vekinis

Matrix fed data into its study collected by the independent evaluators over a four year period between 2004 and 2008. To reiterate, its report combines and analyses findings from the lots of projects assigned to VDI/VDE-IT (Lot 1) and Technomar (Lot 5).

The Vekinis report has been useful in understanding the issue of exploitation. It conducted its own analysis based on its own ‘project exploitation factors’ or reasons for weak and non exploitation of results. These factors were classified under Partnership, Technology, Market, and Financial, identifying projects according to the current or potential level of exploitation in four categories: Unsuccessful, Pending Low, Pending High and Successful.

3. EVIMP-2 methodology: the next frontier in evaluation

Page 20: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

20_Evimp-2

Page 21: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_21

4. Profileofprojects:asolidreflectionofFP5

Page 22: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

22_Evimp-2

Page 23: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_23

Evimp-2 has evaluated a wide range of projects within the Growth Programme under FP5. These projects aimed to generate knowledge and/or technologies which would lead to increased economic growth and to the creation of new jobs in Europe. They ranged from tunnel-boring technology and waste minimisation technology to ceramic coating and nano-tubes for storing energy.

While this sounds like a wide range of disciplines and industries, the projects assessed by Evimp-2 were grouped according to the Growth Programme’s strict classifications. The projects that were evaluated in Lot 1 fall under the FP5 classification called Key Action 1 (innovative products, processes and organisation) – more commonly known as KA1 – while the projects evaluated in Lot 5 fall under the FP5 classification GA1, which comprised Generic Activity 1A (Materials and their Technologies for Production and Transformation) and 1B (New and Improved Materials and Production Technologies in the Steel Field).

Each of the classifications were subdivided into Research and Technological Development (RTD) projects, Cooperative Research projects (CRAFT) and Thematic Network (TN) projects under the FP5 Growth Programme.

4.1. Clusters developed for the analysis of Evimp-2 results

Cluster analysis is an exploratory tool that reveals natural groupings (or clusters) within a dataset that would otherwise not be apparent or easily determined. To maximise the output of the available data, clusters were developed for the more detailed analysis of the data set. This analysis identifies meaningful groups within the dataset, reducing its dimensionality and providing more detailed analysis.

The most important variables that were available for analysis were defined as:

project type (RTD, CRAFT, Thematic Networks); ■

funding activity (KA1 or GA1/Materials); ■

number of partners; ■

budget (estimated cost as per final contract); ■

percentage of partners in research and higher education. ■

The analysis revealed 5 emerging clusters: 1 of 189 projects (26.1 %), 1 of 64 projects (8.9 %), 1 of 43 projects (5.9 %), 1 of 173 projects (23.9 %) and 1 of 254 projects (35.0 %).

Figure 2: Proportion of total sample occupied by each cluster

35%

24%

6%9%

26%

Cluster 1:CRAFT/KA1

Cluster 2:CRAFT/GA1

Cluster 3:Thematic Networks

Cluster 4:RTD/GA1

Cluster 5:RTD/KA1

Figure 1 shows how many projects there are by project type and funding activity, while figure 2 shows the number of projects by cluster.

LOT 1 (KA1) LOT 5 (GA1) TOTAL

RTD 258 173 431

CRAFT 189 65 254

Thematic Networks 35 6 41

TOTAL 482 244 726

Figure 1: Breakdown of project types by activity and Lot

4. Profileofprojects:asolidreflectionofFP5

Page 24: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

24_Evimp-2

Page 25: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_25

5. Overall results

Page 26: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

26_Evimp-2

Page 27: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_27

The various reports have yielded a number of key findings from the ex-post evaluation of the concerned projects. In general, around two-thirds of the projects were successful to different degrees. The overall results from the analysis of 726 projects under Evimp-2, representing EUR 992 million of EU funding, reveals the following:

depending on the cluster, 60-90 % of the projects wouldn’t have been ■

realised without EC funding (see Figure 3 below);75 % of the projects were effective in achieving their scientific-technical ■

objectives, and 54% were effective in achieving both their scientific-technical and exploitation objectives;

CRAFT/KA1 CRAFT/GA1 ThematicNetworks

RTD/GA1 RTD/KA1

Relatively small budgets Relatively large budgets

Project realised irrespective of EU funding

Project realised within smaller partnership

Project realised within one organisation

No realisation of project

100 %

90 %

80 %

70 %

60 %

50 %

40 %

30 %

20 %

10 %

0 %

7

25

714

3

23

134 34 28 101

5

25

7

143

25

65

40

0

Total 1- Less than 3 years 2- Between 3 and 5 years

3- Between 5 and 10 years

4- More than10 years

8038

3684

3272

979103

An interesting finding has emerged from the Technomar report which had sub-divided the projects into different groups of companies within more or less the same technical area or commercial sector, revealing that the Nanomaterials cluster delivered better results than others. The ranking of the projects by industry of cluster, from most successful to least successful, is as follows:

nanomaterials and nano-techniques ■

building materials ■

coating ■

implants for medical application ■

catalysts (automotive and refinery) ■

aluminium processing ■

Note that partners involved in research projects are enterprises on one hand and research-oriented partners such as universities on the other. Each group has different reasons for participating in EU-funded collaborative projects.

Figure 3: Clusters – if project would have been realised without EC funding

Figure 4: Expected number of jobs created over time

46 % of the projects showed that outcomes were qualified by the ■

project consortia as worth the cost or worth substantially more;41% of the projects were qualified as direct successes, 31% as ■

conditional successes, 17% as projects with low impact but valuable results and 12% as failures;70 % of projects expect at least some benefits in increased sales; ■

34 % of projects expect at least some benefit from cost reduction; ■

41 % of projects expect some benefit from reduced financial risk. ■

Figure 3 illustrates the impact on funding on realisation of projects across clusters as explained in section 4.1. Evaluators identified the extent to which the project may have been undertaken had EC funding not been made available. Four ratings were used that ranged from no realisation of the project to the project being realised in the same way without funding. Between 60 % to 90 % of projects would not have been realised without EC funding.

In addition, the projects have generated or are expected to generate the following results in the short term:

3 724 prototypes/demonstrators/pilots; ■

747 new software tools; ■

18 974 publications; ■

2 152 doctorates; ■

310 inputs into technical standards; ■

423 inputs into EU legislation texts; ■

1 077 patent applications; ■

204 registered designs and other forms of IPR protection; ■

the creation of 248 spin-off companies. ■

Over the long term, the projects are expected to result in:around EUR 7.2 billion additional sales; ■

almost EUR 0.9 billion in cost reduction; ■

the safeguarding of over 37 500 jobs (See figure 4 below); ■

over 8 000 new jobs (See figure 4 below). ■

5. Overall results

Page 28: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

28_Evimp-2

5.1. Benefitsforindustrialpartners: More exploitation, fewer risks and costs

Evaluators identified three main benefits for exploitation partners or businesses, namely increasing sales through improved competitiveness on the market, reducing actual costs through improved efficiency, and reducing financial risk in R&D. Figure 5 outlines the degree of expected benefits for exploitation partners. It also shows the percentage of projects to which this variable was applicable. (E.g. ‘increased sales’ is an expected benefit that applies to around 87% of projects. It is not a consideration for 13% of the projects which may expect other benefits.)

Expected benefits for the exploitation partners

No benefit

Some benefit

Medium to high benefit

Percentage applicable projects

Increased sales (through improved competitiveness on the market)

20.0% 22.9% 57.1% 86.9%

Reduced actual costs (through impwroved efficiency of organisational structure / processes / work methods)

44.3% 22% 33.7% 60.9%

Reduced financial risk in R&D area

23.1% 37.1% 39.8% 53.6%

The expected benefits for exploitation partners overall were very positive. Figure 5 above shows that 80 % of projects expect to increase sales to different degrees through improved market competitiveness. More than half of all projects reported medium to high benefit in estimated increased sales. Over half of the applicable projects also expect a reduction in costs, and over 75 % of applicable projects also expect reduced financial risk in R&D.

5.2. Benefitsforresearchpartners: Furthering research and optimising funding

There were numerous research partners from all corners of Europe involved in these projects, such as academic organisations and scientific participants. The main benefit for such partners does not necessarily focus on exploitation or financial gain. Instead, these parties are motivated mostly by technological development, reputation of the organisation, networking opportunities, financing research, and exchanging staff between partners. The industrially oriented research community will also benefit as Evimp-2 can study the value of research and improve research programmes to enable more successful projects. In other words, guided by the success and failure of the projects, Evimp-2 and its results can help direct the contents of the programme and funding schemes towards projects that would result in maximum impact, explaining choices to be taken in later RTD programmes.

Success stories can also encourage companies in similar fields and motivate them to use EU funding wisely. Ultimately, even society at large stands to

Figure 5: Expected benefits for exploitation partners – increased sales, reduced actual costs, reduced financial risk

Page 29: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_29

5.4. Promising exploitation opportunities

At least 80 % of the projects have at least partly reached their technical goals in terms of a functional prototype, etc. while about 70 % of the projects were able to use these results for broad dissemination activities up to today. Also noteworthy is that about 65 % of projects are expected to transform their results into serial production processes in coming years.

Beyond the basic figures above, an accurate way of looking at exploitation is through the Vekinis study. The main exploitation factors (positive or negative) that may have influenced exploitation of each project, according to the study, were categorised under the following: Partnership, Technology, Market, Financial and Other. The evaluations led to ‘Exploitation Potential’ for each project, as well as an approximate indication of the prospects and main prerequisite activities to achieve this potential.

A significant number of successful projects apparently owed their high exploitation success to stronger regulatory support from local governments and EU-led programmes. In most successful or eminently successful projects, factors such as further R&D, enhanced marketing and additional funding were found to be well under control and are expected to enable or enhance exploitation in the coming years. A number of ‘Pending-High’ projects had developed high quality technologies which are poised to become highly competitive once various regulations (safety or environmental) are tightened or enacted.

The study also found that exploitation factor categories ‘Technology’, ‘Partnership’ and ‘Market’ (in this order) are critical for success or failure, and that most ‘Unsuccessful’ projects displayed negative factors in at least two of the above three critical categories. In addition, weak market need or demand was a major reason for failed projects, while ‘Financial’ obstacles were not seen as a major reason for exploitation failure, but rather for delayed exploitation.

To illustrate, Figure 7 below lists the factors related to the exploitability of the technology that was developed or its level of development.

Figure 6: Expected environmental impact of 726 EC funded projects under Evimp-2

Figure 7: Frequency distribution of exploitation and non-exploitation factors in the category ‘Technology’ for the four Effectiveness Categories

benefit from the enhanced impact of stimulating research and innovation through new technology, products and services that are increasingly user-friendly and those that are sensitive to the environment.

With respect to exploitation, Evimp-2 can reveal where results were not exploited and may identify cases where important technology could be promoted, outlining recommendations for follow-up assessment as well.

5.3. Contribution to the environment

One of the objectives of the Growth Programme is to encourage innovation and exploitation in environmentally friendly technologies, goods and services. The EVIMP-2 study has documented the expected environmental impact of the projects it evaluated, noting that environmental benefits were expected to increase for 72 % of applicable projects, 35 % of which anticipated a medium high or high benefit in reduced environmental impact. Figure 6 below summarises the expected types of environmental impact.

Expected environmental impact

No benefit

Some benefit

Medium or high benefit

Percentageapplicableprojects

Reduction or prevention of emissions (pollution/noise/radiation)

28 % 37 % 35 % 68%

Saving natural resources, energy consumption

27 % 43 % 30 % 69%

Improved treatment of emissions, recycling processes

51 % 24 % 25 % 55%

High

tech

nical

comp

etitiv

enes

sHi

gh co

st be

nefit

Not t

echn

ically

comp

etitiv

eNo

t eco

nomi

cally

viab

le/co

mpet

itive

Tech

nolog

y nee

ds m

ore

RDIP

R pr

oblem

s or n

o IP

R pr

otec

tion

Confi

dent

iality

pro

blems

Tech

nical

prob

lems

Tech

nolog

y life

-cyc

le to

o sh

ort

No e

nabli

ng te

chno

logy

60.0

50.0

40.0

30.0

20.0

10.0

0.0

UNSUCCESSFUL

PENDING-LOW

PENDING-HIGH

SUCCESSFUL

5. Overall results

Page 30: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

30_Evimp-2

Lastly, it is worth noting that the majority of projects whose exploitation was ‘pending’ with low prospects of eventual success showed that there were negative factors in at least one critical criterion, although to a lesser extent than unsuccessful projects. Most required more R&D to reach exploitable results (partners do not or cannot continue activities to the extent required or some other major factor impedes exploitation). Regulatory obstacles, financial obstacles or partnership problems also resulted in weak exploitation.

5.5. Achieving technical and otherobjectives

Overall, 75 % of the projects were effective in achieving their scientific-technical objectives, and 54 % were effective in achieving both their scientific-technical and (at least in part) their exploitation objectives (which only 28 % achieved fully), as outlined in Figure 8. With respect to project aims, it is interesting to note that clean and high quality products were considered more important objectives than the issues of safety and user-friendliness, and nearly 85 % of the projects were dedicated to at least 2 of the 4 general programme aims. Failure of some projects was caused by overambitious technical objectives, high costs for production and the absence of a clear business plan for expected return on investment (in this order). KA1 projects, for example, failed mainly due to overambitious technical objectives.

5.6. Management experience and consortium structure

Pre-existing management experience of the project partners, and specifically the project coordinator, lead to successful projects. With this in mind, 84 % of project management was deemed to be adequate or above, with 12 % classified as outstanding. Only 14 % of management quality was considered poor, while only 2 % was considered as a clear failure. Previous experience of project management increased the degree of success achieved, suggesting that training and/or mentoring could further increase the success of projects. Project management capabilities were particularly important for the success of Thematic network projects, which tended to have a higher number of partners.

In addition, the complete value chain (supplier-manufacturer-user) represented by partners does not affect project success. Whereas representation of the whole value chain seemed of less importance for success, the integration of users in the consortium was of key importance, where relevant. Balanced consortia, where roles and motivations are clearly defined and complementary, have been important to success. In particular, motivation and commitment of

the leading partners are essential success factors. In addition, the structural capability of a consortium for later exploitation of RTD results was also seen to significantly influence project success/failure.

Finally, it is worth noting that neither the number of project partners nor the number of nations involved influences project success, but more participants per nation lead to higher probability of success.

5.7. Projectsuccessandfailure factors

The number one success factor could be seen as the degree of technical capabilities held within the consortia. Other key success factors include consortia exploitation capabilities and how well they were able to manage risks around exploitation. They also included the quality and capabilities of the assigned project management.

When comparing commercial-based projects to research-based ones (also referred to as applied projects and fundamental research projects respectively), the commercial or applied ones seem to be more successful. However, one must consider that the figures would show higher success of commercial projects simply because the results are more easily exploitable. By their own nature it is difficult to assess the success and impact of research-based projects.

In addition, RTD ‘Materials’ projects failed more often due to high costs of introduction/production and absence of a clear business plan. CRAFT ‘Materials’, on the other hand, failed more often due to lack of market knowledge. Interestingly, external factors like unexpected market development or new legislation are not that important for project success or failure.

%

28%

26%

%25

%13%6

2

Technically effective and exploitation effective

Technically effective and mostly met some exploitation objectives

Technically effective and mostly did not meet exploitation objectives

Technically effective and no exploitation objectives

Mostly met some technical objectives

Mostly did not meet technical objectives

Figure 8: Effectiveness classification of projects

Page 31: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

5.8. RTD vs. CRAFT analysis

There is some evidence that RTD projects are more successful than CRAFT projects overall. CRAFT projects include a very high percentage of ‘SME-intense’ projects. However, ‘high-SME-intensive’ projects perform better when implemented as RTD projects rather than CRAFT projects. In 60 % of the RTD projects, SME participation is ‘good’, ‘very good’ or ‘excellent’, implying that the RTD project framework is ‘SME friendly’. Apart from monetary value, the advantage of RTD projects over CRAFT projects also holds for other indicators, such as impact on partners, industry, environment, and society.

Analysing the Outcomes of Ec Funded Projects under FP5_31

In general, a significant amount of information for future selection and funding can be obtained from the factors that lead to success or failure of the projects. The Matrix study has analysed the results obtained by the evaluators of the hundreds of projects to determine the most important success and failure factors. The top success and failure factors are outlined in Figure 9 below:

Main project success factors Main project failure factors

Technical experience of partners / availability of prior knowledge / technology input Overambitious technical objectives / technical complexity

Integration of users in partnership High costs for introduction / production

Technical skill Absence of (clear) business plan for return on investment

Level of technological complexity/ambition challenging but manageable Duration of the project / management of timing

Project management skills of coordinator Knowledge of markets: needs, sales potential not identified

Knowledge of industrial processes (source, design, plan, make) within partnership Project management skills of coordinator

Availability of adequate resources (manpower, budget and equipment) Changed market

Clarity of objectives / management of scope and expectations Users not (or not enough) represented in partnership

Knowledge of markets: identified existing needs / availability of clear marketing plan Availability of (additional) internal funding (from partners)

Experience of partners in similar projects / with similar consortia Communication / management of the project structure and cultural differences

Strength of partnership: ongoing commitment of management/strong sponsorship of main partner

Availability of adequate resources (manpower, budget and equipment)

Support by the European Commission or related Support Organisation Weakness of partnership: ongoing commitment of management/sponsorship of main partners

Communication / management of the project structure and cultural differences Technical experience of partners / availability of prior knowledge / technology input

Exploitation agreement between partners Changed technological environment

Figure 9: Main project success and failure factors as evaluated by Evimp-2

5. Overall results

Page 32: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

32_Evimp-2

Page 33: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_33

6. A closer look at individual studies

Page 34: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

34_Evimp-2

Page 35: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_35

A closer look at each of the reports outlined in section 3.2 will help underline the subtle differences in the approaches and methodology, providing a clearer picture to stakeholders involved in ex-post evaluations. The reader must keep in mind that the report by the Matrix Knowledge Group is based on substantial scientific rigor, while the others provide useful information that is nonetheless anecdotal in its nature.

6.1. Matrix Knowledge Group

The Matrix Knowledge Group was contracted by the Directorate-General for Research, Technology and Development of the EC to undertake statistical and qualitative analyses of data collected as part of an evaluation of projects under Evimp-2.

The evaluators that undertook the data collection assessed each of the 726 projects individually using a standardised, online tool. With its study, Matrix aimed to provide valuable insight to the Commission and other policy makers within the European Institutions (e.g. the European Parliament and the Council) and beyond (e.g. Member States) regarding expected investments into similar programmes. It also intended to help inform experts carrying out the ex-post evaluation of FP6 in 2008.

Another main aim has been to gain as much insight as possible into the critical factors that influence success and failure in achieving impact. Although research tends to focus on perceived successes, this study has taken a more comprehensive approach to also understanding why some projects fail, since this is likely to generate more valuable lessons for dissemination to policy makers and project beneficiaries alike.

Matrix first focused on identifying expected impacts before proceeding to measure the data. After measuring the data, it found that the number

one success factor was the degree of technical capabilities held within the consortia. Other key success factors included consortia exploitation capabilities and how well they were able to manage risks around exploitation. For instance, a clear plan for commercial exploitation increased the success rate of projects. This aspect was particularly relevant for the success of CRAFT projects and projects with a high proportion of SMEs.

Key success factors also included the quality and capabilities of the assigned project management. Feedback from the evaluators showed that previous experience of project management increased the degree of success achieved, suggesting that training and/or mentoring could further increase the success of projects. Project management capabilities were particularly important for the success of Thematic network projects, which tended to have a higher number of partners.

Where projects failed, failure was due to overambitious technical objectives, high costs of production and the absence of a clear business plan for expected return on investment. In light of these findings, the following amendments could be made to expected project requirements in order to improve programme outcomes:

greater emphasis on realistic costing of production costs and realistic ■

business plans in expected selection processes;including business expertise in the setting up of projects; ■

explicit emphasis on the development of environmentally- and consumer- ■

friendly products in line with programme objectives.

6. A closer look at individual studies

Page 36: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

36_Evimp-2

for the coordinator and his project as well as improve reporting for the EC. This will facilitate the extraction of success and failure cases, and will allow a more efficient exploitation and impact follow-up by the EC.

A number of cases showed that the project partners would appreciate a closer and more continuous monitoring of their project by the EC scientific officer or a Project Technical Advisor (PTA). In addition, the plan to achieve the objectives, and the ‘real’ industrial implementation prospects have to be examined in more detail especially when evaluating CRAFT project proposals during the selection phase of projects for funding, taking the limited resources of SMEs into consideration. Such a procedure might lead to projects with clear business plans for technology implementation, probably with rather simple project structures, better meeting the needs of SMEs.

VDI/VDE-IT concluded its findings with a list of crucial parameters for the success of projects:

R&D projects generally yield more impact than CRAFT projects, across a ■

broad range of success parameters.Pre-existing management experiences of the project partners and the ■

project coordinator in specific lead to successful projects.The structural capability of a consortium for the later exploitation of the ■

RTD results is significantly influencing the success/failure of a project.Over-ambitious projects (technical risk ‘very high’) tend to fail. ■

Large projects in terms of budget lead to rather successful projects ■

(critical mass effect).Projects coordinated by industrial organisations show some ■

disadvantages regarding the impact on industry in general (as opposed to the specific companies involved in the project), and on environment. The consortium structure in terms of the market position of the ■

industrial partners is important for success: if the market leading – or at least ‘important’ – company/companies are partners in the consortium, project success is more likely, especially regarding ‘value for money’.In CRAFT projects, a small number of partners is favourable. ■

6.2. VDI/VDE Innovation + Technik

VDI/VDE-IT was contracted by the EC to perform an evaluation of the results and anticipated socio-economic impact of completed projects of the Growth Programme of the European Union (EU). Its report contains the findings of 482 projects.

This study featured evaluation factors such as ‘Valuable competitive technology’ and ‘Exploitation after further development’ so as not to focus too much on commercial success only. While direct commercial success of a research project is an excellent success measure, pre-competitive research projects can be seen as very successful as well, particularly with direct competitive product development based on the newly developed technology.

VDI/VDE-IT identified the most prominent influence on the success of a project to be the capability and the well managed risk of exploitation. Having a clear plan for commercial exploitation during the launch phase of a project that is updated regularly is highly important as well.

It is not sufficient to start planning exploitation issues when a project is already under execution. At that stage, there is little chance to influence the project objectives and the partnership. Exploitation strategy seminars would be more effective at a very early stage of the project, or even as part of the contract negotiation. This would allow the possibility of integrating results of the seminar into the work plan of the project and to monitor the exploitation strategy during the execution of the project.

Another important success factor is the project management capability of a consortium and the coordinator in specific. This criterion should be very well assessed during the project proposal evaluation phase. Pre-existing experience with the management of international projects seems to be of paramount importance. Specific training measures for project coordinators could be helpful

Page 37: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_37

6.4. Vekinis Report

The study by Dr George Vekinis on ‘Exploitation of the EVIMP-2 Evaluations to Ascertain the Reasons for Weak or Non Exploitation of RD Results’ has been instrumental in shedding light on issues related to exploitation. The study represented a meta-analysis of 684 CRAFT and RTD projects.

Based on EVIMP-2 evaluations, the main objective of the meta-analysis by the Vekinis study was to prepare a quantified overview of reasons for non-exploitation of project results and of their impact on the effectiveness of the projects.

Regarding the rate of success in terms of exploitation, the Vekinis report categorised the studies under four exploitation headings: Successful, Pending-High, Pending-Low and Unsuccessful. Again, in similarity with overall success, the Vekinis report showed that no less than two-thirds of the projects achieved at least a moderate level of exploitation or eventual exploitation. It outlined that 187 projects (27.3 %) achieved at least a moderate level of exploitation with a number of projects being significant ‘success stories’, and were considered ‘successful’, while 249 projects (36.4 %) showed at least moderate prospects for eventual exploitation achievement and were considered ‘Pending-High’.

For CRAFT projects, it seems to be essential that at least one major player on the respective market is involved. It is not possible to define a minimum company size, since leading players on niche markets may be very small.

With respect to impacts on partners, industry/economy and environment, Technomar unsurprisingly found that they are linked with project success.

Employment effects however are not equally distributed among projects, but concentrated on a few. The employment effect of CRAFT projects naturally is not as strong as it is for RTD projects, although concerning the safeguarding effect for jobs, it is comparatively strong. This is partly due to the sometimes existential importance of project success for SME partners (project success saves the jobs of complete staff).

All the projects have a positive impact on society, at least by creating long-term international R&D cooperation, in addition to environmental benefits such as reducing energy consumption and related emissions.

Technomar noted that evaluation of potential impacts may not really be determined now, since they depend upon real project exploitation. It recommended establishing a factor regarding the probability of real exploitation, derived from project situation at the time of evaluation. Another point when comparing impacts of projects is that no difference is made between projects that intended to achieve certain benefits, and those that had not. It would be more meaningful to document 1) the intended impacts, and 2) the degree of achievement of these intentions.

6.3. Technomar GmbH

Technomar GmbH was contracted by the EC to perform an evaluation of the results and anticipated socio-economic impact of completed projects of the Growth Programme. It evaluated 244 projects and allocated them in four main success categories. Technomar found that almost half the number of RTD-projects were completely successful, while another 31% displayed success with some restrictions, i.e. 80% were complete or conditional successes. While 15% delivered at least some valuable results, only 6% were real failures. The graph for CRAFT projects has a slight peak at ‘conditional successes’, but when adding successes and conditional successes, the share is 78 %, almost as high as the one for RTD projects (80 %). There were 12 % of the CRAFT projects that delivered at least some valuable results, and only 9 % were real failures.

Technomar outlined that project types are related with project success: applied projects are more successful than fundamental ones. Among applied projects, internal ones were successful more often than externally oriented ones, especially with respect to CRAFT projects. Capability and risk categories are also related with project success. For CRAFT projects, there is a clear gap between exploitation capability and exploitation risk. Appropriate capabilities should thus be checked in proposal evaluation, as carefully as possible.

Another recurrent finding is that motivation and commitment of the leading partners are essential for project success. In very large companies, however, strategic conflicts may occur between different activities of the exploitation partners, e.g. military application versus civil exploitation/dissemination, leading to reduced commitment or blockade of exploitation. Frequent change of responsible persons and delegation of key tasks to less experienced colleagues may be behind loss of motivation and commitment. Another challenge is the blocking of exploitation by an industrial partner. This sometimes occurs with very large partners who dominate the consortium. Due to the lack of appropriate IPR arrangements, the other partners feel helpless.

6. A closer look at individual studies

Page 38: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

38_Evimp-2

Page 39: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_39

7. Conclusions and recommendations

Page 40: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

40_Evimp-2

Page 41: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_41

7.2. Evaluation recommendations

Use evaluator’s feedback to enhance the existing evaluation framework ■

through improved impact and success indicators. Consider a more targeted sector focus to increase accuracy of ■

evaluation. Facilitate the access of information by proposal evaluators on the ■

various exploitation factors during evaluation.Introduce a factor on the probability of real exploitation at the time ■

of evaluation in order to improve evaluation of potential impact on partners, industry, environment, etc.Explore intended impacts vs. non-intended impacts more closely. ■

Rephrase the risk and capability indicators into ‘technical’, ■

‘manufacturing’ and ‘exploitation’ for better comparison and evaluation. Evaluate management capability through direct comparison with ■

management requirements or by measuring management facilities available to the coordinator as well as indicators for organisational project complexity (partners, structure, etc.).

Valuable recommendations have emerged from the various studies, particularly with respect to project selection and evaluation. The findings of the evaluators and consulting firms in this respect are complementary in nature and provide a powerful set of options to hone the selection and evaluation process. In this respect, the EVIMP-2 initiative can be seen as invaluable in its contribution towards this end; its full benefits are sure to be felt increasingly over the coming years.

7.1. Selection recommendations

Increase emphasis on more realistic business plans and estimation of ■

production costs initially.Enhance business expertise in setting up of projects. ■

Put explicit emphasis on developing environmentally and consumer ■

friendly products in line with programme objectives.Use exploitation experts to aid in proposal assessment in order ■

to increase project effectiveness by spotting early warning signs and proposing solutions such as refocusing the project, optimising exploitation, developing funding policy and rearranging partnerships. Encourage SMEs – especially ‘high-tech’ ones – to participate in R&D ■

projects and integrated projects by creating support measures. Reformulate objectives for CRAFT or similar instruments, making them ■

more specific and realistic through additional support, clearer project structure, increased evaluation and a limit on partners.Develop more appropriate questions during proposal evaluation for ■

CRAFT projects.

7. Conclusions and recommendations

Page 42: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations
Page 43: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

SUCCESS STORIES

Page 44: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

44_Evimp-2

Page 45: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_45

Table of Contents

3DIMPRINT 46THE INTEGRATION OF NOVEL PRINTING AND FORMING TECHNIQUES FOR 3D IN-MOULD PRINTING

DOTS 48MAKING THE PULP AND PAPER INDUSTRIES MORE COMPETITIVE AND ECO-FRIENDLY

ECOPRESS 50ECONOMICAL AND SAFE DESIGN OF PRESSURE VESSELS APPLYING NEW MODERN STEELS

LISCOS 52LARGE SCALE INTEGRATED SUPPLY CHAIN OPTIMISATION SOFTWARE

AMOR 54MECHATRONIC UPGRADE AND WHEELCHAIR INTEGRATION OF THE MANUS ARM SERVICE MANIPULATOR

CERHYSEP 56CERAMIC MEMBRANES FOR HYDROGEN SEPARATION

INTELLIDRAW 58INTELLIGENT DRAW PROCESS FOR HIGH QUALITY YARNS

NACODRY 60DRY STAMPING AND DRY MACHINING OF DIFFICULT-TO-CUT MATERIALS BY MEANS OF SUPER-HARD NANOCOMPOSITE COATINGS

Page 46: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

46_Evimp-2

IMD yields high-quality decorated finishes that embellish surfaces with the desired design and/or provide significant resistance against abrasion and general use. The process offers several advantages regarding design flexibility and productivity, particularly when compared to other decoration methods that are applied after the moulding process.

Five industrial partners from Spain and the UK, in addition to one British university and a Spanish research centre, worked closely together to realise the objectives of the Cooperative Research project (CRAFT) called 3DIMPRINT. These objectives aimed to enhance and improve the process of decorating by digital and advanced silk-screen printing techniques, as well as further the potential of IMD by overcoming other major challenges associated with this process. The partners hoped that by achieving these objectives, economic advantages like process cost reduction and increased turnover would be realised. There were environmental objectives as well, such as reducing harmful gases using solvent-free inks, and improving the workplace.

A WORLD OF TECHNICAL IMPROVEMENTS

The 3DIMPRINT project had many ambitious technical objectives that the consortium of companies and research parties investigated. To begin with, the project wanted to perfect a physical method for controlling the ‘print carrier’ during thermo-forming up to a depth of 150 mm in print registration. It also intended to develop printing technologies for direct transfer to the print carrier to achieve deep engraving. Enhancement and adaptation of new and existing printing techniques to achieve a reduction in print setup times and costs was also an important consideration.

While most of the project’s objectives were achieved, the consortium discovered that the process of digital printing could not be made accessible for in-mould decoration. The main technological result of the project, nonetheless, was an innovative carrier method to facilitate the control of the thermoforming process when using printed film. In more detail, the technical achievements of the project included:

novel transfer carrier printing system, new workflow principles; ■

novel measurement devices for online measurement of printing quality; ■

novel positioning technology; ■

concept for an integrated 3DIMPRINT work-cell that facilitated individual ■

technologies;new design rules for in-mould tools, reflecting 3DIMPRINT technologies. ■

In addition, while the project’s initial focus was on digital IMD, towards the end additional attention was placed on adjusting the developed components for the process of high-pressure moulding in IMD as well. This increased the beneficial outcomes of the project and shed more light on innovations in this kind of technology.

Three-dimensional (3D) in-mould decoration (IMD) has a myriad of applications in industry, from printing on products in colour to developing a stronger and/or decorative 3D surface on a wide variety of materials, from metal to plastic. The 3DIMPRINT project has helped advance the IMD process by improving the carrier method for increased control, elaborating better measurement devices and developing new design rules for in-mould tools.

THE INTEGRATION OF NOVEL PRINTING AND FORMING TECHNIQUES FOR 3D IN-MOULD PRINTING

3DIMPRINT

Page 47: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

www.walterpack.comwww.photofoil.com

Analysing the Outcomes of Ec Funded Projects under FP5_47

POSITIVE OUTCOMES FOR STAKEHOLDERS…

These outcomes have been particularly beneficial for the industrial partner Walter Pack, who is very active in field of thermoforming. This has enabled the company to develop a more efficient thermoforming process and to adopt the often superior process of high-pressure moulding in order to realise relevant applications.

Components and process know-how resulting from the project have enabled Walter Pack to enter the IMD business and to establish a solid market position in the field. The company has become the main exploitation partner, and has also established a base in India.

On the other hand, while the former coordinator shifted the focus of his business – Diamond Photofoil Ltd – away from injection moulding due to market changes, he now has the ability to sell a know-how licence (on films for digital transfers) to machine manufacturers who are interested in the process.

The other industrial partners and research partners benefited from an increase of know-how in the field of IMD and thermoforming processes. Small benefits for environment and society have also been gained by a modest reduction of raw material usage and improved working conditions.

Lastly, it is worth mentioning that evaluations of project partners and outcomes show that without EC funding, the project would not have been realised and the advancement in IMD processes would not have been achieved.

Page 48: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

48_Evimp-2

Representing a wide range of European expertise, the DOTS project consisted of 11 partners from Germany, France, Finland and Sweden. Initiators of the project were the Finnish partners Oy Keskuslaboratorio – Centrallaboratorium AB (KCL) and the UPMKymmene Group (UPM). KCL is a private research company focusing on the pulp and paper industry and is partly owned by UPM. As KCL is primarily a research institute, KCL represented the main exploiting partner, particularly since project results could be matched with their already existing process analysis system, known as WEDGE. The world’s third biggest paper manufacturer, UPM, was the main industrial partner.

Besides KCL and UPM, the consortium brought together three research institutes specialised in the pulp and paper industry from Germany, France and Sweden, two universities specialised in process automation from Finland and Sweden, and one research institute for computer science from Sweden. The research partners were mainly interested in expanding their knowledge in the field of dynamic optimisation. While there are a variety of methods for static modelling available, dynamic process models and dynamic optimisation still need to be investigated and adapted to the situation in the pulp and paper industry.

UPM and three paper mills from Germany and France were industrial partners whose motivation for participating in the project was to improve paper production processes, leading not only to a reduction in cost, but also to increased production flexibility and better handling of environmental issues. Project results could be tested in real cases and therefore immediately implemented. In addition, the results could be combined with KCL’s WEDGE platform.

DEVELOPING THE RIGHT SET OF ‘TOOLS’

Paper production lines are most efficient when producing a single grade over long periods. Most of the operational policies are designed from this normal production point of view. Hence, the production lines are not flexible enough to meet upcoming requirements. A major aspect in tackling these challenges successfully is to operate the paper production process more actively and dynamically.

In order to increase flexibility and agility of paper production lines, DOTS had several objectives. The first objective was to develop operations scenario management and dynamic optimisation tools to be used with existing process simulators. Secondly, it aimed at combining scenario management, dynamic optimisation and dynamic simulation into a process operator decision support system for better eco-efficiency and flexibility. Finally, there was the need to test the decision support system in four full-scale paper mill applications.

At the end of the project, the DOTS toolset for operations decisions support was successfully developed, as was a scenario management tool for dynamic simulations and new optimisation tools.

The DOTS toolset was assessed in four real-life case study applications at participating paper mills. Two of the cases were tank farm management problems with simple unit processes but complex volume and material quality management issues, one case was a plant scheduling application with full online optimised decision support, and the other was a sizing management problem. An application methodology, including user requirements, functional specifications and methodology for presenting the optimisation results to operators were developed.

European pulp and paper industries are facing four impor-tant challenges: they have to be flexible towards custom-ers, cost effective, ecologically sustainable, and attractive workplaces. They must also stay competitive in the global market and in the face of other technologies. The DOTS project was established under the Commission’s GROWTH programme to provide the European pulp and paper indus-try with tools that allow process operators and engineers to manage complex dynamic production problems in an efficient and eco-friendly manner.

MAKING THE PULP AND PAPER INDUSTRIES MORE COMPETITIVE AND ECO-FRIENDLY

DOTS

Page 49: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_49

http://ww2.savcor.com/forest/

As simulation and optimisation methods are not generally known or used on a daily basis in pulp and paper industries, a careful study of optimisation needs at mills as well as a definition of user-friendly interfaces for studying results were necessary. Originally, the project partners considered providing a generic end-user interface for the DOTS toolset, but as current practice at production lines is to reduce the number of user interfaces, the pre-existing interfaces had to be used. This may somewhat slow down

BENEFITS: REDUCING COSTS AND MODERNISING THE INDUSTRY

The decision support system developed optimises economic performance and also takes into account environmental issues through weighted economic and ecological objectives. Paper production costs are being reduced, and it is now possible to turn out more products using fewer raw materials. Moreover, the system will improve the operator’s working conditions. It will also result in an improvement in economic activity due to increased competitiveness of the European pulp and paper industries. In turn, this effect will also contribute to the modernisation of European industry, and to some extent to increased research activities of the paper manufacturers.

After the project came to a close in June 2005, KCL placed the further development and marketing of the DOTS project results together with their WEDGE process analysis system in a separate spin-off company named KCL Development Oy, in order to be more flexible and focused in the growing process analysis and optimisations tools market. In October 2006, this company was completely sold to Savcor Process Oy, a large Finnish company specialised in process control solutions and other products for the whole pulp and paper production chain. Savcor integrated the WEDGE and DOTS solutions in their product portfolio, confident that these solutions will bring totally new possibilities for controlling processes more accurately and will achieve the greatest possible output of expensive process equipment.

Lastly, although all test cases were used for papermaking processes, the methods have the potential to be applied for other process industries as well.

the implementation process, but it will increase end user acceptance. The toolset, which consists of six pieces of software and corresponding manuals, provides a variety of optimisation methods, ranging from implementation of stochastic algorithms to links with external optimisation libraries. The toolset can be used with the existing simple process models as well as with large external dynamic process simulators models.

Page 50: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

50_Evimp-2

Pressure vessels are mostly manufactured in Europe by small and medium-sized enterprises (SMEs), but competition is severe and decisions on where to produce are generally dictated by low manufacturing costs, high-tech considerations and high-quality aspects, as well as safety regulations against potentially catastrophic failure of pressure devices or equipment. In order to maintain its market position in the future, Europe’s strategy must be to combine high-potential steel with new design and fabrication methods.

Technical standards like the European EN 13445, first published in 2002, define the safety requirements for the design and fabrication of components in Europe, in order for them to receive the CE label as visible proof of their

TESTING THE LATEST GENERATIONS OF STEEL

Of the new generations of structural steel to have emerged from European research activities over the last thirty years, the partners focused on high-strength quenched and tempered steels (HSS-Q+T) with yield strength of up to 690 MPa, austenitic-ferritic stainless steels known as duplex steels and thermo-mechanical controlled rolled process steels (TMCP). All of these were only partially, and over-conservatively, covered by the actual existing EN on the materials permissible for use in pressure vessels. The main advantages that these steel types offer compared to conventional normalised steel is that a thinner wall thickness can be used because of higher strength or alternatively, that a wall of the same thickness can withstand more pressure because it is stronger. In addition, these modern steel types offer better fabrication properties while increasing toughness, increasing the overall success of the products in question. The resulting pressure vessels are therefore lighter and easier to transport, while fabrication costs from welding can be effectively reduced.

An extensive testing programme was carried out to establish material properties such as strength, hardness, toughness and resistance at lower temperatures – even down to -100 °C. Such properties must be guaranteed not only for the base material, but also after all stages of the fabrication process (welding, cold forming and heat treatment) which the steel will undergo before the vessel is ready for delivery. Another important part of the research work was devoted to the development of suitable welding consumables for very high-strength Q+T 690 MPa type steel with respect to higher resistance during prolonged Post Weld Heat Treatment (PWHT). PWHT is required beyond a certain thickness limit of about 35 mm after welding, whether during fabrication or after repairs.

DEVELOPING A PATH TO SAFER, BETTER AND LESS EXPENSIVE PRESSURE VESSELS

Pressure vessels are containers designed to resist relatively high pressure often used for chemicals, pressurised gas, liq-uid gas, hydrogen, and even pressure cooking pots, among others. With tough competition from the global market and overly strict rules concerning pressure steel, Europe needed to investigate innovations in this area to guarantee competitiveness.

ECONOMICAL AND SAFE DESIGN OF PRESSURE VESSELS APPLYING NEw MODERN STEELS

ECOPRESSsafety. Even cooking pots with a closed lid are pressure vessels and come under the European directive on safety to protect the public against hazards should they explode.

Until now, however, high-strength and very tough steel suitable for more economical manufacture of pressure vessels could not be used, because the section titled ‘Materials’ under the existing standard (EN 13445) excluded such steel for pressure vessel application. The main objective of ECOPRESS was to find out whether these modern types of steel could live up to European standards.

An international consortium of seven European countries, made up of designers, pressure-vessel manufacturers and customers as well as plate material suppliers, research institutes and consultants experienced in structural safety, materials technology, modelling and Finite Element Method (FEM) calculations, came together to carry out the investigations. Altogether there were 17 partners, all members of the European Pressure Equipment Research Committee (EPERC, today called EPERC-TP). The Committee had laid the groundwork for this project by identifying the needs in a questionnaire circulated to the industry in 1998.

The project was coordinated by Ingenieurbüro für Werkstofftechnik (IWT) together with Rheinisch-Westfälische Technische Hochschule (RWTH) in Aachen, Germany. There were many other participants: Air Liquide, CMP Dunkerque, Industeel (Arcelor), and Guy Baylac from France; VTT and Rautaruukki from Finland; Dillinger Hütte, Professor Sedlacek & Partner Technologien im Bauwesen, and Staatliche Materialprüfungsanstalt (MPA Stuttgart) from Germany; the University of Patras in Greece; Steel and Metallic Materials Technological Centre (ITMA) and Felguera Caldereria Pesada (FCP) from Spain; AvestaPolarit, Kungliga Tekniska Högskolan (KTH) and Luleå Tekniska Universitet (LTU) from Sweden; and Exxon Chemical from the UK.

Page 51: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_51

www.i-w-t.de

The results were needed to determine which conditions could not be transgressed during use. A modified safety concept including the method to predict limit conditions such as brittle fracture and ductile fracture was successfully defined. The results are applicable for the design of pressure vessels using Design by Analysis (DBA), a calculation method provided for in the existing standard, which proved to be the most promising route for the application of high-tech steels for high-tech applications. The conventionally used method is Design by Formula (DBF), which limits the use of such steel enormously because of a high safety factor for strength of 2.4. The DBA-method uses a safety factor of 1.875, yielding benefits in stress utilisation or wall thickness reduction of up to 28 %.

CONCLUSIVE PROOF THAT MODERN STEEL IS SUITABLE

The research project took full advantage of a European network for pressure vessels within EPERC and of European research results on modelling limits for welded construction to show the potential of modern European steel where Europe is a leader in terms of technology, but application was impaired by conservative rules. The research proved conclusively that there are no real technical objections to using modern types of steel in pressure vessels, while establishing – from a technological standpoint – the state of current European knowledge concerning the true limitations of these steels and where care needs to be taken in selecting, welding and cold forming high-strength steel grades.

Not only is the steel industry as well as the pressure vessel branch now in a position to internationally compete with countries which already apply high-strength steel in pressure vessels, but an estimated 600 jobs were also safeguarded as a result of this project. Furthermore four PhD theses were completed during the project, helping young engineers to understand European strategies in an important industrial sector.

Within ECOPRESS, the basis was given for the extension of the selection method up to steel with a yield strength of 690 MPa (high strength steels), and the permission for use of Duplex stainless steels. Likewise, permission was given to adapt the latest fracture mechanics principle as developed through the following:

European Structural Integrity Assessment Procedure (SINTAP) under the ■

Fifth Framework Programme (FP5), Fitness-for-service thematic network (FITNET) under the Sixth ■

Framework Programme (FP6), verification of fracture behaviour of such steels based on results from ■

the European Coal and Steel Community (ECSC), today known as the Research Fund for Coal and Steel (RFCS),consideration of European experience in using modern steels in all kind ■

of applications.

The availability of EU funding to help with the research meant that it could be very broad-based and enhanced the credibility of the results when they were presented to international standard-setters. A project seminar was held and participation in international conferences and articles published in scientific journals helped to disseminate the results. The evidence that the results were convincing lies in the fact that the relevant European standard EN 13445 formulated within the CEN committee TC54 for unfired pressure vessels has been revised and takes the project’s results into account. (A CD-ROM containing the project results can be obtained from the Web address given above).

Page 52: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

52_Evimp-2

As supply chains within and across organisations become increasingly complex, sophisticated tools are necessary to help supply chain managers understand what products customers want, as well as where, when and how they want them. For example, a car manufacturer may want to know how and when best to fit orders for non-standard colours of cars into the production line and deliver them at the right time – i.e. not too early and not too late.

The LISCOS project has resulted in a superior tool to improve supply chain management. This new software was developed using a combination of what is known as Mixed Integer Programming (MIP) and Constraint Programming (CP) approaches. In technical terms, the project is based upon Branch-and-Cut and Constraint Programming, a combination that promises significant improvements in addressing a large variety of optimisation challenges and in providing better solutions for the management of supply chains overall.

APPLYING MATHEMATICS TO SUPPLY CHAIN PROBLEMS

In order to handle complex challenges or weak points that need to be optimised within a reasonable timeframe, the project has subdivided supply chain planning into two smaller domains: the ‘planning’ challenge and the ‘scheduling’ challenge. The planning challenge refers to high-level strategic planning decisions regarding which products to produce and where. The scheduling challenge, on the other hand, is more detailed and addresses aspects regarding when and how to make the products.

These challenges were overcome by different software applications using algorithmic technologies adopted by LISCOS. Planning challenges were addressed through mathematical optimisation, also known as Mixed Integer Programming (MIP), while scheduling challenges were generally addressed through Constraint Programming (CP). This led to a situation where different models have to be built and different technologies have to be used to solve the challenges.

With the approach described above, LISCOS was set to tackle the ‘real challenges’ that production companies face on the ground.

STRONG PARTNERSHIPS YIELD SOLID RESULTS

The LISCOS project has involved various important industrial partners that helped shape its development and progress. These were chemicals firm BASF (Germany), personal and household care product manufacturer Procter & Gamble Europe BVBA (Belgium), car manufacturer Peugeot Citroën Automobiles S.A. (France) and Diogo Barbot (Portugal), a small to medium-sized enterprise (SME) which produces paints and coatings. The academic partners in the consortium were Université Catholique de Louvain (Belgium) and Universidade de Lisboa (Portugal), both renowned in the MIP community. In addition, experts from the Université Henri Poincaré Nancy 1 (France) contributed to CP, as well as to the integration of CP and MIP.The consortium also included one company with expertise in MIP known as Dash Optimization, part of Fair Isaac (UK), as well as another company specialising in CP, COSYTEC S.A. (France).

IMPROVING PRODUCTIVITY WITH BETTER SUPPLY-CHAIN PLANNING SOFTWARE

Efficient management of the supply chain is a crucial issue for most companies. Having components in the right place at the right time, minimising stock levels and working out how to run at 100 % capacity even when order flows are erratic can improve production significantly and help create cost-competitive products. An efficient supply chain and production line also make it easier for companies to introduce new products and respond to variations in demand more quickly. Good supply chain management is a key success factor in meeting the challenges of the global economy, and thanks to the Large Scale Integrated Supply Chain Optimisation Software (LISCOS) project, it is now more efficient than ever.

LARGE SCALE INTEGRATED SUPPLY CHAIN OPTIMISATION SOFTwARE

LISCOS

Page 53: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_53

http://www.liscos.fc.ul.pt/http://www.dashoptimization.com/

http://www.artelys.com/

NEW SOFTWARE NOW COMMERCIALLY AVAILABLE

The core objectives of the project have been clearly achieved, resulting in two software products with the desired functionality in addition to other advantages.

The principle achievement is Xpress-Mosel, a software environment which includes a new high-level language for modelling and solving optimisation problems. This product not only integrates MIP and CP, but offers more optimisation strategies as well, if needed. Xpress-Mosel is currently part of Dash Optimization’s Xpress MP suite.

Xpress-CP represents another product that spun off from the project, but it has not been commercialised so far. Nonetheless, development has continued since the project’s completion. Encouraged by the successful pilot applications of the programming combination, Dash Optimization has rebuilt an integrated MIP-CP approach based on Artelys’ CP-framework Kalis. This new integrated MIP-CP-solver is called Xpress Kalis and can also be found in Dash Optimization’s Xpress MP suite.

The power of these tools has been proved in industrial case studies. By applying them within supply chain optimisation projects, the industrial partners were able to realise significant cost reductions.

THE WIDER BENEFITS

LISCOS has developed a generic approach which can be applied across various industries, and has had a broad impact on industry. Xpress Mosel has rapidly become a market leader in its field.

The new technology is useful to a wide range of industries, including manufacturers of machinery and equipment, food products and beverages, chemicals and chemical products and motor vehicles. The involvement in the consortium of software firms that are well established in this field also means that marketing of the new commercial products is assured by European companies likely to reap significant benefits in a market worth several billion euros. Lastly, there will also be indirect benefits to the environment from this project because greater supply chain efficiency generally means less waste and lower energy consumption.

Page 54: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

54_Evimp-2

The CRAFT project, code-named AMOR (Mechatronic Upgrade and Wheelchair Integration of the Manus Arm Service Manipulator), developed a second-generation wheelchair-mounted service manipulator or ‘arm’ to assist disabled people and improve the quality of their lives. The approach has been to completely redesign an existing system for better integration, effectiveness and control.

In more specific terms, the project objectives for the manipulator have been easier control through camera and what is technically known as ‘force feedback integration’. This was coupled with size, weight and noise reduction of the wheelchair, as well as improved lifting capacity (from 1.5 kilograms to 2.5 kilograms), development of a modular system and increased affordability overall. The service arm will also be much more accurate in moving and lifting, which, when coupled with other refinements of the new wheelchair design, will represent significant advancements in human-machine interfacing.

In a well-chosen consortium, four small and medium-sized enterprises (SMEs), five research partners and three associated partners achieved all the general objectives of the project. The main industrial, research and development (R&D) and exploitation partners included Exact Dynamics B.V. of the Netherlands and Hulpmiddelencentrale N.V. from Belgium. Another main industrial partner was De Koningh System Supplier B.V., also from the Netherlands. Additional projects have been established by the partners and commercialisation of the outcomes was planned starting 2008.

In order to succeed in this project, the improved design of the wheelchair arm required much R&D. It was tested by disabled individuals themselves, as well as by experts in relevant associations and a wheelchair manufacturer. The feedback was enthusiastic, leading to even further observations on refining the new technology. It is also noteworthy that the resulting arm or manipulator uses less material and is based on a “simple” design leading to some material savings as well.

Physically challenged individuals and their contributions represent a very important part of society, but their full potential hasn’t always been realised sometimes due to inadequate equipment such as wheelchairs and accompanying control devices. With significant ethical and societal implications, one enterprising project has chosen to tackle this issue by developing a second-generation service arm mounted on wheelchairs, which will render the lives of disabled people more comfortable, rewarding and productive.

MECHATRONIC UPGRADE AND wHEELCHAIR INTEGRATION OF THE MANUS ARM SERVICE MANIPULATOR

AMOR

Page 55: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_55

MULTIPLE BENEFITS FOR STAKEHOLDERS

The general objectives of the project where achieved, with some technical problems set to be solved in the near future. Two prototypes of wheelchair manipulator arms were developed, featuring lighter weight, more compact size and enhanced functionality. A new gripper design for the arm has also improved grasping, along with a camera for improved visual assistance. In addition, the newly developed wheelchair includes multiple mechanical interfaces for other equipment or devices, as well as improved graphical user interface (GUI) which has actually exceeded expectations.

Due to the small size of the companies involved, the expected improvement of the product will be of major importance for their economic growth. Sales will be supported by an already existing solid customer base. In the meantime, the results have been widely discussed and presented. Existing customers as well as the Internet-based forum will provide viable instruments for an ongoing dialog to promote the product and gather feedback for possible further refinements.

With respect to exploitation, because a previous generation of such products exists, marketing this new generation of wheelchairs and devices will be easier. Commercially, the manipulator will be less expensive and provide more flexibility and usefulness over previous technology, leading to savings and increased sales from a business perspective. Over the medium term, an increase of sales between EUR 500 000 and EUR 1 million per year is expected.

While this is not a development destined for the general public, the project will significantly contribute to a better quality of life and integration for disabled people. As a result, the project will also contribute to the reduction of

healthcare costs. Such an invention not only helps the physically challenged, it also raises the standards of living for those who provide care to the disabled, be they family members or caregivers. Furthermore, the newly developed devices are quieter than previous generations, rendering the immediate environment more pleasant for handicapped individuals and for people around them as well.

In all these respects, the impact on society can be considered important. Taking into account a market penetration timeframe, measurable results can be expected within 5 to 10 years

On project level, junior scientists and engineers had the chance to gain technical knowledge and improve their soft skills with respect to international collaboration. Overall, this collaborative work represents a very good CRAFT project with a strong ethical impact. Successful collaborative effor ts were noted also due to the SME-driven nature and a clear connection to progress in technology.

Page 56: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

56_Evimp-2

The CERHYSEP project is seen as a model of solid European cooperation with an outreach to a third-world country. Ten partners joined hands in CERHYSEP, hailing from one African and four European countries, with a good split among academia, research institutes and industry. Because many of the partners had already collaborated together before the project, the management of the project became much easier. The main industrial partner was Membraflow GmbH & Co KG Filtersysteme, and the main exploitation partner was Shell Global Solutions International B.V.

Developing membrane technology for hydrogen separation in industry means that the novel membranes should work under extreme conditions such as those that contain high levels of steam or under very high temperatures. Before the project, such membranes were not available, while technical progress in industry and the prospects for developing them was limited.

The main idea behind this project was to solve general problems in the chemical and petrochemical industries related to high energy and chemical consumption due to the limited capacity and efficiency of existing separation technologies. With the chemical industry aiming to reduce negative effects by improving the performance of traditional separation to its limits, the need to develop a novel membrane technology that overcomes current problems and allows cleaner and more cost-effective hydrogen production cannot be overstated.

MULTIPLE BENEFITS FOR PARTNERS AND SOCIETY

The implications of success are quite significant. With efficient membrane technology (e.g. membrane steam reformers), it may be possible to produce two pure product streams from natural gas: pure hydrogen on the ‘permeable side’ of the membrane, which can directly be used in fuel cells (such as in electric cars), and pure CO2 on the ‘retention side’ of the membrane, which can be utilised efficiently in applications involving mineralisation, enhanced oil recovery and coal bed methane.

Almost all the objectives of the project have been achieved. With respect to viable applications in more detail, Low Temperature Proton Conducting (LTPC) membranes have been developed by GKSS – Forschungszentrum Geesthacht GmbH in Germany. While their performance is lower than initially envisioned, their permeability and selectivity are quite good. The application of LTPC technology for gas-separating membranes is still unexplored and bodes well for further research. It presents significant promise as an alternative to current state-of-the-art membranes.

In a nutshell, technology for LTPC, organic composites and silica membranes has been developed to withstand temperatures up to 450° C. The objectives regarding issues of permeability and selectivity – integral aspects for superior membrane development – have also been achieved for the most part. With two patents issues, Membraflow has already introduced the new membranes into the market and could broaden its product portfolio. The actual benefits from the project for Membraflow at the time of writing amounted to EUR 6 million.

The petro-chemical and chemical industries will benefit the most from the project results as these industries can achieve higher cost-efficiency and improvements in CO2 emissions. The project results are of particular interest

With ecology and clean energy becoming critical issues for the planet, technology is shifting to create cleaner, more efficient energy solutions than ever before. The objectives of the CERHYSEP project have been to develop a novel membrane technology for cleaner and more cost-effective hydrogen production. Advanced membranes for hydrogen separation can be very useful in the chemical and petrochemical industries, among others, as well as in advancing fuel cells for electric cars.

CERAMIC MEMbRANES FOR HYDROGEN SEPARATION

CERHYSEP

Page 57: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_57

for the membrane producer Membraflow GmbH & Co KG Filtersysteme, which has increased the quality of its membranes and has been able to achieve a new variety, all of which are very popular on the market. Membraflow has also been able to adapt the technology for large-scale production. The actual benefits from the project for Membraflow already were valued at EUR 6 million when the project was evaluated.

A MUCH CLEANER FUTURE

Mew membrane technology can certainly help advance much cleaner and more cost-effective hydrogen production for a cleaner future. In this respect, the project can actually be considered an enabler for a hydrogen-based economy (fuel cells) with unparalleled impact on general air quality. In other words, an important advantage for a fuel-cell drive or hydrogen-based economy is that some of the materials developed in this project – namely, the low- and high-temperature proton conductors – can be used as highly efficient materials for fuel cells. If the hydrogen internal combustion engine (ICE) cars now polluting European cities are replaced by hydrogen fuel-cell cars, pollution in the cities would virtually come down to zero. While this is an unlikely scenario in the near future, research in this direction will certainly help in this respect and make it possible.

Environmental impacts were also addressed successfully through a reduction of energy consumption and CO2 emissions onsite at a refinery, thanks to the improved hydrogen permeability and selectivity of the developed membranes under processing conditions.

While the project goals have been met, more improvements with regard to stability, permeability and selectivity of the hydrogen selective inorganic membranes are required to achieve a profitable and reliable membrane reactor process. Shell and Membraflow – two of the main members in the

consortium – currently continue cooperative research works focussing on fuel cells technology based on own investments. Shell could have a return on investments in the fields of fuel cells technology in the next three years.

Lastly, it is worth noting that considerable knowledge transfer occurred between the University of Twente and Membraflow, as well as between the University of Western Cape and GKSS, furthering collaboration and progress in this direction immensely. The project has generally led to very strong relationships between all partners, with ongoing complimentary projects set up in the field of gas separation.

Page 58: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

58_Evimp-2

The European textile industry has increasingly shifted the focus of its activities from mass production to higher value products such as high-tech fabrics or high-end fashion garments. Increasing demand of high-end design products has led to fast changing requirements on textile products and spinning equipment.

Rapidly changing requirements and small lots are one of the European spinner’s main challenges, particularly since state-of-the-art spinning machines are high performance machines that are optimised for higher volumes. In general, adjustments have to be performed manually by specialists. Such adjustments must be identified through a trial and error process which is generally very time-consuming. This is especially true for draw frames (the quality filter of the spinning mill and the most important piece of equipment in the spinning process). Because draw frames have a strong impact on the efficiency of the spinning process, any maladjusted draw frame slivers (tooth-like combs that separate the yarn) will cause poor yarn quality and less efficient spinning.

A NEWER, MORE EFFICIENT SPINNING SYSTEM

The Intellidraw project has paved the way for more efficient, automated spinning machines by developing a self-regulating draw frame. The draw frame was the first machine of the spinning process flow considered for this project due to its high potential for improvement. The project intended to decrease the setting period for new bales of raw material dramatically and improve the competitiveness of the yarn manufacturer. The rationale was that such technology would be a unique selling point for a supplier of textile machinery based on new software, and therefore not easily copied by competitors.

The consortium for this project was put together by FEG Textiltechnik mbH from Germany, a small research institution with a dynamic profile. It brought together small and medium-sized enterprise (SME) spinning mills from Austria, Germany, Greece and the Netherlands to exchange knowledge and overcome challenges, together with industrial research contractor Truetzschler GmbH & Co. KG. Being small enterprises with their fair share of economic demands, the spinning mills would not have been able to manage such a project and apply the results on their own. They are in a position, however, to contribute with their application experience and to carry out intensive tests with the systems to be developed. The initiator, FEG, received the innovation award of the city of Aachen in 2004 for maintaining its position in the endangered textile industry through constant innovation, which Intellidraw contributed to.

Based on the user experiences of the involved spinning mills, a sophisticated system for assisting adjustment processes in draw frames has been developed. Truetzschler subsequently developed a draw frame with an automatic adjustment mechanism based on a new motor with direct drives instead of adjustable gears. The company then made the new self-adjusting draw frame commercially available on the market.

While the production of textiles is an endangered industry in Europe, the textile machinery industry still holds a leading position worldwide, thanks to innovations in new materials and advanced automation technology. Intellidraw is the perfect example of a project that furthers textile technology to ensure that Europe remains competitive in this domain.

INTELLIGENT DRAw PROCESS FOR HIGH QUALITY YARNS

INTELLIDRAW

Page 59: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_59

KEEPING EUROPE COMPETITIVE

All stakeholders reaped the rewards of this venture. To begin with, the project strengthened the position of FEG as a competent research partner within the textile machinery industry. For Truetzschler, the project and its research and technology development (RTD) efforts led to a new product, strengthening the company’s leading position in the production of machinery and systems within the textile domain. Today, already 10 % of the sales for drawing machinery are equipped with the newly developed automatic feature. This percentage is expected to increase to 30 % within the next few years.

In addition, spinning mills using the new equipment can handle faster changes of products and raw materials during production, which enables them to remain highly competitive. In combination with other advanced quality control systems, the mills can overcome economic challenges posed by global competition and meet increasing demands regarding flexibility and quality of very specific materials.

Although research for modernisation of the European textile industry cannot stop the general trend of transferring textile production to low-wage countries, it can certainly help maintain core textile production competences in Europe. This core competency represents the leading edge for the textile machinery industry, while supplier-user cooperation helps both parties advance future business and technology leadership, staying ahead of global competition.

Page 60: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

60_Evimp-2

Because of the high costs involved in recycling materials and disposing of coolants, the ultimate goal is to develop dry, fast machining. Under such conditions, the temperature of cutting tools and the wear-resistant coatings can easily reach 800° C or more.

The NACODRY project succeeded in improving the quality of the coatings in many ways. This included enhanced hardness, resistance against wear and cracking, adherence to the surface, stability in high temperatures, resistance to oxidation, low friction, compatibility with the material being machined and much more. NACODRY successfully developed new super-hard nanocomposites together with an appropriate coating technology that is thermodynamically driven to achieve improved cutting performance, especially under dry cutting conditions. There was particular focus on processing difficult-to-cut materials such as nickel alloys used for the aircraft and electronic industries.

REMARKABLE RESULTS

Due to the systematic and scientific approach, considerable improvements in nano-structured coatings were successfully achieved by the project. Different new coatings for various cutting conditions were successfully developed and tested under industrial conditions. In addition, improved deposition technology needed for mass applications was also developed.

The coated tools yielded a much improved process. To begin with, the cutting lifespan of the tools was increased considerably in various applications. In addition, difficult-to-cut materials manufactured by using the new coatings yielded remarkable cutting performance of the tools and much enhanced surfaces for the manufactured materials. The coated tools displayed excellent results under both dry and near-dry conditions, leading to much less use or even complete elimination of lubricants.

Important knowledge was also gained in terms of understanding physical phenomena taking place during the deposition process, while additional microanalysis was helpful in understanding certain processes during the stress tests of coatings. From a scientific perspective, more than 30 papers were published and several presentations were given on the subject.

EXPLOITATION AT ITS BEST

Two years after project-end, several of the developed coatings were successfully introduced to the market. These coatings are offered by PLATIT under the trademark nACo and by SHM under the trademark MARWIN, with slight differences in structure and composition. PLATIT also offers the coating unit (called pi80) optimised for depositing these nanocomposite coatings. Interestingly, this deposition unit has been specially designed for SMEs and for in-house coating use as well.

EUROPE LEADS IN HIGH-TECH MACHINING COATINGS

Machining refers to a process of mechanically cutting materials using power-driven machine tools with sharp cutting tools in order to achieve the desired geometry. The demand for improved machining technology has always been a goal of European industry in general, which requires machines that offer increased cutting speed, use less lubricants and coolants, and work more efficiently overall.

DRY STAMPING AND DRY MACHINING OF DIFFICULT-TO-CUT MATERIALS bY MEANS OF SUPER-HARD NANOCOMPOSITE COATINGS

NACODRY A DYNAMIC CONSORTIUM LEADING TO SUCCESS

Originally, the industrialisation of the superhard nanocomposites had been pioneered by SHM Ltd. (Czech Republic) in collaboration with the Technical University of Munich (TUM) before NACODRY was initiated. TUM had actually developed these nanocomposites between 1994 and 1996, launching their industrialisation together with SHM in 1996. But thanks to NACODRY, TUM and SHM had the opportunity to collaborate with other companies including PLATIT, taking the project from academia to market with outstanding results. To illustrate, SHM went from a very small and medium-sized enterprise (SME) of 5-7 people in 1996 to almost 60 employees today. Its turnover per year went from less than EUR 500 000 to at least 10 times that.

In the meantime, the new coatings themselves and corresponding deposition techniques were produced and developed by PLATIT Advanced Coating Systems (Switzerland), SHM and GENTA/Trattamenti Termici Ferioli e Gianotti SpA (Italy) which had also performed field tests of dyes and other cutting tools. Additional functional characterisations of the coatings as well as laboratory testing of cutting tools were performed by the National Research Council of Italy (ILM).

Two main end-users were also involved in the consortium (Gammastamp SpA and Centro Ricerche Fiat SCpA), undertaking considerable efforts in practical field tests under industrial conditions. The functional behaviour of the developed coatings and coated tools were checked and validated under conditions that mimicked mass production. GAMMASTAMP was especially active in what is termed the ‘chipless’ area of machining, such as stamping, with particular focus on the reduction or elimination of lubricants.

Page 61: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

Analysing the Outcomes of Ec Funded Projects under FP5_61

Overall, coatings and deposition units have certainly contributed significantly to the turnover for PLATIT and SHM, with more new product varieties on the way. The commercial impact of the results has been nothing short of excellent: SHM improved its coating-related turnover between 2002 and 2004 by around 40 %. In 2004, around 65 % of all sold coatings were MARWIN or related coatings, with turnover increasing steadily. In addition, the total number of employees increased by 25 % between 2002 and 2005 in this field.

Today, both PLATIT and SHM offer advanced coating versions based on the main results of the NACODRY project. In an exciting collaborative move, they founded a joint venture called PIVOT in the Czech Republic for various coating deposition services, mainly nanocomposite coatings. With the next generation of nanocomposite coatings and optimised deposition units developed and available, the consortium has successfully identified and met the demands and expectations of the manufacturing industry in this respect.

PIVOT was founded as a joint company of PLATIT and SHM in 2003, achieving turnover of around EUR 3.3 million in 2005. There are 55 employees currently working within PIVOT, which has produced and sold more than 120 Physical Vapour Deposition (PVD) coating units since the beginning of the joint business venture, with PVD units sold in over 22 countries. There are also currently six joint patent applications related to the technology, design and coatings.

Apart from both these end-users in the project consortium, other important companies are widely using the nanocomposite coatings in cutting and manufacturing. One example is AIRBUS, where the coated tools are currently used for cutting nickel alloys and other materials used in manufacturing the new A 380. Lastly, other industrial partners have also exploited the results and have profited from the project outcomes. GAMMASTAMP, for example, is now using these coatings and is able to reduce the use of lubricants.

SECRETS OF SUCCESS

There was a variety of factors from the onset which led to a very successful project. Besides the consortium members and the structure, the scientific and industrial partners have held a clear and comprehensive vision and approach on how to tackle all related challenges and to identify the real industrial needs. Because difficult-to-cut materials were chosen, industrial interest in further improvements of cutting these materials was considerable.

Before the actual project started, fundamental experiments were undertaken to ascertain that coatings with the desired functionality were possible. Three of the consortium partners have had long relationships with each other and had already jointly performed various research and development (R&D) activities. The R&D approach was very goal-oriented, supported by analyses and theoretical investigation to better understand the fundamental issues behind the deposition process. Field tests for depositing the coating and cutting test were performed under industrial conditions. The industrial partners exerted a lot of pressure on development and testing of tailor-made coating for their purposes. Ultimately, there was considerable effort to exploit and disseminate the results gained during the project.

The coatings fulfilled the requirements and partly exceeded the expectations. They were available shortly after project-end, while coating deposition units were developed in parallel. Lastly, even industries in Japan have tried to produce similar results but were not as successful, underlining the EU’s primacy in this domain.

Page 62: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations
Page 63: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

European Commission

EUR 24054 — EVIMP-2 : Analysing the outcomes of EC funded projects under fP5

Luxembourg: Office for Official Publications of the European Communities

2009 — 64 pp. — 29.7 x 21.0 cm

ISBN 978-92-79-13385-5doi 10.2777/50716

How to obtain EU publications

Publications for sale:• viaEUBookshop(http://bookshop.europa.eu);• fromyourbooksellerbyquotingthetitle,publisherand/orISBNnumber;• bycontactingoneofoursalesagentsdirectly.YoucanobtaintheircontactdetailsontheInternet(http://

bookshop.europa.eu)orbysendingafaxto+3522929-42758.

free publications:• viaEUBookshop(http://bookshop.europa.eu);• attheEuropeanCommission’srepresentationsordelegations.Youcanobtaintheircontactdetailsonthe

Internet(http://ec.europa.eu)orbysendingafaxto+3522929-42758.

Page 64: EVIMP-2OF EC FUNDED PROJECTS UNDER FP5 ANALYSING THE … · collecting, processing and analysing data to generate significant and meaningful results from the ex-post project evaluations

KI-N

A-24054-EN

-C

This brochure contains the results of the Evimp-2 evaluation study, which refers to the ‘Evaluation and long-term impact assessment of industrial research’. Evimp-2 is the most recent evaluation initiative within the European Commission (EC) of completed projects conducted under various industrial and technology programmes within the European Union (EU) since the early 1990s. The main goal of Evimp-2 is to identify if the Growth programme under the Fifth European Community Framework programme for Research (FP5) has produced the results and impact expected from it. Evimp-2 also examines if the programme was worth the investment and what could be improved in later Research and Technological Development (RTD) funding programmes. Two versions of this study are available online:

Evimp-2 Executive Summaryhttp://ec.europa.eu/research/industrial_technologies/impacts/list_findings-summary_en.html

Evimp-2 Main Reporthttp://ec.europa.eu/research/industrial_technologies/pdf/evimp2-mid-term-report_en.pdf