criterion 1 excellence, critical aspects of evaluated ...horizon 2020 6 evaluation criterion 1:...

CAMPUS BIO-MEDICO UNIVERSITY OF ROME

Via Álvaro del Portillo, 21 - 00128 Rome - Italy

www.unicampus.it

Criterion 1 Excellence, critical

aspects of evaluated proposals

and main strengths of a

successful proposal

Loredana Zollo

Laboratory of Biomedical Robotics and

Biomicrosystems

A Step closer to success in

FET Open

Workshop 23 Novembre 2016

Via Cavour 71, Roma

FET-Open – Novel ideas for radically new technologies


www.unicampus.it

• Experience in H2020 – Fet Open

• What to know when writing a proposal

• Evaluation Criterion 1 “Eccellence” and sub-

criteria

• Structuring Sect. 1 “Excellence” of FET Open

proposals

• Interpretation of Scores

• Main weaknesses of FET proposals

• Conclusions

Outline


www.unicampus.it HORIZON 2020

3

Experience in H2020 – Fet Open

Selected as expert in 2014 and Served for two calls

H2020-FETOPEN-2014-2015-RIA call with the cut-off on 31st of March 2015

▪ Number of proposals: 670

▪ Number of funded proposals: 11 (1.6%)

H2020-FETOPEN-2014-2015-RIA call with the cut-off on 30 September 2014

▪ Number of proposals: 644;

▪ Number of funded proposals: 24 (3.7%).

Selected as Vice-chair in 2015

H2020-FETOPEN-2014-2015-RIA call with the cut-off on 30th of September 2015

▪ Number of proposals: 800

▪ Number of funded proposals: 11 (1.4%)


www.unicampus.it

THINK AS AN EVALUATOR WHEN

WRITING A PROPOSAL



5

Evaluation Criterion 1

Excellence: it is the main criterion for the evaluation with a threshold of 4 out of 5 and a relative weight of 60%! There are four sub-criteria:

1. Clarity and novelty of long-term vision, and ambition and concreteness of the targeted breakthrough towards that vision.

2. Novelty, non-incrementality and plausibility of the proposed research for achieving the targeted breakthrough and its foundational character.

3. Appropriateness of the research methodology and its suitability to address high scientific and technological risks.

4. Range and added value from interdisciplinarity, including measures for exchange, cross-fertilisation and synergy.



6

Evaluation Criterion 1: Excellence

A good proposal clearly presents the breakthrough and the rationale behind the proposed idea. Analogously, the specific objectives are well posed, with clear progressive investigations that pave the way to science and technological advancement in the long-term.

Look at the substance: Some proposals might

be handicapped by language difficulties, others

deceptively well written



7

Kipling Method



8

Why this project?

Key scientific question

Unexplored or poorly explored topic

High risk - High gain

Clear description of objectives and experimental steps

(list of objectives and sub-objectives)

Feasibility

Preliminary results/evidence and expertise for each objective (possibly)

Expected milestones

Possible (international) collaborations

Such aspects can mostly be traced in the first section of the part B of the

proposals Excellence:

1.1 Long-term vision and targeted breakthrough towards that vision

1.2 Novelty, non-incrementality, plausibility and foundational character

1.3 Research methodology

1.4 Interdisciplinarity



9


General Objectives

Long term; beyond the duration of the project

Improve, strenght, facilitate, realize…

Specific Objectives

To be realized during the project implementation

Testing, pilot plant, develop new knowledge,…

The objectives should be consistent with the expected impact of the project and coherent with the implementation.



10

Describe the long-term vision of the proposal.

Describe the targeted scientific breakthrough to be achieved by the proposal.

Describe how this targeted breakthrough represents an ambitious and concrete step towards the long-term vision of the proposal.

Describe the objectives for the project, which should be specific, measurable, achievable, relevant, and time-bound within the duration of the project.




11

1.2 Novelty, non-incrementality, plausibility and foundational character

• Discuss the relevant state-of-the-art. Describe the advance

your proposal would provide beyond the state-of-the-art, and to what extent the proposed research is based on cutting edge knowledge, new ideas and concepts.

• Demonstrate the plausibility of the proposed research for achieving the targeted breakthrough.

• Provide arguments for the foundational nature of the breakthrough, in the sense that, if achieved, it would establish an essential basis for a new line of scientific and/or technological research and its future uses, not currently anticipated.

Keep in mind that new ideas and concepts are expected, rather than

the application or incremental refinement of well established ones.



12

Describe the overall research approach, the methodology and explain its relevance to the proposal's objectives.

Explain how the research methodology is suitable to address high scientific and technological risks, allowing the exploration of alternative directions and options.

Where relevant, describe how sex and/or gender analysis is taken into account in the project’s content

1.3 Research methodology



13

Describe the research disciplines supporting the targeted breakthrough of the proposal.

Explain the added value from the interdisciplinarity.

Describe the measures for exchange, mutual learning, cross-fertilization and synergistic advances among involved disciplines in order to explore new areas of investigation and directions for joint research.

1.4 Interdisciplinarity



14

Interpretation of the scores

The proposal fails to address the criterion or cannot be assessed due to missing or incomplete information.

Poor. The criterion is inadequately addressed, or there are serious inherent weaknesses.

Fair. The proposal broadly addresses the criterion, but there are significant weaknesses.

Good. The proposal addresses the criterion well, but a number of shortcomings are present.

Very Good. The proposal addresses the criterion very well, but a small number of shortcomings are present.

Excellent. The proposal successfully addresses all relevant aspects of the criterion. Any shortcomings are minor.

0

1

2

3

4

5



15

Main weaknesses

Excellence:

• Novelty: the proposed vision does not appear really novel; it seems more an incremental refinement of already ongoing activities and existing technologies.

• Targeted breakthrough: the targeted breakthrough is often not coherent with the research methods and the rationale is poorly described.

• Ambition: The proposal lacks a clear overview of the state-of-the-art in all the involved disciplines. The provided novelty and correspondingly state of the art is limited to one specific field (typically the field of expertise of the Coordinator)

• The foundational character (i.e. the potential to become the basis for a new line of technology not currently available) is weakly addressed.

• High-risk: The proposal fails to address the high-risk vision



16

Conclusions

The Evaluation Process

Thought to be fair

Designed by humans, carried out by humans, so always improvable

The number of submitted proposals is very high and, consequently, competition is tight.

The assignment of individual evaluators is crucial, each one with his own background, and none of them is immune from errors

Receipt of proposals

Individual evaluation

Consensus group

Panel Review Finalisation

Evaluators Evaluators / Vice Chairs

Consensus Report

automatically generated



17

Conclusions

Preparing a proposal in 5 easy steps

1. Ask yourSelf if Horizon 2020 is the right fit

for you

2. Get intimate with your call

3. Create the best consortium you can possibly

imagine

4. Put together your application from the point

of view of the evaluator

5. Submit your Proposal early and often

If the idea is good and the background is rational and solid resubmission can be an option. Don’t give up!


www.unicampus.it

THANK YOU !

criterion 1 excellence, critical aspects of evaluated ...horizon 2020 6 evaluation criterion 1:...

Documents