EU Joint Programme - Neurodegenerative Diseases Research (JPND)

Developing a framework for monitoring and evaluation

Joint Programming Conference 2013

Dublin, 28 February & 1 March 2013, Parallel Session 8Edvard Beem, Co-director ZonMw, member of the Executive Board of JPND

This presentation

• Rationales and objectives of JPND

• Development of a monitoring and evaluation framework• Logical Framework Analysis as a systemic tool• Overview of the defined set of indicators of performance

• Execution of monitoring to date

• Key issues, challenges and lessons learned

0

10

20

30

40

60 70 80 90 100Age (years)

Cas

es /

100

Stroke

Dementia

Parkinson's

From Rocca W et al, Eurodem

Alzheimer’s Disease in Europe

Number of cases

Societal costs (€)

2010 6,000,000 72,000,000,000

2040 12,000,000 144,000,000,000

A major societal challenge for the coming years

A major societal « Grand Challenge »

Energy

Neurodegeneration

Climate Change

Food & Health

Scientific• Animal models• Biobanks• Cohorts/registries• Disease pathology

Scientific• Animal models• Biobanks• Cohorts/registries• Disease pathology

Social• Health care delivery• Home automation• Health economics• Ethics

Social• Health care delivery• Home automation• Health economics• Ethics

Medical• Early diagnosis• Prevention• Clinical trials

Medical• Early diagnosis• Prevention• Clinical trials

Focus on Three Domains

Scope of JPND

Diseases TargetedAlzheimer’s Disease and other dementiasMotor Neurone DiseasePrion Disease

Parkinson’s Disease & related disorders Huntington’s DiseaseSpinocerebellar Ataxia (SCA)Spinal Muscular atrophy (SMA)

What are the JPND goals?

• Increase coordinated investment in ND research aimed at finding causes of disease, developing cures, and identifying appropriate ways to care for those with ND

• Our immediate goals:• Launch activities that add value to national research efforts• Leverage additional resources through partnerships • Link and support national plans with relevance for ND

Why JPI?

• Awareness• common challenge, common vision

• Mapping• Inventory running national programmes (database)• Overlaps and gaps, opportunities for joint actions• Baseline for monitoring

• SRA (>10 years)• Roadmap for EU-wide investment

• Implementation plan phase 1 2012-14• Joint actions: calls, enabling activities, action groups

CALLS Total commitment 2011-2014• Pilot Joint Transnational Call 2011 = €15M

• CoEN program call 2011 (€ 4M) and calls 2012-2014 (€12M)

• Joint Transnational Call 2012• “identification of genetic, epigenetic and environmental risk and protective

factors” = €19M• “evaluation of health care policy, strategies and interventions = € 10M

• Estimate commitment 2013 = €25M• Estimate commitment 2014 = €25M

• Total projected investment: over €100M in cash contribution (no in-kind resources included)

Enabling activities

• Knowing our research capability

• Opportunities for infrastructure and platforms

• Working in partnership with industry

• Working with the regulators• Global partnership (third

country collaboration)

• Capacity building

• Education & training

• Connection to policy makers

• Communication and outreach

JPND will execute the SRA within the next ten years through a combination of:

Action groups• Action Groups to determine transnational research needs and

opportunities in specific scientific areas• Longitudinal Cohort Studies (disease- & population-based)• Animal and Cell Models• Assisted Living Technologies

• Action Groups to promote engagement, commitment and partnerships• Engagement and Partnership with Industry• Engagement and Partnership with the EC and beyond• User and Public Involvement in ND Research• Alignment of National Plans and activities

Why monitoring and evaluation?

Critical questions

• Does a co-ordinated EU-wide approach makes a difference achieving the outcomes?

• Will the outcomes of a concerted approach be more than the sum of the outcomes of national approaches?

• Can we reduce fragmentation?

• Can we recognize and learn from successes and failures?

Monitoring and evaluation framework• Analysis of the links between (societal) challenges,

objectives and activities of a programme or initiative related to the expected effects and impacts

• By mapping the intervention logic it will be easier to assess whether the objectives have been met and what the success criteria are

• However it is “not as easy as it looks”• Different objectives and effects• Changing objectives• Unexpected effects

Logical Framework Analysis (LFA)

The logic:

• A public action is undertaken for a reason (rationale)

• It has objectives which address (societal) needs

• It provides inputs which lead to activities

• These activities create (direct) outputs

• Which on their lead to outcomes/results and finally to (long-term) impacts

Intervention logic

Programmelogic

Inputs Activities

Outputs

Outcomes or results

Long-termImpacts

Rationale or reasons

(Societal) challenges

Result chain

Operational goals

Specificgoals

Mission

Intervention

logic

Logical Framework of JPND

Indicators of performance

• Based on the Logical Framework Analyse, indicators of performance have been developed:• Type A indicators monitor the effect of JPND on (European)

research programming, research policy and funding (the concept of Joint Programming)

• Type B indicators monitor the scientific and societal impact of research on neurodegenerative diseases conducted in the context of JPND

• Distinction is made between:• Input indicators• Output indicators• Outcome indicators• Impact indicators

Development of indicators

Examples of type A indicators

• Input indicators• Attitude towards JPND goals and objectives• The entry or drop out of countries

• Output indicators• The national research (funding) priorities based on JPND’s Research

Strategy• Outcome indicators

• The amount of JPND common research funding for neurodegenerativediseases as share of total EU research funding

• Impact indicators• The number of publications in high impact journals of European

researchers in neurodegenerative diseases research

Examples of type B indicators

• Input indicators• The number of collaborative research projects funded through JPND• The number of non-project funded activities

• Output indicators• The number of Europe-wide population-based studies with contribution

of JPND• Outcome indicators

• Regular interactions between JPND and stakeholder groups • Impact indicators

• Exchange of practices across different types of research (basic,clinical and healthcare)

Execution of monitoring to date1. Monitoring reports based on indicators

• Quantitative indicators based on desk study, information made available by other work packages

• Qualitative indicators obtained through a questionnaire investigating the attitudes towards JPND:• Expectations regarding JPND• Opinions on current JPND activities• Views on the future of JPND

1. External review of experts• Review focusing on JPND’s governance and and processes of

strategic policy programming (strategic direction) that have led to the development of the Research Strategy (SRA) and the implementation plan

Key issues, challenges and lessons learned

• Special attention should be paid to monitoring and evaluation as an important element to show progress and results of the initiative to various audiences

• Development of monitoring and evaluation framework should be aligned with the development of the Research Strategy

• Monitoring and evaluation is NOT just listing a set of indicators and start measuring, but takes some time and effort to develop according to intended objectives and achievements of the initiative

• The systematic use of a Logical Framework Analysis approach is considered to be very helpful in (re)defining the objectives of JPND

More information

• The full version of the monitoring and evaluation framework to be found on the website of JPND:

http://www.jpnd.eu

The team responsible for monitoring and evaluation of JPND:

• Edvard Beem beem@zonmw.nl

• Bastian Mostert bastian.mostert@technopolis-group.com