fps health, food chain safety and environment 1 joint action on health workforce planning and...
TRANSCRIPT
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
1 Joint Action on Health Workforce Planning and
Forecasting
M.Vallimies-Patomaki, J.Lammintakanen
Brussels – April 12, 2013
Work Package 3
Finland
Evaluation of the Joint Action
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Agenda of the WP3 kick-off meeting
• Introduction of WP3 partners
• Scope and objectives of WP3
• Deliverables
• Preparation of the evaluation strategy
• Discussion topics
2
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Core team
Dr Marjukka Vallimies-Patomäki, WP leader, Ministry of Social Affairs and Health, Finland
Mr Jesmond Sharples, WP Co-leader, Ministry of Health, Malta
Prof. Juha Kinnunen, Senior Management Scientist, University of Eastern Finland, Finland
Prof. Johanna Lammintakanen, Management Scientist, University of Eastern Finland, Finland
3
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Partners• DG SANCO D2/EU, EAHC/EU
• Ministry of Social Affairs and Health/FI, Ministry of Health/MT, Ministry of Public Health/BE, Ministry of Health/PL, Ministry of Health/SK
• University of Eastern Finland/FI, Medical University of Varna/BG, NCPHA/BG, University of Bremen/DE, Semmelweis University/HU
• CfWI/UK
• EFN, EHMA
4
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Scope and objectives WP3 provides a systematic approach for the internal evaluation of the Joint Action
Aim is to verify if the JA is being implemented as planned and achieving the objectives:
- output evaluation is to verify compliance to achievement of the objectives
- outcome evaluation is to assess usefulness of outcomes, recommendations and sustainability beyond the Joint Action
- process evaluation is to ensure compliance to the rules and implementation
5
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Engagement of the member states and stakeholders
1.Mentoring group:
- WP leaders and representatives of stakeholders
- Discussing and giving feedback of the evaluation strategy
2.Expert reference groups:
- Representatives of member states and stakeholders
- Discussing and appraising the quality of the deliverables
3.Focus groups:
- Representatives of member states and stakeholders
- Appraising the impact and sustainability of the JA
6
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Deliverables
Evaluation strategy and evaluation tools, August 2013
Expert reference group report, November 2014
Intermediate evaluation report, January 2015
Expert reference group report, December 2015
Final evaluation report, March 2016
7
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Schedule for the preparation of the evaluation strategy• 1 st draft by April 30/Commission & WP1
•Basic contents and evaluation indicators•Comments by May 22
• Evaluation indicators and spesific objectives: consultation through teleconference/Commission, WP1 &WP leaders
• 2 nd draft by June 7 /Commission, WP1 & WP leaders•Comments by June 19(Dates changed by WP3)
• Stakeholder analysis, in collaboration with WP2 (> into an appendix)
• 3 rd version by June 30/Commission, WP1, WP leaders, partners, strategic stakeholders
•Comments by July 31
• Final version to WP1 by August to be submitted to EB
8
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Materials and methods
9
Approach
Material Method Evaluation Tools
Output evaluation
Deliverables / WP2,4,5,6 and 7
• Review process / Executive Board, MO• Expert reference group
Quality criteria for deliverables; semi-structured
Outcome evaluation
Changes in the HWF capacities / WP6Deliverables / WP2 and 7
• Content analysis / In collaboration with WP6• Delphi rounds/ Focus groups
Criteria for comparing the Cprofiles to updated Cprofiles
Interview themes
Process evaluation
Revisor´s reports / WP1Progress reports / MO and WPs
• Content analysis Mainly structured criteria for reports
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
1. Introduction
The aim of the evaluation (WP leader)• Some words on the whole EUHWForce project• Role, aims and tasks of evaluation in relation to JA
2. Evaluation Team
Members and division of work (WP leader)• Short introduction of members and areas of expertise• Responsibilities
3. Evaluation reports and timeline
Description of each evaluation deliverable• Interim evaluation report• Final evaluation report Data and methods used (J.Lammintakanen)• Documents (stage plans), formal reports• Mentoring group• Focus groups• Expert reference groupsTimeline (in relation to GA)Report objectives
4. Ev. criteria and tools
WP2, 4 - WP7, indicators (In collaboration with Commission, WP1, WP leaders, University of Bremen etc.)
5. Risk analysis
E.g. variety of laguages, cultural manners, unrealialistic milestones, conflicting interests
10
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Review of spesific objectivesSpesific
•Spesify the target group and the factors that need to be
changed
Measurable•Formulate objectives in a measurable format
Appropriate•Make sure objectives are acchievable, attainable, acceptable for the target group
Realistic• Ensure that you can realisticly acchieve the objectives
given the project resources
Timebound•Connect objectives to a timeline
11
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Key performance indicators• Ensure that WP leaders are involved in identifying KPIs for all
WPs of the JA
• KPI help the JA define and measure progress toward desired outcomes
• KPIs are quantifiable (measurable) measurements, agreed to beforehand, that reflect the critical success factors of the JA
School: graduation rates of the students
Social service organization: the number of clients assisted during the year
Source: www.management.about.com.
12
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Overview of the discussion topicsEvaluation tools:- Find examples and experiences
Spesific objectives and evaluation indicators : Review in collaboration with Commision, WP1 and WP leaders
Stakeholder analysis: In collaboration with WP2
Preparation of the evaluation strategy: In collaboration with Commission, WP1, WP leaders, University
of Bremen etc.
13
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Evaluation process: in co-operation with all WPs
14
Process evaluation (+ WP 1)Compliance to the rules
Output evaluation (WP2,4,5,6,7)Compliance to the content
Outcome evaluation (+WP2,6,7)Lessons learnt and recommendations
ACHIEVEMENT OF THE JA OBJECTIVES, IMPLEMENTATIONAND SUSTAINABILITY OF THE JA2016
Each WP’s objectives
Deliverable: Final report (> EB)
2013:SETTING OBJECTIVES FOR THE JA AND WPs”…to create a platform for collaboration and exchange between MSs to better prepere for the future of the health workforce”
Interim report > EB
FPS HEALTH, FOOD CHAIN SAFETY AND ENVIRONMENT
Thank you for your attention !15