![Page 1: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/1.jpg)
OPEN Evaluation study: Design and preliminary results of
the baseline measurements
Lisbon, November 26th 2014
![Page 2: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/2.jpg)
To appraise the methodology of community-
based programmes-targeting childhood obesity
prevention across Europe- based on EPODE
methodology
Aim
![Page 3: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/3.jpg)
Objectives
1. To identify the strengths and weaknesses of the
community-based programmes in reference to the
four EPODE pillars
2. To identify potential improvements of the
programmes’ weaknesses after the OPEN interventions
![Page 4: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/4.jpg)
Methods
![Page 5: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/5.jpg)
Design• 2-year follow-up study
– Baseline measurements: June-September 2014
– Final measurements: May-July 2016 (after the OPEN
trainings)
• Descriptive research
– Qualitative, through interviews and questionnaire
– Quantitative scoring of the qualitative data
![Page 6: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/6.jpg)
Study population• Principal programme coordinators and project
managers (national and/or local level)
• 13 European programmes targeting childhood obesity prevention– Community Based programmes (CBPs), Initiatives
(CBIs) and Public Organizations, willing to implement sustainable strategies and actions to prevent obesity at local and national level
![Page 7: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/7.jpg)
The OPEN Participant programmes
Child Health programmePaideitrofi
SETS
Keep Fit!
Ex-IDEFICSPartille
JoggViasanoEpode
Fladre-Lys
Salud MadridMunsi
Sporttube
Health Promotion and disease Directorate
Ex-IDEFICS Delmenhorst
![Page 8: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/8.jpg)
Data collection (1)“Good practice appraisal tool for obesity prevention programmes, projects, initiatives and interventions”:– Developed by WHO– Self-administrated– Assessment of
I. main intervention characteristicsII. monitoring and evaluationIII. implementation
– Data reviewed for missing information-discussed with the programme teams
![Page 9: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/9.jpg)
Data collection (2)today’s results
Interview guide:– Semi-structured interview guide, flexible to additional
information– In-person interviews– Assessment of
I. General organisation II. Political commitment III. Public-Private partnershipsIV. Communication and PRV. Scientific aspects and dissemination
– Proteins team and the VU team
![Page 10: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/10.jpg)
Data analysesInterview guide
1. Semi-transcription2. Operationalization by Proteins team3. Data editing and scoring by two VU
researchers; • Eventually by Proteins team
• Scoring (0-2) conducted in reference to the EPODE pillars
![Page 11: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/11.jpg)
Preliminary Results
![Page 12: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/12.jpg)
Descriptive characteristicsProgramme/Organisation,country
Programme range (region/city)
Year of initiation Final target group (s) Communities/cities/towns reached (n)
People reached (n)
Child health programme, Cyprus
Regional (Nicosia) 1995 Children and their families 8 towns (municipalities)
4.500 children and families (incl. 1000 of the control area)
Salud Madrid, Spain Regional (Madrid) N/A Children and adolescents 0-17 years old
179 1.185.156 children and adolescents
Delmehorst research Institute, Germany
Local (Delmenhorst) 2008 Children from 2-10 years old and their families
1 city 45.000 (targeted)
EPODE Falndre Lys, France Regional (Community of Municipalities French Flanders - Lys)
2004 3 months to 11 years old 8 towns 34.000 people of which 7000 children
JOGG (Youngsters at a Healthy Weight), The Netherlands
National 2010 Families with children 0-19 years old
62 1.000.000 (300.000 children)
Keep fit, Poland National –school based 2006 Adolescents from 13-15 years old 60% of the secondary schools in the country
700.000/year
Health Promotion and disease Directorate, Malta
National N/A Whole population approach N/A N/A
MUNSI, Portugal National 2007 (school-based pilot)
Children and teachers 1
N/A
PAIDEIATROFI, Greece National 2008 Families with children 6-12 years old
6 N/A
IDEFICS Partille, Sweden Local (Partille) Children from 2-10 years old 1 7.000 children
SETS movement, Romania National 2011 Children from 6-12 years old 3 94.000
Sporttube, Slovakia National 2009 Children from 6-19 years old 200 schools 53.000 children
VIASANO, Belgium National 2007 children 3-12 and their families 18 towns 700.000 inhabitants
![Page 13: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/13.jpg)
Descriptive characteristics• 6 EPODE-like programmes (46%)
• Level of implementation
₋ 8 at the National level, including a central and a local coordination;
₋ 3 regionally
₋ 2 locally (ex-IDEFICS)
• Final target groups:
₋ families and children
₋ five programmes include adolescent population
• Children reached: from 7.000 to 300.000
• Range:
₋ 1 to 62 cities
₋ School approach to community approach
![Page 14: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/14.jpg)
Political commitment
Formal agreement Type of contribution Advocacy0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes
NoIn kind Financial
Enabling programme
Active
PassiveNo
![Page 15: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/15.jpg)
Public-Private Partnerships (PPPs)
PPPs PPP charter Private Interference0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pub-lic/non-profit
Yes
No
None
occa
siona
l
No
Some conditions
Public & Private
stru
ctur
al
![Page 16: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/16.jpg)
Social Marketing
Regular CBI
Multiple target g
roups*
Target gro
up analysis
Variaty of to
ols
Intervention mix
Environmental ch
ange
Diversity of to
pics0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%Yes
Yes
Yes
Yes
Yes
Yes
Yes
*3 or more of the following target groups: children 0-3, children 4-12, adolescents, parents, intermediate target groups (e.g. teachers), local stakeholders
![Page 17: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/17.jpg)
Scientific aspects and evaluation
Scientific b
oard
Validati
on of tools
Evaluati
on framework
Type of e
valuati
on
Behavioural
chan
ge
BMI
Budget
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
5-10%Individual experts
None
No None
Process & effect
No
No
Process or effect
No
No
Yes
Yes
Yes
Yes
Yes
![Page 18: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/18.jpg)
Dissemination and communication
0%10%20%30%40%50%60%70%80%90%
100%
No
YesYes
Yes
No
No Social media
LittleMinimal
PoorNot a specific person
Yes
Good
NoNo
![Page 19: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/19.jpg)
Discussion
![Page 20: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/20.jpg)
Variability between the programmes
1. Community-based programmes VS Community-based
approaches (strategies, initiatives)
2. One setting VS multiple settings targeted
3. Various communities VS one targeted
4. Independent VS less independent communities
5. National level VS local level
![Page 21: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/21.jpg)
Political commitment
• Best case-scenario– Formal agreement– In kind and financial contribution– Active advocacy
• Possible improvements– Increase contribution which is minimal in most
cases
![Page 22: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/22.jpg)
PPPs• Best case scenario for PPPs– PPP charter– Inclusion of a variety of public and private
partners– No interference of private partners in context
• Possible improvements– 30% of the programmes need a PPP charter– Inclusion of more private partners
![Page 23: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/23.jpg)
Social Marketing• Best case-scenario– Regular target group analyses– Environmental and behavioural change– Multiple target groups in different levels– Intervention and marketing mix
• Possible improvements– Adapt target group analyses– Focus on environmental change– Whole community approach– Improve intervention and marketing mix (variety of topics,
tools, activities)
![Page 24: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/24.jpg)
Scientific aspects and evaluation• Best case-scenario– Experts gathered in a board– Expert validation of tools and intervention contents– Programme evaluation framework – Process and effect evaluation of the interventions– At least 10% of the budget allocated to evaluation
• Possible improvements– In all the above stated
![Page 25: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/25.jpg)
Dissemination and communication
• Best case-scenario– Political and Scientific spokesperson-clearly identified– Dissemination of results in communities, stakeholders,
partners, scientific conferences/events– Expert in communication– Use of available communication channels– Communication plan
• Possible improvements– In all the above stated
![Page 26: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/26.jpg)
Strengths and limitations
Strengths• In-person interviews• 2 researchers for data appraisal (eventually 3)• Use of two assessment tools
Limitations• Information mostly on national level of coordination• Scoring categories are arbitrary; though constructed mainly
by the Proteins team
![Page 27: OPEN Evaluation study: Design and preliminary results of the baseline measurements Lisbon, November 26 th 2014](https://reader030.vdocuments.us/reader030/viewer/2022032600/56649d945503460f94a7c560/html5/thumbnails/27.jpg)
Conclusions
• Scoring dependent on the context; context of
implementation varies between programmes
• Different aspects to be improved in every
programme
• Tailored trainings for each programme