national child nutrition program cluster evaluation: preliminary findings evaluation team colin bell...
TRANSCRIPT
National Child Nutrition Program Cluster Evaluation: Preliminary Findings
Evaluation team
Colin Bell
Jan Garrard
Dionne Holland
Boyd Swinburn
Deakin University 2003
Background: National Child Nutrition Program
• A community grants program established in 1999 by Commonwealth Department of Health and Ageing
• Aimed at improving nutrition among children aged 0 to 12 years and pregnant women
• Focus on priority communities:• Rural and remote communities• Aboriginal and Torres Strait Islander communities• Lower socio-economic communities
• 110 community-based projects funded nationally • 18 projects in Victoria - $2,190,300 over three year
period
Deakin University 2003
Evaluation: Background
• Evaluation not built into overall program planning • No specific evaluation budget• Importance of evaluation recognised by Victorian
office of Dept of Health & Ageing• 9 projects committed $3,000 each to fund a
‘cluster’ evaluation• Additional $60,000 from Dept of Health & Ageing• Deakin University contracted to conduct the
evaluation
Deakin University 2003
Evaluation: Preview
• A ‘retrofitted’ evaluation strategy• Optimum balance of common measures while
acknowledging the uniqueness of projects, populations and settings
• Evidence base for effective community-based child nutrition programs is poor – good practice often not documented, disseminated and acknowledged
• A practical, feasible evaluation strategy designed to meet the needs of funding body, project management agencies and practitioners
Deakin University 2003
Evaluation: Objectives
• To assess intended and unintended impacts of individual projects
• To gain an understanding of key implementation issues impacting on project success
• To draw upon collective learnings from the 9 projects to guide future efforts in community-based child nutrition
Deakin University 2003
Evaluation: Design & Methods
• Multiple case study design (9 ‘cases’)• Process measures (ie. what have they done?) and • Impact measures (ie. what have they achieved?) • At individual project level - evaluation team assisted
each project to conduct process and impact evaluation• Across projects, evaluation team conducted a total of
53 key informant interviews – combination of core questions and project-specific questions
• Enabled ‘detection’ (but not quantitative measurement) of impacts – useful for unintended impacts
• Provided data on important how? and why? questions
Deakin University 2003
Evaluation: Lessons Learned
• Build evaluation planning into project planning• Provide adequate resources for evaluation
(approx 10-15% of project budget)• Recognise that responsive, needs-based,
individually tailored community-based projects will rarely meet the requirements for rigorous quantitative assessment
• Such projects rarely find their way into conventional evidence-based reviews.
Deakin University 2003
Lessons Learned (cont)
• Focus instead on what can be achieved (measured impacts) in what settings with what population groups under what conditions – recognise this will vary in different priority settings
• Distil common achievements and learnings if possible
• Provide a mechanism for sharing experiences via a ‘community-based projects’ evidence-base.
Deakin University 2003
Cluster Evaluation Projects
• ‘High Five’ School project – Warrnambool• ‘Eat Well, Grow Well’ – Hamilton, Southern
Grampians Shire• H2O for Moonee Valley Kids – North Melbourne• ‘Breast is Best’ – Brimbank, West Melbourne• ‘Healthy Eating for Country Kids’ – Swan Hill• ‘Who’s Eating Gilbert Grape and Vicki Vegetable’ –
Murray Valley Aboriginal Cooperative, Robinvale• ‘A healthy family is built on a good diet of nutritious
food’ - Robinvale• ‘Child nutrition in the West’ – West Melbourne• ‘Food Facts for preschoolers’ - KPV
Deakin University 2003
Aim: To increase the fruit and vegetable intake in order to improve the overall nutrition of prep to year 4 children in the South West of Victoria
Children with fruit in lunchbox
0%
20%
40%
60%
80%
100%
School1 School2 School3 AVERAGE
PRE
POST 1
POST 2
• ↑largest in yr 2&3
• Also ↑ in serves
• Vegetable intake was very low (~5%) & only increased at one of the schools
• Supporting curriculum changes and parent involvement
South West Health Care
Deakin University 2003
Aim:To increase the proportion and duration of breastfed babies in the City of Brimbank, particularly in low SES and CALD communities
• Distributed BF promotion kits to GPs & MCHNs
• Prenatal BF promotion checklist adopted by Sunshine Hospital
• Breastfeeding guide for new mothers (Vietnamese)
• Specific support and promotion of BF in Vietnamese community
• 66 BF-friendly venues• Adoption of Lactation policy by
ISIS
Change in Brimbank BF rates 00/01 to 01/02
0
1
2
3
4
5
Discharge 2 weeks 3 months 6 months
Per
cent
Cha
nge
01/02 75% 69% 51% 34%
ISIS Primary Care
Deakin University 2003
Murray Valley Aboriginal Co-Op
• Murray Valley River Quest
• PAN messages at ‘stops’ on route
• Reached 516 children
• Teachers said it was easy to implement and raised awareness
• In 2003 hours have ↑
Murray Valley Aboriginal Cooperative
Hours walked as a class on MRQ 2002
0
5
10
15
20
25
30
35
School1 School2 School3
Hou
rs w
alke
d Average
Prep
Grade 1/2
Grade 3/4
Grade 5/6
48 towns53 towns100 towns
Aim: Improve nutritional knowledge and healthy eating patterns of pregnant women, young mothers/caregivers of children aged 0-12 years and children within local Aboriginal and Torres Strait Islanders
Deakin University 2003
Western District Health Service
Increases in knowledge after education sessions
0 20 40 60 80 100
Toddler SM tour
Toddler food talk
Intro solids
Healthy eating & BF
Healthy eating & preg
Percent
Pre
Post
• Education sessions effective in improving knowledge of participants
• Supermarket tours and visits to new mums also beneficial
• ↑awareness of conditions which may require dietetic referral among WDHS staff
Eat Well – Grow Well
Aim: To improve the nutritional health of children aged 0-5 years in the Southern Grampians Shire
Deakin University 2003
Swan Hill District Hospital
• ↑access to nutrition education for pregnant/new mums
• Good readership of newsletter and newspaper column
• These were helpful for making healthy food choices
Evaluation of newsletter & newspaper column (n=125)
0 20 40 60 80 100
Agree/SA news helps with healthy food choices
Occ/Always read newspaper column
Agree/SA tips help with healthy food choices
Agree/SA tips useful
Read tips in newsletter
Percent
Aim: To improve the eating habits of all children and pregnant women in Swan Hill and surrounding district, providing GPs with nutrition information kit
Deakin University 2003
Doutta Galla Community Health Service
• ‘H2O for Moonee Valley Kids’• Drink water rather than
sweet drinks
• ‘Food and Families’• 4 week course on
nutrition information and parents skills
• 10-14 Somali women
• Breastfeeding, bottle, solids, vitamin D
Sweet Drink Consumption
0
20
40
60
80
100
Moonee Valley Colac
Aim: To improve the healthy food intake of infants and children aged 0-12 years with Kensington, Flemington, North Melbourne and Ascot Vale
Deakin University 2003
Western Region Health Centre
Fruit in Lunchboxes
0
20
40
60
80
100
Prep Grade 1/2 Grade 3/4 Grade 5/6
Mar-01 Nov-02 Mar-03
• ‘Eating altogether-5 times better’• Fruit, vegetable & water
promotion at 2 primary schools
• Bilingual child education program• In 1 year, 700 people from LD
communities participated in 75 nutrition education sessions
Western Region Health Centre
Aim: Increase nutritional knowledge and capacity of pregnant women, parents and children ages 0-12 with CALD and recently arrived communities, to enhance healthy eating patterns
Deakin University 2003
Robinvale District Health Service
At the MCCD I Learnt How To ...
0
20
40
60
SA S Neither
A/DA
DA SD NR
Cook healthy food Give healthy food to family
Make favorite food healthier Make food from different cultures
• Parent cooking classes• Multi-cultural food day
• Attended by 120-180 people
• Nutrition education sessions
• Large Polynesian population
• Breakfast program at schools
• Linked with MVAC
Aim: Improve nutritional knowledge and healthy eating patterns of parents and children in Robinvale
Deakin University 2003
Kindergarten Parents Victoria
• Staff training sessions• 9 locations throughout Victoria with over 170 participants• ↑ knowledge of Aust. Dietary Guidelines for children, role of nutrition
policy, value of partnerships b/w parents and early childhood staff
• Parent workshops• At sessional preschools, long day care, other services• ↑ understanding of childhood nutrition
• Resources• Staff training sessions, ‘parent workshop’ package, Food Safety
Information Sheet, nutrition built into existing KPV resources (eg Preschool Matters’
Aim: To increase the capacity of early childhood services to deliver consistent, relevant and current nutrition information to preschool children and their parents
Deakin University 2003
Cluster Evaluation
• 53 semi-structured key informant interviews completed• 11 project officer interviews• 39 key informant interviews (single and focus groups)• 2 Steering Committee member interviews
• Combination of core and project specific questions
• Presenting preliminary findings today• Report available early 2004
Deakin University 2003
Reported Impacts
Increased collaborative links
Human services NGOs Govt Depts
• Community
Health Centres
• PCPs
• Royal Children’s
Hospital
• Breast Feeding
Association
• Ecumenical
Migration Centre
• Dept. of Health
and Ageing
• LGAs
• City West Water
Deakin University 2003
Reported Impacts (cont)
Diffusion of nutrition messages• Reached target groups
• Children, parents, carers, pre-schools, kindergartens, primary schools
• And beyond…..• Program participants ‘spread the message’
• Fathers read brochures on breastfeeding aimed at mothers
• Program participants ‘modelled’ healthy eating behaviours
Deakin University 2003
Reported Impacts (cont)
Spin-offs• Children
• Increased concentration in class• Likely reduction in dental carries
• Families• Budget savings (eg. not purchasing soft drink)
• Bilingual workers• Future work• Empowerment
Deakin University 2003
Capacity Building Framework
Organisational Development
Workforce development
Resource allocation
Partnerships
Leadership
Context
Build Capacity
• Infrastructure
• Program sustainability
• Problem solving
Source: Hawe 1999
Deakin University 2003
Organisational Development
• Management support• Projects more effective when management
understood the complexity of community work and supported it
• Recognition of work• Projects better supported within the organisation
when encouraged to showcase their achievements• Additional support from co-workers
• Policy development
Deakin University 2003
Workforce Development
• Project officer training and previous experience in community work
• Needed a sound knowledge of community work and how it differs from clinical work
• Generally, new graduates who lacked field experience struggled unless provided with additional support within the workplace
• Projects easier to implement when working in a team environment
• Important to share learnings with co-workers to gain support and increase organisational capacity to carry out community work
Deakin University 2003
Resource Allocation
• Control of project resources (incl budget)
• Advice on how to get the ‘best bang for your buck’• Project officers need to be aware existing resources
• ‘Cost-effectiveness’ of different strategies
• Continual planning towards obtaining additional resources, especially into the future
Deakin University 2003
Partnerships
• Can be extremely effective• Working towards a common goal, but from varied view
points• Provided additional ‘sponsorship’• Often opened up opportunities for shared access to experts
and resources
• Enhanced opportunities for sustainability as multiple organisations/individuals were able to share the role lessening the burden (time and money) on one person
• Often enabled strategic responses across multiple settings
Deakin University 2003
Leadership
• Essential
• Creative problem solving skills• Needs to be able to take on the view points and the
needs of the local community and develop and re-develop strategies where needed
• Continuity• Leadership role best held by the same person
throughout the project
• Empowerment of others
Deakin University 2003
Context
• Physical, economic, political, organisational and cultural environments of the project• Currently high profile of nutrition• Organisations with community-based and
population focus
Deakin University 2003
What does the literature say about success factors for community-based interventions?
• Limited literature around community-based child nutrition programs
• Community-based smoking programs have been at it a lot longer!
• Systematic review of 32 studies covering 30 years of community-based interventions to reduce smoking (Secker-Walker et al. 2003)
Deakin University 2003
Lessons for child nutrition promotion(Adapted from Secker-Walker et al. 2003)
• Child nutrition needs to be recognised by the community as an important issue
• Community organisation, assessment of community capacity, and the identification of individuals and organisations interested in supporting nutrition interventions are important during project development.
• Community members and staff need skills in working with diverse groups and in health education
• Coalitions need several months to form, and a year or more to become effective change agents in their community
Deakin University 2003
• Interventions to improve child nutrition need to continue for several years
• Resources for promoting good nutrition and support for healthy eating should be readily available throughout the community
• It is important to involve, train, and provide referral services and resources for health professionals
• The use of mass media (print, radio, television) is especially useful for modelling behaviour change, and for changing community norms about nutrition
Lessons for child nutrition promotion (cont)
Deakin University 2003
Conclusion
• Projects have had some good impacts at the local level
• Contributed to the knowledge base about effective implementation
• Important to disseminate these findings