evaluating physical activity intervention programs thomas schmid, phd physical activity and health...
TRANSCRIPT
![Page 1: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/1.jpg)
Evaluating Physical Activity Intervention Programs
Thomas Schmid, PhDPhysical Activity and Health Branch
CDCAtlanta, Georgia, USA
![Page 2: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/2.jpg)
Evaluation Definitions and Principles
CDC Physical Activity Evaluation Handbook – 6 Steps
Evaluation issues
Overview
![Page 3: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/3.jpg)
The systematic examination and assessment of features of a program and its effects. Evaluation is designed to produce information that can be used by those who have an interest in the improvement or effectiveness of an initiative or program.
What is Evaluation?
![Page 4: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/4.jpg)
To assess the effects and value of a program
◦for accountability◦for progress ◦to identify best practice◦as a planning tool◦for research
Why Evaluate?
![Page 5: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/5.jpg)
the program target groups and needs realistic achievable objectives is the program ready for evaluation?
Match Evaluation of Health Promotion Programs to:
![Page 6: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/6.jpg)
Identify needs
State goals and measurable objectives
Assess progress towards objectives
Process of conducting evaluation
![Page 7: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/7.jpg)
EfficacyIntervention tested under ideal conditions with optimal study designEffectivenesField based intervention testing whether the intervention works as intended for a defined population in the real worldEfficiencyResults in comparison to the efforts required to obtain them – often cost or cost effectiveness
Evaluation Terminology
![Page 8: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/8.jpg)
EVALUATION
IMPACT
OUTCOME
Measures activities, program quality, reach
Measures long term effects (goals)
PROCESS
Formative Measures short term objectives
Develop program & resources
![Page 9: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/9.jpg)
What is likely to work Best practices in health promotion Idea or concept development Developing and pilot testing materials Focus groups
Formative Evaluation
![Page 10: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/10.jpg)
Formative Evaluation
Focus groups and interviews with partners about their needs
Reviewing best practices from other programs such as Agita Sao Paulo
Pilot testing in one company
![Page 11: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/11.jpg)
Process evaluation◦Program implemented as intended ?
◦ Quality of program components
◦Attendance rates
◦Audience awareness of message
◦ Appropriate message learned
◦Participant satisfaction
![Page 12: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/12.jpg)
Process Evaluation
Companies enrolled
Workshops held
Promoters trained
Companies participating in community events
![Page 13: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/13.jpg)
Impact & Outcome Evaluation
◦ Short and long term effects of program
◦ Hierarchy of effects : what level of effects indicates program success
◦ Assessing the impact upon those who attend or participate
◦ Community effects
![Page 14: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/14.jpg)
Impact & Outcome Evaluation
Physical activity level of employees
PA level in community
Disease rates or costs
![Page 15: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/15.jpg)
Program Evaluation FrameworkPhysical Activity Evaluation Handbook
![Page 16: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/16.jpg)
Program Evaluation FrameworkPhysical Activity Evaluation Handbook
1. Engage Stakeholders1. Engage Stakeholders
Include people with a vested interest in the program: implementers, partners, participants, possible opponents and decision makers.
2. Clearly Describe the ProgramThe problem being addressed, descriptions of the planning, implementation and maintenance
phases of the program, and finally, defining and communicating how all phases work together (eg. the "logic model").
![Page 17: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/17.jpg)
Program Evaluation FrameworkPhysical Activity Evaluation Handbook
3. Clearly Focus3. Clearly Focus
Asking a clear, straightforward question and ensuring linkage to the problem being addressed. Includes clarifying process and/or outcome evaluations of interest
4. Develop and Implement Data CollectionDesign of data collection efforts, selection of
appropriate control groups, designation of appropriate baseline values and decisions about how much time should be built in to adequately measure "change”
![Page 18: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/18.jpg)
Program Evaluation FrameworkPhysical Activity Evaluation Handbook
5. Develop Conclusions5. Develop Conclusions
Data analyses and interpretation.
6. CommunicatePromotes the importance of tangible products from the evaluation, dissemination of those products, and follow-up (as appropriate).
![Page 19: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/19.jpg)
www.iuhpe.org
![Page 20: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/20.jpg)
20
![Page 21: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/21.jpg)
All physical activity interventions should be evaluated. Evaluation begins with program planning
There are many good programs, but very few are well evaluated
Involve stakeholders Community – University partnerships can be
very useful
Observations on Evaluation
![Page 22: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/22.jpg)
Process as well as outcome
evaluation Quantitative and qualitative Tailor evaluation to stakeholders and
situation Evaluation is often undervalued
compared with “hard science” Adequate resources should be
devoted to evaluation. 10% is recommended
Observations on Evaluation
![Page 23: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/23.jpg)
Evaluation results should be used to
adjust programs and inform policy Share results with stakeholders Academic and practical training are
needed The CDC Physical Activity Evaluation
Handbook provides a good framework
Observations on Evaluation
![Page 24: Evaluating Physical Activity Intervention Programs Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA](https://reader036.vdocuments.us/reader036/viewer/2022062308/56649d9c5503460f94a84634/html5/thumbnails/24.jpg)
Gracias/Thank you