lessons learned evaluating a complex public health program · lorena disha, m.p.h. learning...
TRANSCRIPT
![Page 1: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/1.jpg)
C
LESSONS LEARNED EVALUATING A COMPLEX PUBLIC HEALTH PROGRAM
Lorena Disha, M.P.H.
![Page 2: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/2.jpg)
Learning Objectives
• Lessons learned and best practices from evaluating a complex, multi-level, multi-
program project
• Evaluation Toolkit
• For beginners
![Page 3: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/3.jpg)
Getting to the Heart of the Matter in Michigan
Program
Getting to the Heart of the Matter in Michigan
Building Healthy
Communities
Diabetes
Oral Health
Tobacco
Adverse Childhood
Experiences (ACEs)
Wise Choices
Worksite Wellness
• Michigan Department of Health and Human Services, Division of Chronic Disease and Injury Control.
• Chronic disease rates are higher in those areas of Michigan where residents don’t have many opportunities to make healthier lifestyle choices. Consequently, there are higher rates of diabetes, obesity, heart disease, and cancer, among others.
• Seven program areas within the Division joined to create this new initiative to have a collective impact on increasing healthy lifestyles, decreasing tobacco use, and decreasing obesity among high-risk, vulnerable populations.
• Funded by the Centers for Disease Control and Prevention
![Page 4: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/4.jpg)
• Implementation
• Year 1
implementation –
9 months
• Year 2 – 12
months
• Local level: one
rural and one
urban
• Antrim, Otsego,
Charlevoix,
Emmet,
Washtenaw,
Western Wayne
• Expanding
geographically in
Year 2
![Page 5: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/5.jpg)
Start with the end in mind
![Page 6: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/6.jpg)
Types of Evaluations
• Process
• Outcomes
• Qualitative
• Quantitative
• Formative
• Summative
![Page 7: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/7.jpg)
Which Type Should YOU Use?
• When making your decision, take into consideration some of the following conditions:
• Timeframe of the program
• Who your stakeholders are
![Page 8: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/8.jpg)
Stakeholders
• Who are your stakeholders?
• What will they do?
• What do they need to implement the program?
• Engage your stakeholders
![Page 9: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/9.jpg)
SMART Objectives
• Acronym*
• Specific – well-defined
• Measurable – quantities
• Achievable – feasible
• Realistic
• Time – Related
*CDC definition
![Page 10: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/10.jpg)
Logic Model
INPUTS ACTIVITIES OUTPUTS OUTCOMES
Staff
Funding
Materials
What you are doing to
contribute to
accomplishment of
outcomes
Deliverables,
what your funder
will collect
Short – about
1 year
Intermediate – 1
– 3 years
Long – 3+ years
![Page 11: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/11.jpg)
Data Collection and Analysis • Types of data collection methods
• Surveys
• Questionnaires
• Interviews
• Websites
• Methodology
• Clearly define all of your terms
• Create a glossary of these definitions
• Define your parameters
• Disseminate data collection tool to partners
• Provide training and meetings
• Provide technical assistance
• Analysis
• Excel
• SPSS
• SAS
• NVivo
![Page 12: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/12.jpg)
Work Plan
• Objectives
• Activities
• Performance Measure
• Timeframe
• Responsible Staff
• Budget
![Page 13: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/13.jpg)
Back to Your Stakeholders!
• Share your findings and results with your partners, staff, program, leadership, public,
etc.
![Page 14: Lessons learned evaluating a complex public health program · Lorena Disha, M.P.H. Learning Objectives •Lessons learned and best practices from evaluating a complex, multi-level,](https://reader035.vdocuments.us/reader035/viewer/2022063004/5f92b58dab138a4f796c5253/html5/thumbnails/14.jpg)
Toolkit
1. Start with the end in mind
2. Evaluation methodology
3. Engage stakeholders
4. SMART objectives
5. Logic model
6. Data methodology
7. Work plan
8. Data collection and analysis
9. Stakeholders