evaluating the quality and impact of community benefit ... · 9/27/2016 · the quality and impact...
TRANSCRIPT
![Page 1: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/1.jpg)
Community Benefit
Evaluating the Quality and Impact of Community Benefit Programs
September 27, 2016
Julie TrocchioSenior Director, Community Benefit and Continuing Care
The Catholic Health Association
Kimberly PeerenSenior Public Health Consultant
Healthy Communities Institute, a Xerox Company
![Page 2: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/2.jpg)
© Catholic Health Association of the United States
• Understand the mission and legal basis for evaluating
the quality and impact of community benefit programs.
• Learn how the CDC’s Framework for Program
Evaluation can be used to evaluate community health
improvement activities.
• Become familiar with tools and resources for
evaluating and reporting the impact of community
health improvement programs to help meet new IRS
requirements for tax-exempt hospitals.
2
Learning Objectives
![Page 3: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/3.jpg)
© Catholic Health Association of the United States 3
Resource: CHA Evaluation Guide
Source: https://www.chausa.org/communitybenefit/community-benefit
![Page 4: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/4.jpg)
© Catholic Health Association of the United States
To improve programs
To assess the organization’s impact
To ensure resources are being used wisely
To meet federal requirements
4
Why Evaluate?
![Page 5: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/5.jpg)
© Catholic Health Association of the United States
Federal Requirements
5
Source: https://www.federalregister.gov/articles/2014/12/31/2014-30525/additional-requirements-for-charitable-hospitals-community-health-needs-assessments-for-charitable
ImplementationStrategy
Community Health Needs Assessment
Include anticipated impact of activities
taken in response to CHNA
Report impact
![Page 6: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/6.jpg)
© Catholic Health Association of the United States 6
Community Benefit Cycle
Community Health Needs Assessment
Implementation / Program Planning
Implementation and Evaluation
Community Health Needs Assessment
REPORT EVALUATION
FINDINGS
EVALUATE PROGRESS
DESIGN
EVALUATION
IDENTIFY
PRIORITIES
![Page 7: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/7.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
7
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 8: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/8.jpg)
© Catholic Health Association of the United States 8
Resource: CDC Guide
Source: http://www.cdc.gov/eval/guide/index.htm
![Page 9: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/9.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
9
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 10: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/10.jpg)
© Catholic Health Association of the United States
• Who is involved in program operations?
• Who is served?
• Who will use findings?
10
Step 1. Engage Stakeholders
![Page 11: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/11.jpg)
© Catholic Health Association of the United States 11
Resource: HPOE Guide
Source: http://www.hpoe.org/resources/hpoehretaha-guides/2846
![Page 12: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/12.jpg)
© Catholic Health Association of the United States
• Decision-makers
• Program planners
• Individuals reporting to regulators
• Senior staff
• Board members
• Funders
12
Who Will Use the Findings?
![Page 13: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/13.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
13
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 14: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/14.jpg)
© Catholic Health Association of the United States
Step 2. Describe the Program
14
• Goal
• Resources to be used
• Activities
• Anticipated outcomes
![Page 15: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/15.jpg)
© Catholic Health Association of the United States
Inputs Outputs Outcomes
15
Program Mapping
ResourcesWhat we invest in our
programs
Activities + ParticipantsWhat we do and whom we reach through our
programs
ResultsWhat happens as a
result of our programs
![Page 16: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/16.jpg)
© Catholic Health Association of the United States 16
Simple Logic Model
INPUTS OUTPUTS OUTCOMES
Family & friends
Budget
Time
Backpacking gear & supplies
Prepare and pack
Drive to destination
Set up camp
Learn about nature; develop
backpacking skills; increase bonding among family and
friends!
Backpack & explore
![Page 17: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/17.jpg)
© Catholic Health Association of the United States 17
Resource: UWEX Logic Models
Source: http://www.uwex.edu/ces/lmcourse/
![Page 18: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/18.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
18
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 19: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/19.jpg)
© Catholic Health Association of the United States
• Determine evaluation purpose
– Program expansion/replication
– Funding
– Objectives
• Identify what program aspects to evaluate
• Identify what questions to ask
• Identify indicators for evaluation questions
• Design the evaluation
19
Step 3. Focus Evaluation Design
![Page 20: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/20.jpg)
© Catholic Health Association of the United States
• Determine evaluation purpose
• Identify what program aspects to evaluate
– Short, medium, long-term outcomes
– Cost effectiveness, etc.
• Identify what questions to ask
• Identify indicators for evaluation questions
• Design the evaluation
20
Step 3. Focus Evaluation Design
![Page 21: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/21.jpg)
© Catholic Health Association of the United States
• Determine evaluation purpose
• Identify what program aspects to evaluate
• Identify what questions to ask
• Identify indicators for evaluation questions
• Design the evaluation
21
Step 3. Focus Evaluation Design
![Page 22: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/22.jpg)
© Catholic Health Association of the United States
• How was the program carried out?
– Who did the program reach?
– How involved were participants in program activities?
– Were activities implemented as planned?
– How satisfied were participants?
– What else happened?
22
Process Questions
![Page 23: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/23.jpg)
© Catholic Health Association of the United States 23
Resource: Evaluation Briefs
Source: http://www.cdc.gov/healthyyouth/evaluation/pdf/brief4.pdf
![Page 24: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/24.jpg)
© Catholic Health Association of the United States
Impact Questions
24
• What changed?
– Did participants increase their knowledge or awareness?
– Did participants acquire new skills?
– Did participants change their behavior?
– Did health status improve?
– Did program achieve anticipated impact?
![Page 25: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/25.jpg)
© Catholic Health Association of the United States
• Determine evaluation purpose
• Identify what program aspects to evaluate
• Identify what questions to ask
• Identify indicators for evaluation questions
• Design the evaluation
25
Step 3. Focus Evaluation Design
![Page 26: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/26.jpg)
© Catholic Health Association of the United States
Evaluation Designs
• Post-test only
• Pre-test and post-test (single group)
• Pre-test and post-test with two or more groups
• Pre-test and post-test with control group
26
![Page 27: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/27.jpg)
© Catholic Health Association of the United States
Simple design; many factors can influence results and no other results to compare with
27
Post-Test Only
PROGRAM AFTER
![Page 28: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/28.jpg)
© Catholic Health Association of the United States 28
Pre-Test Post-Test (one group)
PROGRAMBEFORE AFTER
Can compare pre/post results of one group; must consider factors that can influence participants and results (e.g., participant selection,
external events)
![Page 29: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/29.jpg)
© Catholic Health Association of the United States 29
More than One Group
BEFORE AFTER
PROGRAM
BEFORE AFTER
Can compare different groups’ results; must consider factors that can influence participants, groups and results
NO PROGRAM OR DIFFERENT
PROGRAM
![Page 30: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/30.jpg)
© Catholic Health Association of the United States 30
Control Group
PROGRAM
NO PROGRAM OR DIFFERENT
PROGRAM
AFTER
AFTER
BEFORE
BEFORE
Groups with
similar
characteristics
Can compare results of similar groups receiving different interventions; consider group comparability and other factors that can influence results
![Page 31: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/31.jpg)
© Catholic Health Association of the United States 31
Resource: Community Toolbox
Source: http://ctb.ku.edu/en/table-of-contents/evaluate/evaluate-community-interventions
![Page 32: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/32.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
32
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
![Page 33: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/33.jpg)
© Catholic Health Association of the United States
• Quantitative data = numbers
– Counts, ratings, scores
– E.g. # of participants, survey scores
• Qualitative data = narrative
– Descriptions
– E.g. key informant interviews, observation studies
33
Types of Data
![Page 34: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/34.jpg)
© Catholic Health Association of the United States
Types of Data
PRIMARY DATA SECONDARY DATA
Original data collected for your own purposes
Data collected by someone else
• Surveys, questionnaires• Focus group discussions• Key informant interviews• Program records, existing
databases• Observation
• County Health Rankings• State and local health
departments• CMS• Small Area Health Insurance
Estimates• USDA Food Environment Atlas,
etc.
![Page 35: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/35.jpg)
© Catholic Health Association of the United States
• Have a plan for:
– Questions to be answered
– Data sources for answers
– Method of collecting data
– Timing
– Who is responsible
35
Step 4. Gather Credible Evidence
![Page 36: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/36.jpg)
© Catholic Health Association of the United States 36
Data Collection Plan
Source: http://www.cdc.gov/wisewoman/evaluation_toolkit.htm
![Page 37: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/37.jpg)
© Catholic Health Association of the United States 37
Resource: WISEWOMAN Toolkit
Source: http://www.cdc.gov/wisewoman/evaluation_toolkit.htm
![Page 38: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/38.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
38
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 39: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/39.jpg)
© Catholic Health Association of the United States
Step 5. Justify Conclusions
39
• Analyze and interpret results
• Examine patterns
• Consider factors that may have influenced results (e.g., external events)
![Page 40: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/40.jpg)
© Catholic Health Association of the United States
Step 5. Justify Conclusions
40
• Process:
– Was the strategy implemented correctly?
– Did any external events intervene?
– If program was evidence-based, was it faithful to the model?
![Page 41: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/41.jpg)
© Catholic Health Association of the United States
Step 5. Justify Conclusions
41
• Impact:
– Did program participants change?
– Were the changes measurable?
– Were the changes likely because of the program?
![Page 42: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/42.jpg)
© Catholic Health Association of the United States
Resources: Data Analysis
42
• CDC’s WISEWOMAN Evaluation Toolkit
• CDC Evaluation Briefs:
– Analyzing Qualitative Data: http://www.cdc.gov/healthyyouth/evaluation/pdf/brief19.pdf
– Analyzing Quantitative Data: http://www.cdc.gov/healthyyouth/evaluation/pdf/brief20.pdf
![Page 43: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/43.jpg)
© Catholic Health Association of the United States
Look At Your Evaluation Findings
43
• What does information suggest about program?
• Was program carried out as planned?
• Any surprising information?
• What lessons were learned?
• Positive findings? Negative findings?
![Page 44: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/44.jpg)
© Catholic Health Association of the United States
CDC Evaluation Framework
44
PLAN
1. Engage stakeholders
2. Describe the program
3. Focus evaluation design
IMPLEMENT
4. Gather credible evidence
ANALYZE AND USE FINDINGS
5. Justify conclusions
6. Ensure use and share lessons learned
1. Engage Stakeholders
2. Describe the Program
3. Focus Evaluation Design
4. Gather Credible Evidence
5. Justify Conclusions
6. Ensure Use and Share Lessons
Learned
Source: http://www.cdc.gov/eval/framework/
Standards• Utility• Feasibility• Propriety• Accuracy
![Page 45: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/45.jpg)
© Catholic Health Association of the United States
• Program participants
• Program managers, staff
• Funders
• Hospital leaders
• Community members
• Policy makers
45
Audiences to Share Findings With
![Page 46: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/46.jpg)
© Catholic Health Association of the United States
Federal Requirements
46
Source: https://www.federalregister.gov/articles/2014/12/31/2014-30525/additional-requirements-for-charitable-hospitals-community-health-needs-assessments-for-charitable
ImplementationStrategy
Community Health Needs Assessment
Include anticipated impact of activities
taken in response to CHNA
Report impact
![Page 47: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/47.jpg)
© Catholic Health Association of the United States
• Improve program
• Identify resources needed
• Make decisions about program
• Replicate/expand program
• Disseminate information to those who might be interested in adopting program
47
Step 6. Ensure Use
![Page 48: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/48.jpg)
© Catholic Health Association of the United States 48
Resource: BetterEvaluation
Source: BetterEvaluation - Report and Support Use of Findings (May 2013) http://betterevaluation.org
![Page 49: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/49.jpg)
© Catholic Health Association of the United States 49
Resource: HPOE Guide
Source: http://www.hpoe.org/resources/hpoehretaha-guides/2858
![Page 50: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/50.jpg)
© Catholic Health Association of the United States 50
Resource: CDC CHI Navigator
Source: http://www.cdc.gov/chinav/
![Page 51: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/51.jpg)
© Catholic Health Association of the United States 51
Resource: CHA Evaluation Guide
Source: https://www.chausa.org/communitybenefit/community-benefit
![Page 52: Evaluating the Quality and Impact of Community Benefit ... · 9/27/2016 · the quality and impact of community benefit programs. •Learn how the CDC’s Framework for Program Evaluation](https://reader033.vdocuments.us/reader033/viewer/2022050217/5f62e3ab92dedf6b171afd5c/html5/thumbnails/52.jpg)
© Catholic Health Association of the United States
Thanks for participating in this webinar!
52
Questions?