improving the practice and performance of public health departments: results from the evaluation of...
TRANSCRIPT
Improving the Practice and Performance of Public Health Departments: Results from the Evaluation of CDC’s National
Public Health Improvement Initiative (NPHII)
Cassandra Martin Frazier, MPHApplied Systems Research and Evaluation Branch
Division of Public Health Performance ImprovementOffice for State, Tribal, Local and Territorial Support
Keeneland Conference, April 22, 2015
Centers for Disease Control and Prevention
Office for State, Tribal, Local and Territorial Support
Objectives
Describe progress toward achieving outcomes of increased accreditation readiness, quality improvement, and performance management capacity
Illustrate the reach of NPHII program funding and support to help improve public health practice and performance
Discuss plans to sustain current accreditation readiness, quality improvement, and performance management activities
NPHII: The Program at a Glance
Advance public health departments’ readiness for accreditation
Improve organizational efficiency and effectiveness through quality improvement activities
Increase performance management capacity
Supported through the Prevention and Public Health Fund of the Affordable Care Act
FY2010: $42.5M to 76 awardees
FY2011: $33.5M to 74 awardees
FY2012: $33.5M to 73 awardees
FY2013: $32.5M to 73 awardees
NPHII Evaluation Approach Purpose: extent to which NPHII supported improved
public health practice and performance through Increased readiness for accreditation Improved efficiencies/effectiveness Increased performance management capacity
Design: non-experimental program evaluation Multi-strand, sequential mixed methods Quantitative data: annual surveys,* progress reports Qualitative data: stakeholder interviews,* focus groups*
*OMB No. 0920-0879
Data Sources for PresentationData Source Program Progress Reports
Description Semi-annual reports submitted by awardees detailing progress toward completing required program activities
Response Rate Year 3 IPR—n=73 (100%)Year 3 APR—n=73 (100%)Year 4 IPR—n=72 (100%)Year 4 APR—n=68 (94%)
Limitations Self-reported dataLimited validationQuestion revisions over time
Analysis Cross-sectional, frequencies—Year 4 APRCross-time, aggregate—all sources (for select measures)
ACCREDITATION READINESS
Completed all PHAB prerequisites within
past five years* (n=67)
54%
58%
76%81%
Completed PHAB prerequi-site within the past five
years*
Public Health Accreditation Board (PHAB)Prerequisites
* Year 4 APR
S1.1 S1.2 S1.3 S1.4 S2.1 S2.2 S2.3 S2.4 S3.1 S3.2 S4.1 S4.2 S5.1 S5.2 S5.3 S5.4
71%65%
71%
61%
69%65% 66% 65%
77%
61%
81%
74% 74%
55%60%
74%
For domains 1–5, at least 55% of awardees that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards*
*Year 3 IPR and APR; Year 4 IPR and APR
Organizational Self-Assessment Against PHAB Standards and Measures
S1.1 S1.2 S1.3 S1.4 S2.1 S2.2 S2.3 S2.4 S3.1 S3.2 S4.1 S4.2 S5.1 S5.2 S5.3 S5.4
71%65%
71%
61%
69%65% 66% 65%
77%
61%
81%
74% 74%
55%60%
74%
Organizational Self-Assessment Against PHAB Standards and Measures
S6.1 S6.2 S6.3 S7.1 S7.2 S8.1 S8.2 S9.1 S9.2 S10.1 S10.2 S11.1 S11.2 S12.1 S12.2 S12.3
69%
76%
65%
55% 55%61%
39% 37%
45%
73%
66%
52%
77%
68%
61% 61%
At least half of awardees (52%) that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards for domains 6, 7, 10, 11, 12*
*Year 3 IPR and APR; Year 4 IPR and APR
S6.1 S6.2 S6.3 S7.1 S7.2 S8.1 S8.2 S9.1 S9.2 S10.1 S10.2 S11.1 S11.2 S12.1 S12.2 S12.3
69%
76%
65%
55% 55%61%
39% 37%
45%
73%
66%
52%
77%
68%
61% 61%
QUALITY IMPROVEMENT
Improvements Made in Efficiency
78%
46%
17%12%
7% 5%
Overall, 41 out of 72 awardees (57%) increased their program and/or organizational efficiency through quality improvement initiatives*
*Year 3 APR; Year 4 IPR and APR
Improvements Made in Effectiveness
Quality
Enhan
cemen
t, Serv
ices
Quality
Enhan
cemen
t, Sys
tems
Customer
/ Staff
Satisfa
ction
Increas
ed Rea
ch to
Targe
t Populati
on
Organiza
tional Desig
n Impro
vemen
ts
Dissem
ination of In
form
ation
Increas
ed Prev
entative B
ehav
iorsOth
er
Decreas
ed In
ciden
ce / P
revale
nce
46%
39%35% 33%
25%
18% 16%12%
9%
Overall, 57 out of 72 awardees (79%) increased their program and/or organizational effectiveness through quality improvement initiatives*
*Year 3 APR; Year 4 IPR and APR
REACH OF NPHII SUPPORT
Support to Other Public Health Agencies
0 100 200 300 400 500 600 700 800
697
434
433
199
144
68
Received Mini-grants Received TA, Training, Workshops, Other
Accreditation Readiness
Performance ImprovementQuality Improvement
Performance Management
Number of Public Health Agencies Receiving Support
Non-awardee agencies received indirect support to conduct NPHII-related activities; 52% of awardees used NPHII funds to provide this support (n=68)*
* Year 4 APR
INFRASTRUCTURE AND SUSTAINABILITY OF PERFORMANCE IMPROVEMENT ACTIVITIES
Performance Management CapacityMore than half of awardees established at least one of the four performance management system components required by NPHII* (n=66)
36%
67% 62% 59% 56%
* Year 4 APR
Sustaining Staff/Performance Improvement Efforts
Performance Improvement Manager (n=67)
Performance Improvement Office (n=55)
Quality Improvement activities (n=67)
Accreditation Readiness Activities (n=67)
Performance Management Activities (n=67)
63%
75%
72%
75%
78%
Most awardees plan to sustain their NPHII-related activities and performance improvement office*
* Year 4 APR
Summary Awardees increased their readiness for accreditation
Most established key documents and plans for their organization and partners
Most moved toward meeting standards necessary for accreditation
Awardees demonstrated more efficient processes that save time, and enhanced the quality of their services
Awardees plan to continue working on or maintaining these activities
NPHII not only supported its direct awardees, but reached agencies within their jurisdictions
Acknowledgments Centers for Disease Control and Prevention
Anita McLees Andrea Young Craig Thomas Garry Lowry Saira Nawaz*
Laura Hsu*
NPHII Program Leadership and Staff National Network of Public Health Institutes
Nikki Rider Sarah McKasson Brittany Bickford Sarah Gillen
NPHII awardees* Former CDC contractor and evaluation
fellow
For more information, please contact CDC’s Office for State, Tribal, Local and Territorial Support
4770 Buford Highway NE, Mailstop E-70, Atlanta, GA 30341Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: http://www.cdc.gov/stltpublichealth
The findings and conclusions in this presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Questions?
Please contact Cassandra Martin [email protected]
Centers for Disease Control and PreventionOffice for State, Tribal, Local and Territorial Support