improving the practice and performance of public health departments: results from the evaluation of...

Improving the Practice and Performance of Public Health Departments: Results from the Evaluation of CDC’s National

Public Health Improvement Initiative (NPHII)

Cassandra Martin Frazier, MPHApplied Systems Research and Evaluation Branch

Division of Public Health Performance ImprovementOffice for State, Tribal, Local and Territorial Support

Keeneland Conference, April 22, 2015

Centers for Disease Control and Prevention

Office for State, Tribal, Local and Territorial Support

Objectives

Describe progress toward achieving outcomes of increased accreditation readiness, quality improvement, and performance management capacity

Illustrate the reach of NPHII program funding and support to help improve public health practice and performance

Discuss plans to sustain current accreditation readiness, quality improvement, and performance management activities

NPHII: The Program at a Glance

Advance public health departments’ readiness for accreditation

Improve organizational efficiency and effectiveness through quality improvement activities

Increase performance management capacity

Supported through the Prevention and Public Health Fund of the Affordable Care Act

FY2010: $42.5M to 76 awardees

FY2011: $33.5M to 74 awardees

FY2012: $33.5M to 73 awardees

FY2013: $32.5M to 73 awardees

NPHII Evaluation Approach Purpose: extent to which NPHII supported improved

public health practice and performance through Increased readiness for accreditation Improved efficiencies/effectiveness Increased performance management capacity

Design: non-experimental program evaluation Multi-strand, sequential mixed methods Quantitative data: annual surveys,* progress reports Qualitative data: stakeholder interviews,* focus groups*

*OMB No. 0920-0879

Data Sources for PresentationData Source Program Progress Reports

Description Semi-annual reports submitted by awardees detailing progress toward completing required program activities

Response Rate Year 3 IPR—n=73 (100%)Year 3 APR—n=73 (100%)Year 4 IPR—n=72 (100%)Year 4 APR—n=68 (94%)

Limitations Self-reported dataLimited validationQuestion revisions over time

Analysis Cross-sectional, frequencies—Year 4 APRCross-time, aggregate—all sources (for select measures)

ACCREDITATION READINESS

Completed all PHAB prerequisites within

past five years* (n=67)

54%

58%

76%81%

Completed PHAB prerequi-site within the past five

years*

Public Health Accreditation Board (PHAB)Prerequisites

* Year 4 APR

S1.1 S1.2 S1.3 S1.4 S2.1 S2.2 S2.3 S2.4 S3.1 S3.2 S4.1 S4.2 S5.1 S5.2 S5.3 S5.4

71%65%

71%

61%

69%65% 66% 65%

77%

61%

81%

74% 74%

55%60%

74%

For domains 1–5, at least 55% of awardees that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards*

*Year 3 IPR and APR; Year 4 IPR and APR

Organizational Self-Assessment Against PHAB Standards and Measures

S1.1 S1.2 S1.3 S1.4 S2.1 S2.2 S2.3 S2.4 S3.1 S3.2 S4.1 S4.2 S5.1 S5.2 S5.3 S5.4

71%65%

71%

61%

69%65% 66% 65%

77%

61%

81%

74% 74%

55%60%

74%

Organizational Self-Assessment Against PHAB Standards and Measures

S6.1 S6.2 S6.3 S7.1 S7.2 S8.1 S8.2 S9.1 S9.2 S10.1 S10.2 S11.1 S11.2 S12.1 S12.2 S12.3

69%

76%

65%

55% 55%61%

39% 37%

45%

73%

66%

52%

77%

68%

61% 61%

At least half of awardees (52%) that completed a self-assessment against the PHAB Standards and Measures (n=62) met the standards for domains 6, 7, 10, 11, 12*

*Year 3 IPR and APR; Year 4 IPR and APR

S6.1 S6.2 S6.3 S7.1 S7.2 S8.1 S8.2 S9.1 S9.2 S10.1 S10.2 S11.1 S11.2 S12.1 S12.2 S12.3

69%

76%

65%

55% 55%61%

39% 37%

45%

73%

66%

52%

77%

68%

61% 61%

QUALITY IMPROVEMENT

Improvements Made in Efficiency

78%

46%

17%12%

7% 5%

Overall, 41 out of 72 awardees (57%) increased their program and/or organizational efficiency through quality improvement initiatives*

*Year 3 APR; Year 4 IPR and APR

Improvements Made in Effectiveness

Quality

Enhan

cemen

t, Serv

ices

Quality

Enhan

cemen

t, Sys

tems

Customer

/ Staff

Satisfa

ction

Increas

ed Rea

ch to

Targe

t Populati

on

Organiza

tional Desig

n Impro

vemen

ts

Dissem

ination of In

form

ation

Increas

ed Prev

entative B

ehav

iorsOth

er

Decreas

ed In

ciden

ce / P

revale

nce

46%

39%35% 33%

25%

18% 16%12%

9%

Overall, 57 out of 72 awardees (79%) increased their program and/or organizational effectiveness through quality improvement initiatives*

*Year 3 APR; Year 4 IPR and APR

REACH OF NPHII SUPPORT

Support to Other Public Health Agencies

0 100 200 300 400 500 600 700 800

697

434

433

199

144

68

Received Mini-grants Received TA, Training, Workshops, Other

Accreditation Readiness

Performance ImprovementQuality Improvement

Performance Management

Number of Public Health Agencies Receiving Support

Non-awardee agencies received indirect support to conduct NPHII-related activities; 52% of awardees used NPHII funds to provide this support (n=68)*

* Year 4 APR

INFRASTRUCTURE AND SUSTAINABILITY OF PERFORMANCE IMPROVEMENT ACTIVITIES

Performance Management CapacityMore than half of awardees established at least one of the four performance management system components required by NPHII* (n=66)

36%

67% 62% 59% 56%

* Year 4 APR

Sustaining Staff/Performance Improvement Efforts

Performance Improvement Manager (n=67)

Performance Improvement Office (n=55)

Quality Improvement activities (n=67)

Accreditation Readiness Activities (n=67)

Performance Management Activities (n=67)

63%

75%

72%

75%

78%

Most awardees plan to sustain their NPHII-related activities and performance improvement office*

* Year 4 APR

Summary Awardees increased their readiness for accreditation

Most established key documents and plans for their organization and partners

Most moved toward meeting standards necessary for accreditation

Awardees demonstrated more efficient processes that save time, and enhanced the quality of their services

Awardees plan to continue working on or maintaining these activities

NPHII not only supported its direct awardees, but reached agencies within their jurisdictions

Acknowledgments Centers for Disease Control and Prevention

Anita McLees Andrea Young Craig Thomas Garry Lowry Saira Nawaz*

Laura Hsu*

NPHII Program Leadership and Staff National Network of Public Health Institutes

Nikki Rider Sarah McKasson Brittany Bickford Sarah Gillen

NPHII awardees* Former CDC contractor and evaluation

fellow

For more information, please contact CDC’s Office for State, Tribal, Local and Territorial Support

4770 Buford Highway NE, Mailstop E-70, Atlanta, GA 30341Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: OSTLTSfeedback@cdc.gov Web: http://www.cdc.gov/stltpublichealth

The findings and conclusions in this presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Questions?

Please contact Cassandra Martin Frazierbkx9@cdc.gov404-498-0581

Centers for Disease Control and PreventionOffice for State, Tribal, Local and Territorial Support