Neal D. Kohatsu, MD, MPH

Medical Director

May 17, 2012

Evaluation andQuality Improvement

2

What is Evaluation?“ The systematic investigation of the merit, worth, or

significance of an ‘object’ .” Michael Scriven

“…the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards…”

Carol Weiss

“ A systematic way to determine the “value” of a program, program components, or activity.”

cited by Ann Powers

3

“Research seeks to prove, evaluation seeks to improve…”

M.Q. Patton

CDC Evaluation Framework

4

5

Steps in Program Evaluation

• Step 1: Engage Stakeholders• Step 2: Describe the Program• Step 3: Focus the Evaluation Design• Step 4: Gather Credible Evidence• Step 5: Justify Conclusion• Step 6: Ensure Use and Share Lessons

Learned

MMWR; Vol. 48; 9/17/99

6

The Four Standards• Utility: Who needs the information and

what information do they need?

• Feasibility: How much money, time, and effort can we put into this?

• Propriety: What steps need to be taken for the evaluation to be ethical?

• Accuracy: What design will lead to accurate information?

MMWR; Vol 48; 9/17/99

Quality Improvement

7

Shewart/Deming Cycle

8

Rapid-Cycle QI

• Accelerates system change

• Smaller change cycles and samples

• Greater focus on improvement (vs. measurement)

9

Approach to Rapid Cycle QI

• Determine specific aim

• Define measurement

• Specify intervention

10

Conclusions

• Evaluation and QI are related.

• Must adapt to a system context.

• Foster a learning environment.

• Encourage collaboration.

• Focus on outcomes.

11