change starts here. the one about outcomes and indicators icpc national coordinating center this...
TRANSCRIPT
Change Starts Here.The One about Outcomes and
IndicatorsICPC National Coordinating Center
This material was prepared by CFMC (PM-4010-080 CO 2011), the Medicare Quality Improvement Organization for Colorado under contract with the Centers for Medicare &
Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy.
Recap: measurement for IC-4
1. Time series outcomes– Effect on root cause/driver– Success of the intervention
• Rates; scores; rating scales• Best-fit line or other signal indicating improvement• What to do about outcomes not well portrayed as time-series
2. Intervention implementation– Reach/dosage of an intervention– Who was affected?
• Counts• Rates among eligible population (offered, refused, completed)
Recap: suggested approach
1. Map out a detailed, community-level logic model of the intervention strategy.
2. Select and operationalize outcomes and processes from the logic model.
3. Develop and enforce the system for tracking implementation and outcome.
4. Effectively report time series data.
Who should participate in outcomes selection and measurement?
• Considerations– Who will implement the interventions?– Who will be collecting data?– Community stakeholders
• To whom will the observed changes will be meaningful?
• Resources brought by the QIO– Building capacity for outcome measurement
• Logic models• Data collection
– Technical assistance; analytic support• Data management and reporting• Detecting improvement (tests of trend, run/control charts)
Recap: logic model outcomes
Expected short-, medium-, and long-term changes and improvements• Short-term
– Specific improvements in the targeted driver or root cause
• Medium-term– Related outcomes along the causal path
• Long-term– Improved care transitions– Avoided readmission– Improved health care utilization
• Implications of potential negative changes or non-changes
Short-/medium-term outcomes are measured for IC-4.
Recap: outcomes selection
SMART objectives guide outcome selection.
SpecificConcrete; represents what, or who, is expected to change
Measureable Can be seen, heard, counted, etc.
Attainable Is likely to be achieved
Results-orientedGenerates meaningful, valued results
TimedHas an acceptable target date
Going further: attainability, moveabilty
Short-term outcomes are more likely to show movement…
…but consider downstream (medium-term) outcomes if short-term outcomes are not feasibly measured.
InterventionOutcome A
(short-term; proximal)
Outcome B Outcome CAvoided
readmission(long-term; distal)
Transitions coaching
Improved patient
activation
Improved health self-
management
Improved symptom control
Avoided readmission
Discharge process standardization
Improved patient knowledge, resources;
coordination of follow-up care
Fewer unmet needs and gaps in
care
Improved symptom control
Avoided readmission
Outputs Short-term outcomes --------------- Long-term outcomes
Indicators
What does the outcome look like? How would we know it improved?• Create operational definitions.
– Specifics of how the outcome is measured• Make it a number. Quantitative data can be plotted over time or compared
across groups.– Rate, percentage
• Numerator and denominator
– Score (continuous)• e.g., 0 to 100 points
– Rating scale (ordinal)• e.g., [0 - ‘never’] -- [1 - ‘sometimes’] -- [2 - ‘always’]
Simple counts are not very informative on their own. Find a way to use them as the numerator of a rate or percentage.
Timing and duration
When will improvement be detected?• Considerations
– How long should it take to observe an effect?– What should the effect look like?
• Abrupt, sustained improvement• Rises and falls, with gradual trend towards improvement
• IC-4: ≥4 quarters of data within 18 months of community engagement– Ensure that the measurement period includes pre-intervention baseline data.
• Measure frequently– The more data points, the better. Monthly indicators lend themselves to run/control
charts.
Level of analysis
Where will improvement be seen?• Provider-/initiative-level
– A change occurs across several events, people, or organizations.• Process improvements (re: standardization; information transfer)
– e.g., inpatient satisfaction with discharge information
• Utilization outcomes– e.g, 7-day readmission rates; 30-day primary care follow-up rates
• Patient-level– Something happens to an individual person related to his/her patient experience.
• Patient activation (e,g., change in PAM score)• Utilization (e.g. , prevented readmission)
• Patient-level data should be aggregated to provider-/initiative-level for reporting.
Collecting data
• Considerations– What method?
• Sampling• Survey, case review, etc.
– Who collects the data? Who analyzes it?– How will data quality be ensured?
• Make it explicit.– Standardization, data specifications– Accountability
• Don’t forget to track implementation (process) measures, as well.– Counts, rates among eligible population (offered, refused,
completed)
More to come
• Detecting and reporting improvement• Context and reasons for success/failure
Resources
• Toolkit – measurementhttp://www.cfmc.org/caretransitions/toolkit_measure.htm
• Measuring Program Outcomes: A Practical Approach
http://www.unitedwaystore.com/product/measuring_program_outcomes_a_practical_approach/program_film
Excerpts: http://www.unitedwayslo.org/ComImpacFund/10/Excerpts_Outcomes.pdf
• ICPCA NCC contact: Tom [email protected]
Questions?
The ICPC National Coordinating Center – www.cfmc.org/caretransitions
Change Starts Here.