using measurement to drive improvement: new tools to help you succeed

www.saferhealthcarenow.ca

Using measurement to drive improvement: New tools to help you succeed

Alexandru Titeu and Virginia Flintoft CPSI Central Measurement Team

March 13, 2013


You Joined Incorrectly! There is NO phone icon beside your name. You will be Unable to join the breakout sessions – please follow the steps below!.

You Joined Correctly!

There IS a phone icon beside your name. You will be able to join the breakout sessions.

Did you Join the Call Correctly?

If there is NO phone icon beside your name: 1. Hang up your phone (but stay connected to WebEx) - then click the ‘audio’ button on the right-hand bar of the WebEx screen.

2. A popup will display the phone information.

3. Either have WebEx call you (if you do NOT have an extension), or dial-in (if you are on a line with an extension).

Direct Line Enter

number

Line with Extension

“ I will call in”


WebEx


Today’s Presenters

4

Helène Riverin

Dr. Peter Norton Virginia Flintoft Alexandru Titeu

Anne MacLaurin


• Understand the importance of measurement in driving improvement – Dr. Peter Norton

• Introduce Patient Safety Metrics: a cloud-based tool for data collection and performance monitoring.

• Demonstrate new data collection forms designed to reduce the burden of measurement

• Outline the application of Patient Safety Metrics beyond Safer Healthcare Now!

Objectives


Measurement Peter Norton

Emeritus Professor of Family medicine University of Calgary

March 2013


Fact

Gaps in quality

and safety

Individually, locally, regionally and nationally

Solution

Improve processes

and systems

Individually, locally, regionally and nationally

What are we doing?

How will we know if we are moving forward?

7


The simple answer....

'If you cannot MEASURE it, you cannot IMPROVE it'.

Lord Kelvin, International Electrotechnical Commission’s first President (1906)


• How we know a change is an improvement • Measurement is only a handmaiden to

improvement, but improvement cannot act without it

• In our work measurement is not for the purposes of judgment

• Rather it is for purposes of learning

Why measure?


Remember, measurement is a means for answering questions. It is not an end in itself.

WARNING


Local champion(s) and empowered local teams Goal

Improvement and Safety

Leadership support and involvement

Board commitment

Adequate human and financial resources

Organization Culture -Learning -Quality and safety -Change

Skills - quality improvement - Change management - Team dynamics

Measurement

• Communication • Put patient at center • Appreciate local and facility context

11


Measurement for Turn around time

Accuracy Level

Reseach Long Very Project Accountability Matched as far as

possible to management

cycle

Moderate Unit/facility/region

Improvement Quick - small changes that cummulate

Good enough Unit on specific processes/cases

But measurement is complex…


Judgment judge us against performance targets - you can use measures to judge and manage your own progress

Diagnosis to understand the process, to see if there is a problem and how big it is

Improvement a few specific measures, linked to the your aims, demonstrate whether the changes are making improvements

Sustainability ensure the changes and the improved outcomes are maintained and are part of everyday practice

Spread specific measures to demonstrate the extent to which learning and change principles for improvement have been adopted

In healthcare we use measurements for…


• Surveys patients providers staff • Patient record

• Direct observation

• Administrative data

• Primary collection

Data Sources


• Makes more sense to most • We can see more things

Graph Data


Run charts Plot data over time


Example

• A clinical improvement team was formed in a gyne clinic

• An identified goal was to reduce inappropriate ordering of FSH testing for women 40-44 years old. (FSH stands for “follicle stimulating hormone,” which is sometimes used to determine the onset of menopause.)

• The lab reported the total number of FSH tests performed each month

• Guidelines were issued in October (Observation 12)

• Were they effective? Davis Balestracci


Run chart

Don’t draw the regression line!! R2 = 0.39 p = 0.003 It would imply that 0 would be obtained at month 42


Enhanced run chart

Median = 22

Median = 12


Stages of Facing Reality: “To live divided no more”

• “The data are wrong” • “The data are right, but it’s not a problem” • “The data are right; it is a problem; but it is

not my problem.” • “I accept the burden of improvement”

D. Berwick


Why are results questioned?

There are three kinds of lies: lies, damn lies, and statistics.

• Avoid Measurement Bias • Define what you are counting – everyone should

understand and be able to get the same answer • Make it systematic • Analyze & display the results properly

Benjamin Disraeli


Questions?

23


• All CPSI’s performance data arises from it’s flagship program, Safer Healthcare Now!

• SHN began collecting improvement data from enrolled healthcare sites in 2006

• All performance data is collected and stored by the Central Measurement Team at the University of Toronto.

History: CPSI and Measurement


“It is not acceptable for …[healthcare facilities] to make blanket statements about providing high-quality care without backing it up with proof.”

Dr. Jack Kitts, President & CEO The Ottawa Hospital Metrics for Healthcare: The Leader’s Role. CMAJ, Feb.2010


• Outcome – Indicates if the organization has successfully achieved the primary

outcome of the project – Example: Reduced Healthcare-Associated Infections

• Process – Indicates if the team has successfully made the desired changes in a

targeted process e.g., • Are the parts/steps in the system performing as planned? • Are key changes being implemented in the system?

– Example: Percent appropriate hand hygiene practices

• Balancing – looks at the system from different directions – indicate if other parts of the system have been disrupted by the

changes (adverse effects). – Example: Percent staff reporting skin breakdown?

Three types of Improvement Measures


• "Process measures give employees the knowledge and motivation to succeed"

Process measures


• To generate support for your efforts.

• To promote spread to other parts of your organization

Why Plot Data Over Time


8

3

0123456789

10

Before Chance (measure point on week 4) Af ter Change (measure point on Week 11)

Dela

y Ti

mes

(Hrs

)

Before and After Test(Change Made Between Week 7 & Week 8)

Case 1

0

2

4

6

8

10

12

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks

Del

ay T

ime

(Hrs

)

Made Change

Why Use a Run Chart?

Changes are not fixed but are adapted over time.

Summary statistics hide information (i.e. patterns, outliers).


• Safer Healthcare Now! has over 8 years experience in developing and guiding organization on process measures to guide improvement work. We serve the front line providers

• Strengthen the use of process measures across the country

• Link CIHI outcome data to SHN process measures • Create process measures and outcome data to support

Accreditation Canada’s ROP’s.

Value in process measures


Patient Safety Metrics An introduction


Features: • Cloud-based data collection and reporting tool • User friendly and simple to navigate • Accessible from website with login details • Tracks >100 process and outcome measures over 14

interventions • Provides real time reporting. • Reduces burden of data collection, entry and analysis • Capacity to customize measures and reports

Patient Safety Metrics


• Measurement Worksheets – Aggregate data - monthly – Numerator and Denominator

• Data Collection Forms – Patient-level data (de-identified) - daily – Multiple data elements – Print form Collect data Fax form – Automatic roll-up to Measurement Worksheets

Worksheets vs Data Collection Forms


PS Metrics can be used to support: • Small and Large Scale Improvement Initiatives

– Roll Up or Drill Down Reports [e.g. Unit Site Program Corporation Region Province Node National]

– Produce automated run charts • Reporting for accountability

– Possible to customize indicators to meet provincial, regional and local reporting needs

Potential applications of the system


“…The system is great and teams are more connected with their own data…” Dawn Hallohan, Performance Measurement Coordinator Cape Breton Regional Health Authority, NS

35


PS Metrics

Live demonstration of PS Metrics https://shn.med.utoronto.ca/metrics-test/Login.aspx

36

https://shn.med.utoronto.ca/metrics-test/Login.aspx




Thank you … Questions?

37


Poll Sondage

using measurement to drive improvement: new tools to help you succeed

Health & Medicine