california chronic care learning communities initiative collaborative learning session i how do we...

California Chronic Care Learning Communities Initiative Collaborative

Learning Session I

How Do We Get There Quickly?Model for Improvement - Part 2: Testing Changes with PDSA Cycles

Angela Hovis, Improvement Advisor

© 2004 Institute for Healthcare Improvement

Session Objectives

Participants will be able to:• Describe the Model for Improvement and its

utility in accelerating improvement initiatives

• Use the change package

• Describe the importance of testing changes for learning and improvement

• Test Changes with PDSAs


“Not all changes are improvements, but all improvements are a result of changes!”

Testing Changes

• Test• Trial and Learn - try and adapt ideas

• Implement• make a change part of the day to day

operation of the system in the pilot

• Spread• make a change part of the day to day

operation of the system outside of the pilot

Side Bar: Test vs Implement vs Spread


Fundamental Questions for Improvement

• What are we trying to accomplish?• Team Aim Statement

• How will we know that a change is an improvement?

• Measures

• What changes can we make that will result in an improvement?

• Changes

What are we trying toaccomplish?

How will we know that achange is an improvement?

What change can we make thatwill result in improvement?

Model for Improvement

Act Plan

Study Do

3. What Changes Can We Make

That Will Result in Improvement?

• The “change package” contains good concepts and ideas (based on best practices and research) for changes that can help us accomplish our goals.

• Use the change package to identify the changes you want to make to your system to achieve your aim.

Informed,ActivatedPatients andCaregivers

ProductiveInteractions

Prepared,Proactive,Practice Team

Improved Outcomes for Patients with Diabetes/CAD

DeliverySystemDesign

DecisionSupport

ClinicalInformation

Systems

Self-Management

Support

Health System

Resources and Policies

Community Health Care Organization

Using the The Care Model for Self-Management Support

ConceptAn opportunity to create

a new connection

Thoughtprocess

Specificidea B

Specificidea A

Change Concept: A general notion or approach to change that has been found to be useful in developing specific ideas for changes that lead to improvement.

Change Concepts vs. Specific Changes

Vague, Strategic, Arrange: Provide follow up to patients after they have set goal (agree)

Conceptual

Contact patients one week after they have set goal

Find out how patients are doing with their goal - reassess confidence level

and see if they have questions or concerns

Specific Ideas, Actionable At time of office visit, solicit patient’s

preference for follow up and have MA either call or e-mail patient within 3 weeks office visit

Using the Change PackageSelf-management Component of Care Model

Concept Key Change Example of Idea to Try2. Use effective self-

management supportstrategies that includegoal setting, actionplanning and problem-solving and follow-up.

d. AGREE on collaboratively set goalsbased on patient’s confidence in their abilityto change the behavior

a. Assess patient confidence to achieve their action plan using a 1-10 scale.(Also described in Lorig et al.)

http://www.healthdisparities.net/resources.html (scroll down) lookfor Provider Support for Patient Self-Management

b. Use CARE vital signs from howsyourhealth.org to set goalsc. Use techniques from motivational interviewing (express empathy, develop

discrepancy, avoid argumentation, roll with resistance, support self -efficacy)

e. ASSIST patients with problem-solving byidentifying personal barriers, strategies, andsocial/environmental support

1. Use problem-solving method described by Lorig et al individually and ingroups to teach patients problem solving skills.

2. Hold a group diabetes/CAD visit and incorporate goal-setting, action-planning and problem-solving into the agenda. (see Health CareOrganization)

3. Screen for depression with PHQ-9http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/

• Other depress ionmanagement resources:MacArthur Foundati onToolkithttp://w .ww depression-primarycare.or /gclinicians/toolkits/

f. ARRANGE aspecific follow-u p plan. 1. Schedule telephon e follow-u pan d mak e it pa rto f someone’ s jo bdescription.2. Engag e volunteer s to assi stwit hfollow-up.3. Us eema il tocommunicat ewit h patient sabou t goals, act ion pla nsand

follo -w . up Relay Healt ?h






Act Plan

Study Do


What is the PDSA Cycle?

Act

• What changes are to be made?

• Next cycle?

Plan• Objective• Questions and predictions (why)• Plan to carry out the cycle (who, what, where, when)

Study• Complete the analysis of the data

•Compare data to predictions

•Summarize what was learned

Do• Carry out the plan• Document problems and unexpected observations• Begin analysis of the data


The PDSA Cycle

Why Test?

Act Plan

Study Do

Three Doors and Goat: Experiencing PDSAPresented by Angela HovisOffice Practices Summit

March 27, 2004

Created by: Roberto Colacioppo ([email protected])IMECC / UNICAMP (www.ime.unicamp.br)Campinas – São Paulo – Brazil

The Three Doors Game

Learning how to run PDSA and use forms

Pick a door, please.

1 2 3

Without opening your door, I will show you a goat, ok?

1 2 3

Do you want to switch?

1 2 3

Wow! You got the car!

1 2 3

Do you want to switch?

1 2 3

Sorry, You got the goat!

1 2 3

Flowchart

The host of the show shows the three doors

Contestant chooses a door

Host opens a door with a goat from the two doors not chosen

Host asks if the contestant wants to switch doors

Contestant decides yes or no

Host opens the other two doors

Background info

• This story is true, and comes from the American T.V. game show “Let's Make a Deal”, presented by Monty Hall in the 70’s.

• This situation was discussed in the popular "Ask Marylin" question-and-answer column of the Parade magazine in the 90’s.

• Her answer was: "Yes, you should switch."

• To date, Ms. vos Savant has received over 10,000 letters, mostly disagreeable.

Marilyn vos Savant

Monty Hall

Is Ms. vos Savant correct?

• What does your intuition tell you about this problem?

• Does the contestant have a 50-50 chance of winning with either choice?

• Or is it better to switch from the original choice?

• To learn about this question, we will work on a PDSA cycle.

The PDSA Cycle

Act

• What changes are to be made?

• Next cycle?

Plan• Objective• Questions and predictions (why)• Plan to carry out the cycle (who, what, where, when)

Study• Complete the analysis of the data

•Compare data to predictions

•Summarize what was learned

Do• Carry out the plan• Document problems and unexpected observations• Begin analysis of the data

Let's use a basic PDSA form

© Associates in Process Improvement 2002All Rights Reserved

MODEL FOR IMPROVEMENT Cycle: Date: _____

Objectives for this PDSA Cycle:

QUESTIONS:

PREDICITONS:

PLAN:

PLAN FOR CHANGE OR TEST (INCLUDING ANY COLLECTION OF DATA): WHO, WHAT,WHEN, WHERE

DO: CARRY OUT THE CHANGE OR TEST; COLLECT DATA AND FEEDBACK. DESCRIBEWHAT HAPPENED INCLUDE REPORTING OF ANY UNEXPECTED EVENTS. BEGINANALYSIS.

STUDY: COMPLETE ANALYSIS OF DATA ; SUMMARIZE WHAT WAS LEARNED.

ACT: ARE WE READY TO MAKE A CHANGE? PLAN FOR NEXT CYCLE.

•Project: Monty Hall Problem •Cycle #: 1 •Date: 03/27

•Objective: Finding a solution to the Monty Hall Problem

PLAN

•Questions •Predictions

Does the contestant have the same chance of winning with either staying or changing the original choice?

Yes, it doesn't matterNo, staying with original is betterNo, switching is better

•What data will be collected during this time? (Forms to be used)

Fill in the objective, questions and predictions


•Objective: Find a solution to the Monty Hall Problem

PLAN•Questions •Predictions


Yes, it doesn't matterNo, staying with original is betterNo, switching is better


• Work in pairs (host and contestant)• Each pair will simulate both situation 10 times with playing cards• Data will be collected in the form attached• A Run Chart (attached) will be filled with results from all pairs• Observations focused on the difference between the two situation will be collected

Make a plan to collect data

EXAMPLE

Using playing cards to collect data

original choice

Staying: 10 times

Switching: 10 times

Here we have a goat

(red card)





Yes, it doesn't matter. Chances are equal.No, staying with original is betterNo, switching is better


• Work in pairs (host and contestant)• Each pair will simulate both situation 10 times each with playing cards• Data will be collected in the form attached• A Chart (attached) will be filled with results from all pairs• Observations focused on the difference between the two situation will be collected


WorksheetData Collection for 3 doors and Goat

Result ResultDecision Run

Car GoatDecision Run

Car Goat

1 12 23 34 45 56 67 78 89 9

10 10

StaywithOriginalDoor

Totals

SwitchDoors

Totals


•Objective: Find a solution to the Monty Hall Problem•PLAN

•Questions •Predictions


Yes, it doesn't matter. Chances are equal.No, staying with original is betterNo, switching is better


• Work in pairs (host and contestant)• Each pair will simulate both situation 10 times each with playing cards• Data will be collected in the form attached• A Chart (attached) will be filled with results from all pairs• Observations focused on the difference between the two situation will •be collected


Chart for Data

3 doors and goat

0123456789

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

cycle or trial

#of times out of 10 = car

Staying

Switching





Yes, it doesn't matter. Chances are equalNo, staying with original is betterNo, switching is better


• Work in pairs (host and contestant)• Each pair will simulate both situations 10 times each with playing cards• Data will be collected in the form attached• A Chart (attached) will be filled with results from all pairs• Observations focused on the difference between the two situation will •be collected


Let's carry out the plan

DO

• What went wrong? What happened that was not part of the plan? Begin analysis.

• Write your observations made during simulation: What happened? What are the differences between staying with and changing the original choice?

STUDY

• Complete analysis of data. Summarize what was learned include results of predictions.

• 1.

• 2.

Let's study!

STUDY

• Complete analysis of data. Summarize what was learned include results of• predictions.

• 1. What does the variation the data graph indicate?

• 2. What hints can we extract from the observations to explain such behavior of the data?

• 3. Can we conclude and formulate a theory about the Monty Hall Problem?

• 4. Summarize what was learned. Compare to predictions .

Decisions...

ACT

• What decisions were made from what was learned?

• What will the next cycle be?

What did we learn from this exercise?


Why Test Changes?• To increase the belief that the change will

result in improvements in your setting• To avoid surprises or challenge

assumptions• To learn how to adapt the change to

conditions in your setting• To evaluate the costs and “side-effects”

of changes• To minimize resistance when spreading the

change throughout the organization

Some comments and feedbacks: PDSA…

… helps to focus and to organize our job (objective, questions and predictions). (P)

… helps to plan which data will answer our questions (it stimulates using data). (P)

… stimulates to plan the analysis of the data before obtain it (avoid waste). (P)

… stimulates to document observations (D)

… stimulates insights and creativity. (D)

… stimulates to test paradigms by comparing predictions and actual data. (S)

… helps to learn what is necessary to go ahead. (S)

… stimulates to apply the specific knowledge obtained. (A)

… is an efficient way to document our improvement project. (form)

WATCH OUT!!

Some misuses of PDSA by beginners• Tendency to make objective statements too broad• Confusion between objective and questions• Tendency to formulate many irrelevant questions

to the PDSA objective• Tendency to put Plan issues (e.g. What data do we

need? Who will collect data?) mixed with PDSA questions

• Tendency not to anticipate in the Plan step how they will analyze data to be collected

• Tendency not to record useful observations in “Do”

• In Study, they draw conclusions that don't answer PDSA questions (they forget the original focus)

• Tendency to do too much in one THE PDSA, instead of cycles

• Tendency to use PDSA for collecting data on a measure rather than to test an idea for change


“Persuasion is not enough…… After adopters understand the new ways, they must be provided with the necessary guidance and ample opportunity to perfect the modeled activities under circumstances where they need not fear making mistakes.”

Albert Bandura

Promoting Adoptive Behavior: Moving to Action


Bon Secours First PDSA Cycle(Office Practices - October, 2002)

• Plan • Objective of first test: To know how to motivate patients to

healthier behavior with respect to HbA1c education• Question: What is the patient's perception related to

HbA1c>7?• Prediction: Lack of knowledge (pt) and not motivated to lose

weight, test blood, etc. • (who, what, where, when, how) Pt identified in teams first data

collection (pre-IMPACT meeting) contacted to come in for visit on 10/14 to identify barriers.

• Measure(s) to assess the success of this test - How's your health?" tool and patient interview.


Bon Secours First PDSA Cycle

• Do • One patient was interviewed and used the

How's Your Health tool. The pt is well educated and had insight that when the MD calls him when his HbA1c is going up-he adheres to diet and exercise. He also identified that travel is a problem with compliance. They mapped a trend related to his blood. He also identified from the tool some areas that had not really focused on such as foot care.



• Study

• See above-we disproved our theory-pt is knowledgeable but we need to customize teaching relative to travel and staying on the path. Pt more motivated when someone is calling him back or contacting him and he is motivated to be contacted by email so this will

be part of his plan.



• Act • As noted in first test- pts surveyed at the practice in a day

>50% do want to be contacted and wish to contact office by email. Computer being set up in office space to go ahead and arrange to answer email by staff and gather email addresses.

• Jeff Glover developed a questionnaire for diabetic pt that he is giving as they are being scheduled to come in (those with HbA1c>7) to start.

• Letter drafted to introduce the survey tool and we will look to add some incentive for returning the survey such as return envelope included or perhaps movie tickets? Will reformat tool to 1 page and we will email to Marie after reformat.

• Actively investigating the registry tools and expanding review of diabetic charts to reschedule pts to be seen.


Tips for Testing Changes

Identify changes that will assist the team to reach goals

Harvest good ideas and adapt them to your setting

Solicit team members ideas Start with changes that are easy to

implementLink multiple testing cycles to accelerate

learning


Tips for Testing Changes

Test large innovative changes on a small scale (e.g. one MD, one patient)

Collect useful data during each testLearning from “failed” testsTest over a wide range of conditionsDevelop plans to simulate the changeMake changes side-by-side with the

existing system


Move Quickly to Testing Changes

• Year• Quarter• Month• Week• Day• Hour

“What tests can we complete by next Wednesday?”

Repeated Use of the PDSA Cycle

Hunches Theories

Ideas

Changes That Result in

Improvement

A P

S D

APS

D

A P

S D

D SP A

DATA

Very Small Scale Test

Follow-up Tests

Wide-Scale Tests of Change

Implementation of Change

Build knowledge sequentially with multiple PDSA cycles for each change idea.

Peak flow meters for high-risk patients

Routine use of flow meters by

high-risk patients

A P

S D

APS

D

A PS D

D SP A

DATA

D SP A

Cycle 1:Test communication on use of flow meters with 1 patient

Cycle 2: Revise and test with 2 provider and 6 patients

Cycle 3: Revise and test with 3 providers and their patients for one week

Cycle 4: Train providers on teaching patients to use flow meters

Cycle 5: Monitor communication and use of flow meters with high-risk patients

A P

S D

A PS D

A P

S D

D S

P A

A P

S D

A PS D

A P

S D

D S

P A

A P

S D

A PS D

A P

S D

D S

P A

A P

S D

A PS D

A P

S D

D S

P A

Recruitment Size and Location

The Right Team

Example of Testing Multiple Changes

ProgramContent

Cycles for ImplementingGroup Visit

Table Exercise

Plan for Action Period 1 and First PDSA

1st PDSABy next Wednesday,November 3






Act Plan

Study Do


References

• The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. G. Langley, K. Nolan, T. Nolan, C. Norman, L. Provost. Jossey-Bass Publishers., San Francisco, 1996.

• Quality Improvement Through Planned Experimentation. 2nd edition. R. Moen, T. Nolan, L. Provost, McGraw-Hill, NY, 1998.

• “Understanding Variation”, Quality Progress, Vol. 13, No. 5, T. W. Nolan and L. P. Provost, May, 1990.

• A Primer on Leading the Improvement of Systems,” Don M. Berwick, BMJ, 312: pp 619-622, 1996.

• “Accelerating the Pace of Improvement - An Interview with Thomas Nolan,” Journal of Quality Improvement, Volume 23, No. 4, The Joint Commission, April, 1997.

california chronic care learning communities initiative collaborative learning session i how do we...

Documents

improvement test changes

improvement advisor

testing changes slide

result of changes

change concepts

pdsas slide

improvement actplan

importance of testing