reducing provider cognitive workload in cpoe use: optimizing order sets

23
Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets Yiye Zhang Rema Padman, PhD James E. Levin * , MD, PhD The H. John Heinz III College Carnegie Mellon University, Pittsburgh, PA, USA [email protected] ; [email protected] MedInfo2013, Copenhagen, Denmark * Dr. James E. Levin passed away on February 11, 2013. We are greatly indebted to his vision, contributions and support that made this study possible.

Upload: ciel

Post on 23-Feb-2016

54 views

Category:

Documents


0 download

DESCRIPTION

Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets . Yiye Zhang Rema Padman, PhD James E. Levin * , MD, PhD The H. John Heinz III College Carnegie Mellon University, Pittsburgh, PA, USA [email protected] ; [email protected] MedInfo2013, Copenhagen, Denmark. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Yiye ZhangRema Padman, PhD

James E. Levin*, MD, PhDThe H. John Heinz III College

Carnegie Mellon University, Pittsburgh, PA, [email protected]; [email protected]

MedInfo2013, Copenhagen, Denmark

*Dr. James E. Levin passed away on February 11, 2013. We are greatly indebted to his vision, contributions and support that made this study possible.

Page 2: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Introduction• Significant healthcare delivery challenges in the U.S. and worldwide

– Cost, quality, safety, efficiency, satisfaction– 1999 landmark Institute of Medicine report indicated that 44,000

to 98,000 Americans die each year from medical errors1 – Medication errors are a major component of these errors2

• Potential of healthcare information technology (HIT)– Traditional paper prescription prone to errors due to poor

legibility and miscommunication during patient transfers – Computerized provider order entry (CPOE), a core feature of the

electronic health record (EHR) system, has been recommended to mitigate errors in inpatient orders

1. Institute of Medicine. (1999). To Err is Human: Building a Safer Health System. Retrieved March 28, 2004, from http://www.iom.edu/2. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA 2001;285(16):2114–20.

Page 3: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Computerized Provider Order Entry (CPOE) • CPOE systems are software applications designed to enhance

patient safety by allowing clinicians to enter inpatient orders electronically; Used by one third of US hospitals1

• CPOE systems have been shown to improve patient care through better order legibility, reduced rule violations, improved clinician compliance with best practices, and advanced clinical decision support features2

• Within CPOE, order sets allows clinicians to place multiple, relevant orders for each patient with fewer mouse clicks, thus the creation of order sets is an important prerequisite to successful CPOE implementation and use

1. HIMSS Analytics: Healthcare IT Data, Research, and Analysis. http://www.himssanalytics.org/hc_providers/emr_adoption.asp.2. Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004

Jan;113(1 Pt 1):59-63.

Page 4: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

• Collection of individual orders commonly entered as an aggregate for a specific clinical purpose or procedure

• Typically developed by clinical experts in a generic format

• Support clinicians in high risk situations by serving as expert-recommended guidelines, reducing prescribing time by eliminating unnecessary duplication of work, and increasing clinician compliance with the current best practices1

Order Sets

1. Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of pre-constructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc. 2003 Jul-Aug;10(4):322-9.

Page 5: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Challenges with Order Set Usage • Large Variability in Order Set Usage1 • Difficult to maintain order set content and combinations up-to-date

with current best practices• Lack of involvement in order set development by physicians who

are familiar with both the guidelines as well as the actual practice • Providers switch to ‘a la carte’ orders instead of ordering from

order set, potentially resulting in unsafe and inefficient ordering process

• Poorly designed order sets contribute negatively to treatment quality by exposing users to excessive mouse clicks (physical cost) and cognitive workload (cognitive cost)

1. Zhang Y, Levin JE, Padman R. Data-driven Order Set Generation and Evaluation in the Pediatric Environment. AMIA Symposium Proc. 2012.

Page 6: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Physical and Cognitive Costs• “poor usability--such as poorly designed

screens, hard-to-navigate files, conflicting warning messages, and need for excessive keystrokes or mouse clicks--adversely affects clinical efficiency and data quality” - a recent report from Agency for Healthcare Research and Quality (AHRQ)1

• There is a need to design features of CPOE according to “human factor best practices.” 2,3

1: Schumacher RM, Lowry SZ. NIST Guide to the Process Approach for Improving the Usability of Electronic Health Records. 2010.2. Wright P, Lickorish A, Milroy R. Remembering While Mousing: The Cognitive Costs of Mouse Clicks. SIGCHI Bulletin. 1994.3. Horsky J, Kaufman DR, Oppenheim ML, Patel VL. A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. J Biomed Inform 2003;36(1–2):4–22.

Number of order sets

Physical/cognitive cost

One order set All a la carte

Optimal number of order sets

Page 7: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Research Question• Can the development of order sets be automated using

historical ordering data to learn new order sets that are evidence-based, up-to-date with current best practices, and incur least physical/cognitive costs?

Page 8: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Study Setting• Children’s Hospital of Pittsburgh (CHP) of UPMC, a HIMSS

level 7 pediatric facility• Since October 2002, all inpatient orders at CHP have been

entered directly into the CHP eRecord (Cerner Millenium™)• Over 12,000 pediatric patients admitted each year• Over 10 million order actions in total• On average, a patient at CHP is hospitalized for 5.5 days, and

during that time, 36 unique individuals create 871 order actions

• ~ 2000 departmental, in-house order sets

Page 9: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Sample Appendectomy Orders

PatientID Order Name Order time since admission (hours) Order Set Name Defaults

4092 NPO -1.08Admission Orders

Appendicitis, Complicated

ON

4092 Up Ad Lib -1.08Admission Orders

Appendicitis, Complicated

ON

4092 Vital Signs 4.85 Post Anesthesia Care Orders - Pediatric ON

4092 fentanyl 4.85 Post Anesthesia Care Orders - Pediatric OFF

4092 BUpivacaine 4.87 N/A N/A

A la Carte being utilized

Order set being utilized

Page 10: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Distribution of Orders: Appendectomy MinorSolid blue: a la carte, dotted yellow: order set

Page 11: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Optimization and Clustering ModelsMinimize Cognitive Click Cost (CCC)Subject to1) ‘Default option’ choice constraints2) Cluster formation constraints3) Time interval constraints

Approach: • Order set development from order items

Page 12: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Eliciting Cognitive Costs• CCC with expert estimate (CCCE): Expert input

• CCC based on survey result (CCCS): Survey of 15 subjects including physicians and nurses

• Each survey contains 6 questions with sub-questions, asking subjects to estimate the time it takes them to perform tasks while placing orders with large, mid-size, and small order sets

Page 13: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Approach: Order Set Development• Determine optimal time interval and number of order

sets within the time interval that minimize MCC/CCC

• Cluster orders using bisecting K-means clustering within each time interval

• Map new order set assignment back to historical treatment data to evaluate goodness of clustering using MCC/CCC and coverage rate

Page 14: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Patient Time of order placement Order Order set Default setting CCC

Patient 1 1.4 A O1 ON ?

Patient 1 1.4 B O2 OFF ?

…. … … … … …

Patient 1 10.0 N O3 ON ?

Ex. Fixed patient, time, and order

ON if more than 80% patients use; OFF otherwise

Select De-select

Order Set/ A la Carte 1.2 --

Default ON 0.2 1.5

Default OFF 0.5 0.1

Order Set 1

Order Set 2

Such that CCC can be lowered !Order A

Order B Order COrder E

Order D

Time interval 1, 2,…., n

Page 15: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Results: Significant Reduction in CCC and Increase in Coverage Rate

  CCCE per patient(actual mouse clicks)

CCCS per patient(actual mouse clicks)

  Current New % change Current New % change

AppendectomyMinor 145.7 111.1

(77)23.4%

** 229.6 116.1(83)

49.4%***

Appendectomy Moderate 208.4 143.3

(110)31.2%

*** 289.7 162.7(103)

43.8%***

***: p-value less than 0.01, **: p-value less than 0.05, *: p-value less than 0.1

Page 16: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Closer Look: Appendectomy Minor - CCCE

Time Interval Training Set Test Set

T5: 0 to 2 174 patients, 153 items 23 patients, 72 items

Number of orders

Number of order

sets

Average Coverage Rate per

OS

CCC per Patient

% Reduction

in CCC per Patient

Average Coverage Rate per

OS

CCC per Patient

% Reduction

in CCC per Patient

Current 68 12 0.34 10.2

25.0%

0.29 19.1

31.9 %

New 66 20 0.75 7.6 0.47 13.0

• 12 order sets used per patient on average in training set• 6 order sets used per patient on average in test set

Page 17: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Sample Case: Under Current Order Set

Item Order Set (size) Default

Admit to Admission Orders General Pediatric Medical Order Set (63) ON

Height Admission Orders General Pediatric Medical Order Set ON

Weight Admission Orders General Pediatric Medical Order Set ON

Notify MD For Oxygen Saturations Admission Orders General Pediatric Medical Order Set OFF

Notify MD For TPR Admission Orders General Pediatric Medical Order Set OFF

Regular (4 yrs & >) Diet Admission Orders General Pediatric Medical Order Set OFF

Up Ad Lib Admission Orders General Pediatric Medical Order Set OFF

Vital Signs Admission Orders General Pediatric Medical Order Set OFF

Subsequent Oxygen Therapy Oxygen Therapy (2) ON

Initial Oxygen Therapy Oxygen Therapy OFF

Subsequent Pulse Oximetry Continuous Pulse Oximetry Continuous (2) ON

Initial Pulse Oximetry Continuous Pulse Oximetry Continuous OFF

CCCE = 20.3, CCCS = 76, number of actual mouse clicks (MC) = 15

Page 18: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Sample Case: Under New Order Sets

Item MCC (size) CCCE (size) CCCS (size) Default

Admit to C1 (3) C1 (2) C1 (5) ON

Height C2 (3) C2 (3) C2 (4) OFF

Weight C2 C2 C2 OFF

Notify MD For Oxygen Saturations a la carte C3 (3) C3 (6) ON

Notify MD For TPR a la carte C2 C2 OFF

Regular (4 yrs & >) Diet a la carte C3 C3 OFF

Up Ad Lib a la carte C4 (2) C3 OFF

Vital Signs C1 C1 C1 ON

Subsequent Oxygen Therapy C3 (3) C5 (2) C1 OFF

Initial Oxygen Therapy C3 C5 C1 OFF

Subsequent Pulse Oximetry Continuous C4 (3) C6 (2) C4 (3) ON

Initial Pulse Oximetry Continuous C4 C6 C4 ON

CCCE = 12.9 (36.4% drop ), CCCS = 23.1 (69.6% drop), MC = 13 (15.4% drop)

Page 19: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

• Order Set development based on data-driven approaches is promising

• Can be generalized for not only CHP order sets but also for order sets in other settings with different workflows

Conclusions

Page 20: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Limitations and Challenges

• Large variations in ordering patterns• Influence on usage by the current order sets• Rare combinations of orders need to be

addressed separately in a data-driven approach• Constant CCC weights assumption• Incorporation of new scientific evidence

Page 21: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Future WorkDevelop new approaches and extend/test current methods on

other diagnoses and in other settings1

• Implemented an order set development platform and tested on pneumonia patients

• Incorporate alternate methods using heuristic optimization

Evaluation by physicians on the usability and clinical validity of newly created order sets

• Currently looking for interested institutions to partner on the clinical evaluation studies

1: Zhang Y, Padman R, Levin JE. Data-driven Order Set Development Using Tabu Search. Heinz College Working Paper, Carnegie Mellon University, Pittsburgh, PA, May 2013.

Page 22: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

Relevant Publications• Zhang Y, Padman R, Levin JE. Clustering Methods for Data-driven Order Set

Development in the Pediatric Environment. INFORMS 2012 DM-HI Workshop Proc., October 2012

• Zhang Y, Levin JE, Padman R. Data-driven Order Set Generation and Evaluation in the Pediatric Environment. AMIA Symposium Proc., November 2012.

• Zhang Y, Levin JE, Padman R. Toward Order Set Optimization Using Click Cost Criteria in the Pediatric Environment. HICSS-46 Proc., January 2013.

 • Zhang Y, Padman R, Levin JE. Data-driven Order Set Development in the

Pediatric Environment: Toward Safer and More Efficient Patient Care. Heinz College Working Paper, Carnegie Mellon University, Pittsburgh, PA, December 2012.

Page 23: Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets

THANK YOU!

QUESTIONS?