usability & human factors - remote learner · 15min-1hr if patient was in extremis •entering...

Usability & Human Factors

Designing for Safety

Lecture a

This material (Comp15_Unit10a) was developed by Columbia University, funded by the Department of Health and Human

Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.

Designing for SafetyLearning Objectives

2

• Apply principles underlying the design of

healthcare systems for safety (Lecture a)

• Identify common sources of error

documented in research studies in

medicine (Lecture a)

Health IT Workforce Curriculum

Version 3.0/Spring 2012



Lecture a


"The origin of primum non nocere." http://en.wikipedia.org/wiki/British_Medical_Journal electronic responses and commentary, 1 September

2002.

3Health IT Workforce Curriculum




Lecture a

•“First do no harm”

First principle of medical practice

http://en.wikipedia.org/wiki/British_Medical_Journal

Patient Safety





Lecture a

IOM report (1999: To err is human): 44,000 to 98,000 die from preventable medical errors

More than vehicle accidents, breast cancer, AIDS; 8th largest cause of death

Nosocomial and Iatrogenic

Feb 2010 – half of all infection deaths attributable to hospital

Complexity v. complications v. simple errors

Medical complexity is a main barrier to safety

Errors





Lecture a

Computers switched patient record spontaneously

• Huffington Post Aug 4, 2010

Pharmacy orders failed to be delivered

• >shutdown

No national mandatory monitoring procedure

6 deaths and more than 200 injuries tied to EHRs

• FDA official Jeffrey Shuren

Pediatrics: Increased Mortality with Computerized Physicians Order Entry (CPOE)





Lecture a

• 2.80% (39 of 1394) before CPOE implementation

• to 6.57% (36 of 548) after CPOE implementation

Mortality rate significantly increased from:

Multivariate analysis revealed that CPOE remained independently associated with increased odds of mortality (odds ratio: 3.28; 95% confidence interval: 1.94–5.55) after adjustment for other mortality covariables

Increased Mortality: Reasons (from Sittig, 2009)





Lecture a

• After CPOE implementation, order entry not allowed until patient physically present & registered into system

• All medications centrally located in the pharmacy dept

• Because drug orders could now only be processed after nurses activate them, nurses had to spend time at computers and away from the bedside

Hospital-wide implementation of CPOE + clinical apps in 6 days

Order entry and activation through computers reduced face-to-face doctor-nurse communication

Increased Mortality: Reasons (cont.)





Lecture a

• Before CPOE, clinicians converged at bedside to stabilize patient

• After CPOE, clinician stayed at computer to enter orders in the first 15min-1hr if patient was in extremis

• Entering stabilization orders required 10 clicks/order (1-2min/single order) compared to a few seconds for the previous written order

Increased time burden changed organization of bedside care

• Bandwidth exceeded at peak periods

• Additional delays between each click

Majority of terminals were wireless

Horsky: Dosing Error (Detailed Analysis)





Lecture a

Potassium chloride (KCl)

• methods limited in scope to their distinct analytical domains would not identify

Error was interaction among human & system agents

• Confusing on-screen laboratory results review

• System usability difficulties

• User training problems

• Suboptimal clinical system safeguards

Errors in several converging aspects of the drug ordering process:

Errors, Safety, Perfectibility:Errors Viewed in 2 Ways





Lecture a

Person approach

• ‘Perfectibility model’

• If clinicians work hard enough & trained, errors will be avoided

• Blame-oriented

• Emphasis on dealing with increased complexity with more training (Gawande)

System approach

• Multiple contributions to error, including work environment

• Recognize that perfection will not happen, anticipate and avoid errors, build a resilient system

• Aviation view (after crash)

Human Factors (Elrod, 2009)





Lecture a

National Patient Safety Goals from JCAHO

• E.g. CPOE look-alike, sound-alike drugs

NPSG 3 – medication safety

Impact of drug names

Anticipate and prevent errors around drug selection

Conflict notification

Design Considerations(from Kaye, 2010)





Lecture a

“Error tolerance” good attributes for many devices

• Potential difficulties are anticipated/identified

• Design of the UI to control their likelihood

Features of the User Interface (UI) that prevent activation of critical actions following minor, incorrect actions by user

• Request verification before proceeding

• Parameter limits (e.g., values greater than “250” not accepted)

Examples:

“Use Safety” Evaluation (Kaye 2010)





Lecture a

• Major issues are best addressed prior to final/validation testing through early user involvement and “formative” evaluations

• Test protocol focused according to identified priority of tasks or “use scenarios.”

• Environment of actual system use and design configuration are addressed

• Performance measured meaningfully

• Performance (a.k.a. “usability”) goals, such as “80% of users were successful indicates up to 20% failure rate.

Retrospective Incident Analysis





Lecture a

Originally for ID of factors leading to failure

Now also for analysis for error recovery, ID near misses

Near misses can ID recovery factors as well

Planned recovery opportunities v. unplanned

Of 52 medication errors (->death): 127 recovery opportunities (absent, missed, failed); 0 to 11 opportunities/error, avg. 2.4; of the 52, only 4 presented no opportunities

Order Sets





Lecture a

Diffusion of standardized evidence based

protocols, particularly when the protocol is

appropriate for a number of areas, such as all intensive care units

CPOE system is a massive, institution-wide undertaking; sharing of

sets rather than reinvention optimal

“When order sets are implemented without

organization standards and clinical review or

inadequately maintained, they

become templates for efficiently practicing

outdated medicine on a widespread basis.”

(Bobb 2007)

Controversies Surrounding Order Sets

• A number of design features would increase the utility and safety of the care prescribed through order sets. Individual orders within order sets should be linked, if so desired by the client.

For example, drug A is to begin at time zero, and linked orders for drug B and drug C begin 4 and 8 hours after drug A.

When drug A is delayed by 2 hours, drug B and C are automatically moved back by 2 hours.

• This decreases the risk for error and amount of downstream re-work and is particularly useful for fully integrated EHRs with online electronic medication administration records.





Lecture a

Patient Controlled Analgesia





Lecture a

Linked orders should also prompt the clinician to discontinue all orders

originating from an order set when appropriate.

Patient Controlled Analgesia (PCA) orders typically include a number of

additional orders for patient monitoring, rescue medications, and medication used to treat side effects. When the PCA order is discontinued, the user should be asked and given

the ability to discontinue all associated orders with a single click.

Examples of CPOE Design Features





Lecture a

Linked orders in an order set should be mutually exclusive (esp. high-risk drug orders)

Multiple choices but exclusion of alternates after choice

logic to present only those options appropriate for the patient (e.g. ‘dithered’ for allergy, unavailable for wrong sex)

Standards (based on the literature, local consensus, and institution-specific drug formulary) for common care elements such as surgical antimicrobial prophylaxis, deep vein thrombosis (DVT) prophylaxis, glucose management, post-operative nausea

No Default Selections

Bobb, et al. (2007).





Lecture a

Other Design Requirements





Lecture a

Overall template design and details:

• Consistent naming conventions that facilitate finding

• No default selections when set opened

• Pre-selected orders ok for nursing and lab orders where the same treatment is prescribed for virtually all patients

• Set location: e.g. set name, service name, procedure

Order Set Safety





Lecture a

Author can explain intent

Misinterpretation: at best -> re-work

At worst -> error and patient harm

Multidisciplinary review important esp. prior to clinician CPOE experience can discuss changes to procedures, standardization, items not suitable to electronic formats

Review and Supervision





Lecture a

Review by committees; new after deployment, for set removal when necessary

Input from established oversight committees such as Pharmacy and Therapeutics, Critical Care, Blood Transfusion and Quality Committees.

Rapid advance of clinical knowledge poses problems

Overall strategy required

Basic building blocks of a decision support program

1. more/new work for clinician;

2. unfavorable workflow issues

3. never ending system demands

4. problems related to paper persistence

5. untoward changes in communication patterns and practices

6. negative emotions

7. generation of new kinds of errors

8. unexpected changes in the power structure

9. overdependence on the technology. Clinical decision support features introduced many of these unintended consequences

Identifying Unintended Adverse Consequences (UAC) can allow design to avoid negative consequences

Unintended Consequences of CPOE (Campbell et al. 2006)





Lecture a

Checklists – Gawande





Lecture a

Atul Gawande: ‘Checklist Manifesto’ 2010

Modern medicine very complex, changing, extensive expertise, unheard of accomplishments, but

Falls short due to lack of completeness, expert memory; checklists are a solution

Case: girl drowned under ice brought back to life; extensive procedures (100s) with dependencies

Building: another example of extensive complex procedures with dependencies; checklists used to coordinate work

Checklists – Gawande (cont.)





Lecture a

Resistance to checklists due to self-perception as expert

80% of surgeons agreed after using, saw it catch errors, 20% still resistant

‘would you want it if being operated on?’ –93% said yes

OR checklists based on aviation

Pronovost





Lecture a

Authorization for nurses to prompt if 5-item checklist not followed (for line infections)

Backup from administration critical

Infections dropped from 11% to zero in first year

2 line infections in subsequent 15 months

Prevented 43 infections, 8 deaths, saved $2mil.

Keystone initiative: involved executive visits to problem-solve in ICU; critical to obtaining resources

Nevertheless not implemented nationwide

Pronovost (cont.)





Lecture a

Had to change the way teams worked together and improve communication.

“Until a junior nurse can correct a senior physician who forgot to use the checklist, until that conversation goes well, we will continue to harm patients”

2006 NEJM paper: nearly eliminated infections in hospital in 3 months (to 0 from 2.7/1000); deployed statewide, saved 1500 lives first year, decreased infections 66%

Pronovost – UIC





Lecture a

Unintended consequences of CPOE can be good (e.g. ID unnecessary procedures) or fatal

Shabot -Ten Commandments for CIS

1. Speed is everything.

2. Realize that doctors won't wait for the computer's pearls.

3. Deliver “just-in-time” information.

4. Fit into the user's workflow.

5. Respect physicians' sense of autonomy.

6. Monitor implementation in real time and respond “right now.”

7. Beware of unintended consequences.

8. Be wary of uncovering long-standing process flaws.

9. Don't disrupt “magic nursing glue.”

10. Speed is everything.





Lecture a

Additional Reference: Top 10 Sentinel Events (reviewed by JCAHO

2008) by type

Event # reviewed in 2008

Wrong-site surgery 116

Suicide 102

Delay in Treatment 82

Unintended retention of foreign body 71

Patient fall 60

Operative/Post-operative complication 63

Medication error

Assault/rape/homicide

Perinatal death/loss of function

Medical equipment-related

46

41

32

23

1.1 Table: JCAHO, (2008).





Lecture a

Designing for Safety Summary – Lecture a

• Patient Safety

• Error

• Design Requirements

• Ten Commandments of Clinical

Information Systems (CIS)

• Patient Control Analgesia





Lecture a

Designing for Safety References – Lecture a

References:

1. The origin of primum non nocere." British Medical Journal electronic responses and commentary. Retrieved on 1

September 2002 from http://en.wikipedia.org/wiki/British_Medical_Journal.

2. To Err is Human: building a safer health system. (1999). Institute of Medicine Report. Retrieved on September 8th,

2010 from http://iom.edu/~/media/Files/Report%20Files/1999/To-Err-is-

Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf.

3. Han, YY, Carcillo, JA, Venkataraman, ST, Clark, RSB, Watson, RS, Nguyen, TC, Bayier, H., Orr, RA Unexpected

Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry

System. Pediatrics Vol. 116 No. 6 December 1, 2005 pp. 1506 -1512.

4. Sittig, D.F. (2009). Eight rights of safe electronic health record use. JAMA,vol.302(10), p.1111-1113.

5. Ash, J.A., Sittig, D.F., Dykstra, R., Campbell, E., Guappone, K. (2009). The unintended consequences of

computerized provider order entry: findings from a mixed methods exploration. International Journal of Medical

Informatics, vol.78 (S1), p.S69-S76.

6. Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical

decision support in computerized provider order entry. Journal of the American Medical Informatics Association,

Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47.

7. Shabot, MM. Ten commandments for implementing clinical information systems. Proc (Bayl Univ Med Cent). 2004

July; 17(3): 265–269.

8. Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical


Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47.

9. Peter Pronovost, Dale Needham, Sean Berenholtz, David Sinopoli, Haitao Chu, Sara Cosgrove, Bryan Sexton,

Robert Hyzy, Robert Welsh, Gary Roth, Joseph Bander, John Kepros, Christine Goeschel. N Engl J Med 2006;

355:2725-2732December 28, 2006.





Lecture a

http://en.wikipedia.org/wiki/British_Medical_Journal

http://iom.edu/~/media/Files/Report Files/1999/To-Err-is-Human/To Err is Human 1999 report brief.pdf


References (cont.):

10. Gawande A. (2007). The checklist. Retrieved on September 10th, 2010 from The New Yorker, December 10

2007. Available at http://www.newyorker.com/reporting/2007/12/10/071210fa_fact_gawande?currentPage=2.

11. Elrod J, Androwich IM.(2009). Applying human factors analysis to the design of the electronic health record. Stud

Health Technol Inform; 146:132-6.

12. Kaye R. (2010). Enhancing User Performance and Avoiding Safety Problems through Analysis, Discovery,

Prioritization and Design Considering usability for Health IT systems from a safety & effectiveness

perspective. National Institute of Standards and Technology Health IT Workshop, Gaithersburg MD. July 13, 2010.

13. Campbell, ME., Sitting, D.F., Ash, J.S., Guappone, K.P. (2006). Types of unintended consequences related to

computerized provider order entry. Journal of the American Medical Informatics Association, vol.13(5), p.547-556.

14. Shabot, MM. Ten commandments for implementing clinical information systems. Proc (Bayl Univ Med Cent). 2004

July; 17(3): 265–269.

15. Joint Commission on the Accreditation of Hospital Organizations (JCAHO). A Guide to The Joint Commission's

Medication Management Standards, Second Edition (PDF book). http://www.jcrinc.com/e-

books/EBMMS02/2100/ available at http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F-

5DA4A87B02F2/0/se_stats_trends_year.pdf.

Images

Slide 19: Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical


Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47





Lecture a

http://www.newyorker.com/reporting/2007/12/10/071210fa_fact_gawande?currentPage=2

http://www.jcrinc.com/e-books/EBMMS02/2100/

http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F-5DA4A87B02F2/0/se_stats_trends_year.pdf


Table:

1.1 Table: Joint Commission on the Accreditation of Hospital Organizations (JCAHO). A Guide to The Joint

Commission's Medication Management Standards, Second Edition (PDF book). http://www.jcrinc.com/e-

books/EBMMS02/2100/ available at http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F-

5DA4A87B02F2/0/se_stats_trends_year.pdf.





Lecture a

http://www.jcrinc.com/e-books/EBMMS02/2100/

http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F-5DA4A87B02F2/0/se_stats_trends_year.pdf

usability & human factors - remote learner · 15min-1hr if patient was in extremis •entering...

Documents