Caprice C. Greenberg, MD, MPH, FACS Associate Professor of Surgery

WARF Professor of Surgical Research Director, Wisconsin Surgical Outcomes Research (WiSOR)

University of Wisconsin - Madison

Systems Engineering, Fieldwork and Point-of-Care Research

Take Home Points

• Research at the point of care is critical to improve quality and safety

• A systems view of healthcare is necessary to understand complexity in the work environment

• Simple 6 step process can get you started

Defining Patient Safety

• NPSF – The avoidance, prevention, and amelioration of adverse outcomes or injuries stemming from the processes of healthcare

• IOM - Safety does not reside in a person, device or department but stems from the interactions of components of a system

* IOM, To Err Is Human, 1999

Traditional Thinking in Medicine

Patient Characteristics

Outcomes

Good Outcome Unsafe Process

Time and Progression of Clinical Course or Treatment

Dec

reas

ing

Pat

ient

Saf

ety

Time and Progression of Clinical Course or Treatment

Dec

reas

ing

Pat

ient

Saf

ety

Time and Progression of Clinical Course or Treatment

Dec

reas

ing

Pat

ient

Saf

ety

Baseline Clinical

State

Good Outcome Safe Process

Bad Outcome Unsafe Process

Time and Progression of Clinical Course or Treatment

Dec

reas

ing

Pat

ient

Saf

ety

Bad Outcome Safe Process

Baseline Clinical

State

Baseline Clinical

State

Baseline Clinical

State

Operations/QI v. Research

Six Steps

1 Finding collaborators 2 Identifying a conceptual framework 3 Data collection and sampling strategies 4 Approaches to analysis 5 Presenting results 6 Dissemination and implementation

FINDING COLLABORATORS Step 1

Engineering

• Industrial Engineering

• Systems Engineering

• Human Factors Engineering

Psychology

• Organizational Psychology

• Cognitive Psychology

IOM

• Engineering has the potential to improve both efficiency and quality

• Continuous improvement principles can lead to more efficient, safer, and higher quality

CONCEPTUAL FRAMEWORKS Step 2

A system is a complex assembly of people, information, resources, equipment and procedures working toward a common goal

Vincent, (2004) Annals of Surgery

James Reason

Donabedian Quality

STRUCTURE

PROCESS OUTCOME

SEIPS

Systems Engineering Initiative for Patient Safety

1. Adds employee and organizational outcomes

2. Specifies relationships between patient and employee/organizational outcomes

3. Includes other processes besides care/clinical

4. Expands to comprehensive definition of “structure”

SEIPS

Our Conceptual Framework

Baseline Risk Profile

Patient Outcome

Individual provider

performance

Team performance

System / organization performance

Expected course

Vulnerability

Resilience

Time

DATA COLLECTION AND SAMPLING Step 3

Types of Data

• Quantitative: anything that can be measured in terms of quantity, amount, intensity, or frequency.

• Qualitative: qualities, processes and meanings of things

• Mixed Methods

Complementary Approaches

• Medical record

• Floor plans to assess flow

• Protocols and policies

• Position descriptions to determine roles

Existing Documentation

• Interviews – Structured – Semi-structured

• Focus Groups

• Questionnaires / Surveys

– Likert scales – Open-ended questions

Frontline Providers

• Ethnography • “Shadowing” • Time and motion studies

• Critical considerations:

– Data collection tools – Content v. domain experts – Training of observers – Sampling strategy

Observational Field Studies

Audio - Video Capture

Video v. Live Observation

• Complete record

• Extract, count, correlate as salient issues evolve

• Larger number of observers

• Self-evaluation and reflection

Limitations to Video • Expensive

• Can be intrusive

• Can limit range of settings

• Analysis is time-consuming

• Discoverability, confidentiality, privacy

• Requires a cultural shift

APPROACHES TO ANALYSIS Step 5

Examples • Systems analysis

• Time and motion analysis

• Root cause analysis

• Failure modes and effects analysis

• Statistical process control charts

Usability Studies

RATE

• Event Tracker List – Time stamps events intrinsic to every

operation • Event Tracker Counter

– Environmental disruptions/interruptions – Start/stop times for specific events

• Flag performance or factor examples * Guerlain S, Shin T, Guo H, Calland JF (2002) Team performance data capture and analysis system. In: Proceedings of HFES 46th Annual Meeting, Baltimore

MVTA

Motion Analysis

Identify objects of interest

Image source: Mayo SPARC

• Identify objects • Code behaviors • Temporal coding • Analyze

sequences • Link with

outcomes

Montague, E., *Xu, J., *Asan, O., *Chen, P., Chewning, B., Barrett, B., (2011) An illustration of modeling eye gaze patterns in doctor-patient interaction with lag sequential analysis, Human Factors: The Journal of the Human Factors and Ergonomics Society, 53(5), 502-16.

PRESENTING RESULTS, DISSEMINATION AND IMPLEMENTATION

Step 5 and 6

Convey Your Message

• Quantitative – Descriptive statistics – Counts – Tables and figures

• Qualitative – Illustrative examples and quotes – Compare and contrast cases – Diagrams

DISSEMINATION IMPLEMENTATION

Goal Active spread of information Active intervention

Characteristics Less structured More theory and frameworks exist

Level Policy Institution and provider

Target Community and population Specific individuals and small groups

* Adapted from Ross Bronson. Training Institute for D& I Research in Health

STUDY EXAMPLES

Study #1: Protecting Patients from an Unsafe System

Deviations and Resilience

in the OR

Hu et al., Annals of Surgery (2012); 256(2): 203 – 210

Pat

ient

Saf

ety

Time and Progression of Operation

Baseline •Patient clinical factors

•Provider factors

Type 2: Increased

vulnerability

Type 1: Forward

progress halts

Patient Enters OR

Patient Leaves OR

Patient’s Clinical Course

Safety Compromised

Recovery

Resilience

Vulnerability

• Total of 24 deviations identified in 8 cases – 5 delays (Type 1) – 14 safety-compromising (Type 2) – 5 combined Type 1 & 2

• One every 1.5 hours of operative time

Quantitative Results

Description: • 16 minutes after incision, a beeping noise

starts. It lasts about 10 minutes. • All anesthesia and nursing personnel become

absorbed in trying to figure out what’s beeping.

• Unclear if it is a monitor or malfunctioning equipment

Qualitative Results

Quantitative Results

Contributing Factors % Involving

Mitigating or Compensatory Factors

% Involving

Individual Provider

Knowledge/Training Leadership

25 17

Monitoring/Vigilance Knowledge/Training Decision-Making Leadership

63 17 17 38

Team Communication Coordination

33 46

Communication Coordination Cooperation

50 33 33

System Equipment (Failure) Organizational Structure

17 29

Resilience

• The ability to anticipate, cope with, recover and learn from unforeseen challenges

• Understand gaps in system operation and expert strategies to bridge – Recognition of variation in workload during a case and

willingness to forego poorly timed breaks – Interim count at timing of handoffs – Competing goals resolved with cross-disciplinary

communication and collaborative problem solving

* Nemeth, Wears, Woods, Hollnagel, Cook. Minding the Gaps: Creating Resilience in Healthcare

Approach to Education

Error Prevention • Current approach to

improving surgical safety

• Erratic people degrade an otherwise safe system

• Increased standardization and tighter coupling

Resilience • Alternative approach

based on error mitigation

• Practitioners primary source of resilience in imperfect system

• Requires adaptability and flexibility

* Patterson, Woods, Roth, Cook, Wears, Render. J Patient Safety (2006) 2(1): 1-6

Study #2: A Prospective Study of Patient Safety in

the Operating Room

Christian et al. Surgery (2006); 139: 159-173.

Emerging Themes

• Communication and Information Flow – Generalized vulnerability

– Multiple points in the peri-operative period

– Significant impact on case progression

• Workload and Competing Tasks – Poor synchronization with case progression

– Counting introduced strain on the system

SNT-SNC1C2C3

[2][1]

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Counting Protocol

SCR

Repeat Counting Protocol

[2] [2] [1] [1] [2] [2] [1] [2] [2]

Time(min)

[1,2] [1,2] [1,2][1,2]

[1,2][1]

Near Miss Near Miss

Temporal Relationships

Study #3: Validating risk-adjusted surgical

outcomes: Site visit assessment of process and structure

Daley et al., JACS (1997); 185:341-51

Study Overview

• Validation of NSQIP included site visits

• Structured protocol for data collection

• Observers were dept. chiefs, RN, MD

• Identify characteristics of outlier status

• Predict high vs. low outlier status

• Exit interview to member-check results *

NSQIP Qualitative Results

• High outliers had: – Less spacious physical layout

– Higher per diem RN staffing & lower skill level

• Low outliers had: – Strong relationship with medicine and cardiology

– More interaction between surgeons and RNs

– More clinical pathways and practice guidelines

NSQIP Results and Implications

• Correctly identified 17 of 20 institutions as high or low outlier status

• Convincing evidence that differences exist in the quality of surgical care

• Risk-adjusted outcomes reliably discriminate quality of care

Take Home Points

• Research at the point of care is critical to improve quality and safety

• A systems view of healthcare is necessary to understand complexity in the work environment

• Simple 6 step process can get you started

Thank You

greenberg@surgery.wisc.edu