Clinical EngineeringEngineers in the Modern Academic Medical Center

Design DisastersConsequences of Blunders, Bad Luck, and Bias

Patrick Norris, Ph.D.Assistant Professor of Surgery,

Biomedical Engineering

patrick.norris@vanderbilt.edu

Clinical EngineeringWhy do hospitals need engineers?

• Definition• Past, Present, Future• Examples

– Facility Design– Biomedical Devices– Information and Technology Management

• Clinical Research, Quality Improvement

Definition

Biomedical Engineering

Biomedical Electronics

Clinical Technology Service

DefinitionThe American College of Clinical Engineering:

A professional who supports and advances patient care by applying

engineering and management skills to healthcare technology.

Definition: Hospitals need engineers when technology requires:

• Special (non-trade/craft skills) customization or maintenance

• Complex selection criteria• Modification of existing facilities or

systems, or special design of new ones

• Design and analytic skills, professional credentials, etc. differentiate engineers from technicians, craftspeople, clerical, administrators, etc…

Examples: Past• Einthoven: EKG, early 1900’s• Other examples:

– Day to day heat, AC, water, electricity, etc.

Examples: Present• Infrastructure Design

– Typical: Water, Electrical, HVAC, Telecom– Special: Medical Gas, Sample Handling– Structural: Imaging Systems

• Biomedical Devices– Selection, integration, tracking– Maintenance is becoming a sophisticated

trade/craft skill• Information

Future

Grimes SL, IEEE Engineering in Medicine and Biology Magazine, March/April 2003 p.91-99

• Information– Medical Informatics– 6 VUSE PhDs

• Integration– People– IT Systems– Medical Devices

• Regulation– Privacy, Safety,

Efficacy• Across Multiple

Healthcare Systems

Clinical Research• SIMON Project

– (Signal Interpretation and Monitoring)– Ongoing since 1994

• Seeks to Advance:– Medical Monitoring Technology– Critical Care– Scientific Knowledge

• Clinical Engineering Component

Trauma• 5th Leading Cause of Death (1st Under 45)• 8% of Medical Expenditures (rank: 3rd)• All Age and Socioeconomic Groups• VUMC

– Only Level 1 Facility, 65,000 Square Miles– 3500 Annual Admissions– 800 to Trauma ICU, ~10% Mortality

Patient Monitoring

• Cushing, early 1900’s:– Importance of Monitoring and

Recording Vital Signs• Technology Has Advanced• Fundamentally, Clinical

Strategies Remain Unchanged– Intermittent Recording– Manual Interpretation

Tools for

Dense Physiologic Data Management

Four Engineering Challenges• Data Collection

– Interfaces to a Variety of Devices– Remote Locations

• Storage– Clinical Applications - Short-Term– Research Applications - “Forever”

• Processing– Time-Critical Tasks (Clinical Decision Support)– Research Analysis

• Architecture– Integration, Reliability, Scalability, Flexibility…

SIMON Data Capture

• Philips CareVue– Routine, Automatic Vital

Signs Capture– HR, ABP, PAP, CVP, ICP,

CPP, PAP, SaO2

– Episodic Waveform Capture

• Edwards Vigilance– CI, EDVI, temp, SvO2, etc.

• Alaris IV Pump (near future?)

SIMON Data Storage

• Relational Database– Time Constraints w/ Limited Resources– Adaptive Sampling, ~0.25-1Hz Storage

• 5500+ TICU Patients– Reliably Identified, Linked to Outcomes

• 450,000+ Continuous Hours• Grows by:

– 2 Million+ Data Points/Day– ~70 Patients/Month

Daily Reports

Data Display

Alerts

• Effective Alerting– Right Information– Right Person– Right Time

• Process– Event– Alert– Notification– Response

SIMON Architecture

• Modular, Simple Components– Scalable– Reliable– Flexible

• Time-Constrained

SIMONS1SIMONT1

Digi Driver

Data Collection Modules (1 per device)

•••

Bed 1

Bed 2

Bed 3

Bed 14

Data Collector

SQL 2k

ODBC

Simon Packet Format

System Mgr.

Database Mgr.

Census AgentCensus Monitor

Event Engine

Alert Engine

Notify Engine

Report Engine

SIMONW1(Secure WWW Server)

trauma.mc.vanderbilt.edu

VUMC Census

VUMC StarPanel

sFTP

sFTP

Research Hypotheses

• Identify failure of communication pathways (uncoupling)

• Linking systems, organs, cells, proteins, and genes

• Illuminate underlying control mechanisms

• …especially in the critically ill

New measurements, available through techniques of dense data capture and analysis, will:

Short-Term HRV - SurvivalP

erce

nt T

ime,

ent

ire s

tay

5-minute HR Standard Deviation

00

0.5

0.5

1.0

1.5

1.0 1.5 2.0 2.5 3.0 3.5

N = 825

Short-Term HRV - DeathP

erce

nt T

ime,

ent

ire s

tay

00

0.5

0.5

1.0

1.5

1.0 1.5 2.0 2.5 3.0 3.5

N = 98

5-minute HR Standard Deviation

Short-Term HRV - Combined

Time normalizedwithin outcome

group

Per

cent

Tim

e, e

ntire

sta

y

00

0.5

0.5

1.0

1.5

1.0 1.5 2.0 2.5 3.0 3.5

N = 923

5-minute HR Standard Deviation

Design DisastersConsequences of Blunders,

Bad Luck & Bias

• What is a Design Failure?• Why Do They Happen?• Examples

– “Recipes for Design Disasters”– Space Program– Transportation– Medical

What is a Design Failure?

• Elements of Establishing Defect:– Identify the design defect– Establish a causal link to harm or cost– Identify alternate designs (correctable)– Compare to similar products

• “A product does not have a design defect when it is safe for any reasonably foreseeable use and meets all applicable functional specifications.”

Geddes, Medical Device Accidents With Illustrative Cases

Example Design Defect(probably from urban legend)

Nurses in Pelonomi Hospital, South African hospital were baffled that every Friday morning the patient in one particular bed would be found dead! Investigation revealed that the cleaning person would unplug that bed’s life support equipment, in order to plug in her floor polisher when she did the floors each Friday. When finished, she would plug the equipment back in unaware that the patient was now dead.

Example Design Defect

Definition• Identify defect

• Causal link

• Alternate designs• Comparison

Pelonomi Hospital Legend• Life support equipment

could be unknowingly unplugged

• Staff were not alerted when machine unplugged, patient died

• Alarms and batteries• All life-critical equipment

offered by vendors X,Y,Z have alarm & battery backup

What is a Design Failure?

• There are plenty of definitions• Numerous example cases• In the end, failures are debatable

– Ultimately, court may have to decide– With testimony from experts– Sometimes difficult to separate liability

from design flaw– Negligence is a legal, not technical, term

Why Do Designs Fail?At least three types of factors

• Blunders (Human Error)– “Everyone makes mistakes”

• Bad Luck (Random Effects)– “S*** happens”

• Bias– People sometimes believe what they

want to, irrespective of facts– Especially when money, power,

relationships are involved

Example: $125M Blunder• 1999 Mars Orbiter• JPL, Lockheed• Metric vs. English

units• Erroneous orbital

entry calculation – engine burn time

Example: Bad Luck (?)• Weather: A random effect• Dense fog on I-75

– 99 vehicle pile-up in TN – Killing 12, injuring 56

• Initially weather blamed• Then local paper mill

– $13.5M settlement– Once = bad luck– Many times = negligence?

Example: Bad Luck

• Tacoma-Narrows bridge• Unforeseeable consequence of

lightweight design, wind profile• No human deaths• $5.2M in 1940,

~$70M today• (Insurance paid)

Types of Bias

• “Statistical”– Sampling– Multiple comparisons– Repeated measurements

• Psycho-Social– Groupthink– “Corpthink”

Examples: Statistical Bias

• More people die in hospitals than anywhere else, therefore don’t go to the hospital! (unfair sampling)

• Similar situation: A medical device designed only for the critically ill

• Randomized, controlled trials are part of the answer

Examples: Statistical Bias

• Suppose you design a device that will roll a six every time – how many times do you need to test it?

• Which results do you report?• Increasingly an issue in medical drug

and device trials• 95% significance (p<.05) means that 1

in 20 studies is a false-positive

Psycho-Social Biases

• Individual– Primacy: The first option mentioned

seems best– Recency: The last option seems best

• Group– Groupthink: Consensus rules– “Corpthink”: Desire to please those

higher in the chain of command

NASA: Ripe for Disaster?• Huge shift in corporate culture

– Space race: Do it at any cost– Increasing cost concerns, cuts,

downsizing, resource pressure, etc.• Feynman, Challenger Disaster Report

– Engineer estimate of catastrophic failure: 1 in 100

– Management: 1 in 100,000– “What is the cause of management’s

fantastic faith in the machinery?”

More Design Failures

• “Recipes for Disaster”– Ignition Source + Flammable Material– …

• More Examples– Transportation– Space Program– Software

Hindenburg

• German airship• Caught fire while

landing in 1937• Design defect:

– Hydrogen?– Skin?

http://www.youtube.com/watch?v=F54rqDh2mWA

Apollo 1

• Pad fire during test• Killed 3 astronauts• Design defects:

– 31 miles of electrical wire– Pressurized pure oxygen

environment– Flammable materials– Substandard wiring

Medical Devices & Fire

Ignition Source• Electrocautery• Nerve stimulators• Short-circuit• Electrostatic discharge• Cigarettes• …

Flammable Materials• Anesthetic gas

– not so much today, ex. O2• Gases in the body,

especially GI system– Geddes reports ~10 cases of

GI explosions during procedures, some lethal!

• Bedding, clothing• Bandages• Cleaning solutions,

solvents, etc.

Medical Software Design

• What type of [medical] technology is least regulated?– Software– There is no professional-level (i.e. PE)

certification for software engineering– Less regulation than devices/drugs

Medical Software Design

• Design failures are being publicized• Computerized Physician Order Entry

– Cedars-Sinai software rollout– Multi-million dollar project scrapped– Software “endangered patient safety”– This story is not unique

• Privacy issues• Will software design failures increase?

Summary – Clinical Engineering

• Definition of clinical engineering• Engineers’ role in the hospital?

– Technology design, management– Increasingly, information management– Clinical research, i.e. VUMC Trauma

• Differences between engineering and trade/craft skills (design & analysis)

Summary – Design Disasters

• Geddes definition of design failure:– Identified defect– Causal link to harm– Available alternative– Deficiency w/ respect to other products

• 3 factors in design disasters:– Human error (blunder)– Random effects (bad luck)– Bias

Sample Questions

Which is not an aspect of establishing design failure (according to Geddes)?

A. Identified defectB. Causal link to harmC. NegligenceD. Feasible alternative

design

What factor best differentiates engineers from trades/craftspeople?

A. Design and analytic skillset

B. Professional ethicsC. Ability to work in

highly regulated fieldsD. Salary

Sample Questions

What kinds of bias is most likely encountered by an individual doing statistical analysis of complex data?

A. Unfair samplingB. GroupthinkC. RecencyD. All of the above

According to Feynman’s appendix to the Challenger disaster report, NASA engineers estimate probability of failure at about 1 in ________, compared to management’s 1 in ________ .

A. 10, 10000B. 1000, 1000C. 100, 100000D. 10000, 100

References/Sources• clinicalengineering.duhs.duke.edu/• cms.clevelandclinic.org/anesthesia/body.cfm?id=124• www.healthsystem.virginia.edu/internet/clinical-eng/• www.wikipedia.org• www.ceasa-national.org.za/• www.mc.uky.edu/clinicalengineering/• cms.clevelandclinic.org/anesthesia/body.cfm?id=156• www.uams.edu/ClinEng/default.aspx• simon.project.vanderbilt.edu/• tafkac.org/medical/hospital_cleaning_lady.html• www.cnn.com/TECH/space/9909/30/mars.metric.02/• mars.jpl.nasa.gov/msp98/orbiter/• www.douglasjfeeslaw.com/achievements.jsp• gtresearchnews.gatech.edu/reshor/rh-ss01/fog.html• www.ralentz.com/old/space/feynman-report.html• youtube.com

patrick.norris@vanderbilt.edu