hit hazard manager: for proactive hazard control james walker md, principal investigator, geisinger...

HIT Hazard Manager: for Proactive Hazard Control

James Walker MD, Principal Investi gator, Geisinger Health SystemAndrea Hassol MSPH, Project Director, Abt Associates

September 10, 2012

AHRQ Contract: HHSA290200600011i,#14

Accident Analysis

“Most reporting systems concentrate on analyzing adverse events; this means that injury has already occurred before any learning takes place.”

DeRosier, et al. (2002) Using Health Care Failure Mode and Effect Analysis. JC Journal on Quality Improvement. 28(5):248-269.

Patient Harm

Analysis (e.g., RCA)

Accident Analysis

Near-Miss Analysis

“Most reporting systems concentrate on analyzing adverse events; this means that injury has already occurred before any learning takes place. More progressive systems also concentrate on analyzing close calls, which affords the opportunity to learn from an event that did not result in a tragic outcome.”

DeRosier, et al. (2002) Using Health Care Failure Mode and Effect Analysis. JC Journal on Quality Improvement. 28(5):248-269.

Patient Harm

Near Miss

Analysis (e.g., RCA)

Near-Miss Analysis

Proactive Hazard Control

“Most reporting systems concentrate on analyzing adverse events; this means that injury has already occurred before any learning takes place. More progressive systems also concentrate on analyzing close calls, which affords the opportunity to learn from an event that did not result in a tragic

outcome. Systems also exist that permit proactive evaluation of vulnerabilities before close calls occur.”

DeRosier, et al. (2002) Using Health Care Failure Mode and Effect Analysis. JC Journal on Quality Improvement. 28(5):248-269.

”Un-Forced” HIT-Use Error

Error in Design or Implementation

Interaction between HIT and other healthcare systems

Proactive Hazard Control

Care-Process Compromise?

Identifiable Patient Harm?

Patient Harm

No Adverse Effect Near Miss

Hazard in

Production

No Adverse Effect

Yes

Yes

HIT-Related Hazards

Yes Yes

NoNo

No

No

Use-Error Trap

Hazard Identified?

Hazard Resolved?

HIT-Use-Error Trap

Proactive Hazard Control: A Case

Pre-implementation Analysis: New CPOE cannot interface safely with the existing best-in-class pharmacy system.

Replace the pharmacy system with the one that is integrated with the CPOE: Expensive 9-month delay

Years later, David Classen studied 62 HER implementations and concluded that CPOE and pharmacy systems from different vendors can never be safely interfaced.

The Hazard Ontology

Why a standard language (ontology) for HIT hazards?

To decrease the cost and increase the effectiveness of hazard control.

Example: Much of the budget of the Aviation Safety Information Analysis and Sharing (ASIAS) system is devoted to normalizing data—because every airline uses a different ontology and can’t afford to change.

Health It Hazard Manager – AHRQ ACTION Task Order

Design & Alpha-Test (266 hazards)– Geisinger

Beta-Test (Website) – Geisinger– Abt Associates – ECRI PSO

Beta-Test Evaluation– Abt Associates– Geisinger

Hazard Manager Beta-Test

7 sites: integrated delivery systems, large and small hospitals, urban and rural

– Usability (individual interviews)– Inter-rater Scenario Testing (individual web or in-

person sessions)– Ontology of hazard attributes (group conference)– Usefulness (group conference)– Automated Reports (group conference)

4 vendors offered critiquesAll-Project meeting: 6 test sites, 4 vendors, AHRQ, ONC, FDA

HIT Hazard Manager 2.0Demo

Hazard Ontology

Discovery: when and how the hazard was discovered; stage of discovery

Causation: usability, data quality, decision support, vendor factors, local implementation, other organizational factors

Impact: risk and impact of care process compromise; seriousness of patient harm

Hazard Control: control steps; who will approve and implement the control plan

Beta-Test Analytic Methods

Content analysis of 495 Short Hazard Descriptions

Frequencies of hazard ontology factors: combinations often selected together; factors never selected

Inter-rater differences in entries of mock hazard scenarios/vignettes

Suggestions from testers to improve ontology clarity, comprehensiveness, mutual exclusivity

Content analysis of “Other Specify” entries

Example: Unforced User Error

Unforced User Error was the second most frequently chosen factor (79 hazards).

In 55 instances, another factor was also chosen:

UsabilityData

QualityCDS

Software Design

Other Org. Factors

22 9 12 9 33

* Multiple selections possible

Inter-rater testing revealed differing attitudes about the role of health IT in preventing user errors.

Ontology Revision: “Use Error”

Use Error was often due to the absence of protections or safeguards to prevent errors:

Added a new factor to Decision Support: “Missing Recommendation or Safeguard”

Re-defined “Unforced User Error” as “Use Error in the absence of other factors”

Hazard Manager Benefits

Value: Care-Delivery Organizations

Prior to an upgrade, learn about hazards others have reported.

Identify hazards that occur at the interface of two vendors’ products.

Control hazards proactively. Estimate the risk hazards pose and prioritize

hazard-control efforts. Inform user-group interactions with vendors. Protect confidentiality.

Value: HIT Vendors

Identify the 90% of hazards that their customers do not currently report.

Learn which products interact hazardously with their own.

Prioritize hazard control efforts.

Identify hazards early in the release of new versions.

Preserve confidentiality.

Value: Policy Makers

Identify and categorize common hazards that occur at the interface of specific types of products (e.g., pharmacy and order entry).

Move hazard identification earlier in the IT lifecycle (especially prior to production use).

Monitor the success of hazard control in reducing health IT hazards and decreasing their impact on patients.

For more information:andrea_hassol@abtassoc.com

Beta-Test Final Report available on AHRQ website:

healthit.ahrq.gov/HealthITHazardManagerFinalReport