implementation of an innovative machine learning-based ... · t 9/2015 ebackground, not visible to...
TRANSCRIPT
![Page 1: Implementation of an Innovative Machine Learning-Based ... · t 9/2015 ebackground, not visible to users-triage built into EHR 10/2015 e-triage visible in EHR to key informant nurses](https://reader035.vdocuments.us/reader035/viewer/2022070904/5f6dc25368b9fa218f10621b/html5/thumbnails/1.jpg)
Implementation of an Innovative Machine Learning-Based Triage Support Tool: Translating Technology and Research to PracticeMadeleine Whalen RN, MSN/MPH CEN1; Heather Gardner RN, MSN1; Diego Martinez, PhD2; Sophia Henry RN, MS1;
Catherine McKenzie BSN, RN, CEN1; Jeremiah Hinson MD, PhD2; Scott Levin PhD2
Purpose
Methods
Results
Large scale practice change is possible, yet requires high levels of end-user engagement, including a mutually-educational relationship between the users and support technology. Trends in overrides related to chief complaint and subsequent outcomes can continue toinform the implementation and development of the triage tool. This transition exemplifies the translation of informatics research to practice as well as the potential of nurses to use the EHR and machine learning to improve healthcare.
Implications
References
This is a prospective cross-sectional study in level one, urban, academic trauma center with approximately 70,000 annual patient visits. Participants included all practicing triage nurses. Over one year, we incrementally rolled out a new e-triage system. E-triage is a site-specific, clinical decision support system developed from the ED’s patient population that does not replace nursing assessment, but provides a triage level suggestion based on computed risk of several acute critical outcomes: mortality, intensive care unit (ICU) admission, and emergent procedure. Statistical evaluation of e-triage included quantifying: nursing uptake, agreement with e-triage level, and patterns of nurse overrides.
Ap
plic
atio
n D
evel
op
men
t 9/2015 e-triage built into EHR background, not visible to users
10/2015 e-triage visible in EHR to key informant nurses
1/2016 Meeting with biomedical engineer and key informant nurses for feedback and algorithm adjustments
3/2016 e-triage introduced to all triage nurses, e-triage visible to all nurses to consider and provide feedback
Imp
lem
enta
tio
n 10/2016 e-triage initiated for acuity level 4 and 5 patients
12/2016 e-triage initiated for all acuity levels
Pro
spec
tive
Eval
uat
ion 12/2016 continuous
data monitoring and analysis
3/2017 Vital Signs algorithm updated based on nurse feedback
Results
Total agreement 80%; 51-55% for high acuity (level 1 & 2), 83% for mid-acuity (level 3), and 72% for low acuity (level 4 & 5) patients.
E-triage and ESI comparison
E-triage implementation timeline
Leveraging the electronic health record (EHR), as well as translating research to clinical practice, are paramount to improving the provision of healthcare in the US (NIH, 2006; AHRQ, 2017).
The nurse-driven, often subjective, triage process can greatly benefit from improved use of technology. The use of an innovative machine-learning clinical decision support tool (e-triage) has been shown to reliably and
accurately triage patients by sorting patients based on likelihood of critical events (Dugas, 2016). The objective of the study is quantify and qualify our evolution from the traditional ESI
triage system, toe-triage as anexemplar ofsuccessful transitionfrom research topractice and nursingintegration ofdata-driven clinicaldecision support.
Pre/Post Patient Differentiation Across Low, Mid and High Acuities
5.31%
3.33%
14.08%
6.25%
9.79%
12.86%
10.08%
18.06%
20.89%
85.87%
84.71%
80.40%
86.62%
61.02%
77.32%
69.42%
73.75%
68.99%
8.82%
11.96%
5.52%
7.13%
29.18%
9.81%
20.50%
8.19%
10.13%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Abdominal Pain (4749)
Chest Pain (3787)
Shortness of Breath (2301)
Headache (1599)
Eye Problem (1542)
Back Pain (1508)
Motor Vehicle (1200)
Leg Pain (1124)
Fall (1106)
Ch
ief
Co
mp
lain
t (N
)
downgrade % agree % upgrade %
Percent Nursing Agreement with e-triage Among 10 Most Frequent Chief Complaints
Chief Complaints with Most Agreement Frequency / N Percentage Agreement
Headache 1385 / 1599 86.6%
Sickle cell 825 / 955 86.4%
Emesis 746 / 865 86.2%
Abdominal pain 4078 / 4749 85.9%
Chest pain 3208 / 3787 84.7%
Chief Complaints with Most Upgrades Frequency / N Percentage Upgraded
Ingestion 242 / 572 42.3%
Seizure 194 / 551 35.2%
Suicidal ideation 349 / 1036 33.69
Eye problem 450 / 1542 29.2%
Alcohol intoxication 133 / 514 25.9%
Chief Complaints with Most Downgrades Frequency / N Percentage Downgraded
Post-op problem 88 / 385 22.9%
Referral 134 / 609 22%
Fall 231 / 1106 20.9%
Foot pain 141 / 680 20.7%
Leg pain 203 / 1124 18.1%
Percentage of Highest Agreement, Upgrades and Downgrades by Chief Complaint
Nurses tended to downgrade (N=33) patients complaining of abdominal pain for reasons such as vital signs (n=13), nursing visual assessment(n=5) and lack of associated symptoms (n=6). Most upgrades (N=126) were attributed to “protocol,” (n=49) past medical history (n=14), andpresence of associated symptoms (n=14).
1Department of Emergency Nursing, Johns Hopkins Hospital; 2Department of Emergency Medicine, Johns Hopkins University School of Medicine; Baltimore, MD USA
Acuity 1
E-Triage
High risk of critical event
ESI
Immediate life saving
intervention
Acuity 2
E-Triage
Elevated risk of critical
event
ESI
High risk situation
Acuity 3
E-Triage
Moderate risk of
admission
ESI
Many resources
Acuity 4
E-Triage
Low risk of admission
ESI
One resource
Acuity 5
E-Triage
Very low risk of admission
ESI
No resources
E-triage and Nurse Assigned Acuity Agreement