implementing a new machine learning algorithm to predict
TRANSCRIPT
Implementing a new Machine Learning Algorithm to Predict
CRC- MHS Roll Out of MeScore
Ran Goshen, MD, Ph.D., Chief Medical Officer,
Medial EarlySign Ltd.
Possible Conflict of Interests
Ran Goshen is the Chief Medical Officer of Medial EarlySign Ltd (MES). MES is the developer of MeScore used by Maccabee
Health Services (MHS). MHS pilot implementation of MeScore will be described.
Ran Goshen
Our Goal for Today
• Brief background on MeScore, a machine learning based
algorithm, designed to flag individuals at risk of harboring
colorectal cancer
• Describe the implementation of MeScore within a large
HMO
• Discuss preliminary results
• Learn from you on your thoughts and plan ahead
MK
G-1
4-0
1-0
1-A
ColonFlag
• Based on:
Standard CBC values + age + gender
• Identifies trends and variations within
the normal ranges as well as exceptions
from the norm
• Provides a simple and easy to use
personal risk indication
MeScore No Flag Red Flag
Expedited Assessment
MeScore- A computed relative risk indicator
Follow Guidelines
1.2
1
0.8
0.6
0.4
0.2
0
-0.2
-1.5 -1 -0.5 0 0.5 1 -1.5 -1
1.2
1
0.8
0.6
0.4
0.2
0
-0.2
-0.5 0 0.5 1
Machine Learning vs. Regression Modeling
Machine Learning Regression Modeling
Clinical Rigor
* Maccabi Healthcare Services: IHCO covering 25% of Israel’s population (~2M individuals) ** The Health Improvement Network
Blindly validate the model (Validation set) 139,205 people
(698 CRC patients)
Blindly validate on independent data-set 25,613 people
(5,061 CRC patients)
Train & develop model (Derivation set) 466,107 people
(2,437 CRC patients)
THIN** (UK)
MHS* (Israel)
Create model
Check model performance on “blind” part of the same
population
Check model performance on different population
1
2
3
Worldwide Validation 4 Worldwide
Ongoing trials in leading institutions around the world
totaling nearly 5M individuals
2nd largest Integrated Healthcare Provider, serving
25% of the Israeli population (over 2 M
covered lives)
Fully integrated healthcare, 100%
penetration of EMR for 3 decades, single
centralized MegaLab
1615 GPs, 110 gastroenterologists, 9 GI
centers, performing 63,000 colonoscopies
annually
Maccabi Healthcare Services - Overview
MeScore Implementation at MHS
• Perform case finding on individuals not up to date with their CRC
screening.
• Extract the outpatient CBC data and demographics of non-
responders to CRC screening, who performed their last outpatient’s
blood count during the last working day.
• Run MeScore on the current (and historical, when available) CBC
test results data of the selected individuals.
• Flag in the GP EMR system individuals above a pre-defined
threshold.
• GPs are trained to refer their MeScore flagged patients as they do
with their FIT positive patients directly to colonoscopy
MK
G-1
4-0
1-0
1-A
MHS Inclusion Criteria
• 50-75 age range
• Outpatient CBC test was performed during the last day
• Has not been treated for cancer during the last 12 months
• Did not perform a FIT/gFOBT test in past 18 months
• Did not perform a colonoscopy in past 10 years
• Is not currently scheduled to perform CRC screening test
• Did not visit a gastroenterologist in past 3 months and is not
scheduled to see gastroenterologist during the coming 3 months
MK
G-1
4-0
1-0
1-A
Blood Count Ordered
Blood Count Performed and
Processed
Age 50-75?
Screening Up-To Date?
Under GI Examination
?
Lab Results
Report Result to GP and Smart
Alerts
Above Cut-Off ?
Calculate ColonFlag
New Results Blood Count
Referral
LAB SYSTEMS
EHR SMART ALERTS
Yes
No No
CS+
MHS Implementation Flow
Create Smart Alert and Reminders
Yes
ColonFlag+
MeScore Implementation Data Flow
CBC
Database
Extract CBC data for patients that
meet criteria
Transfer data to MeScore Server located
on customer site
MeScore software calculates MeScore
Transfer MeScore results back to main database
Alert GP
Population Management
Transfer flagged patients results
per cutoff to GPs
MK
G-1
4-0
1-0
1-A
MHS implementation numbers
534 (~1%)High scores (≥ 99.6)
109* under GP supervision (21%)
56 have not visited GP yet (10%)
369 referred by GP (69%)
* Will not be referred to cs.
79 referred to CS. (21%)
290 * referred to gastro (78%)
106 have not visited the gastro yet (36%)
170* referred to CS. (59%)
14 under gastro supervision (5%)
* If there is referral to CS. or CS. was performed without referral from GP, our assumption is that the patient was referred to/by gastroenterologist.
162 performed CS. (65% compliance)
Total ‘high scores’ in MHS Mid 10/2015-08/2016 (10.5 months)
249 referred to CS.
• 47% of “high scores” are referred to colonoscopy • 30% of high scored patients performed colonoscopy (compliance to CS. is 65%)
~ 60,000 scores
Colonoscopy findings – Castells’ reference table
Reference: Postpolypectomy surveillance in patients with adenomas and serrated lesions: a proposal for risk stratification in the context of organized colorectal cancer-screening programs. Castells A, Andreu M, Binefa G, Fité A, Font R, Espinàs JA. Endoscopy. 2015 Jan;47(1):86-7.
(19) (5)
(11)
(17)
(14)
(90)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1
Portion of colonoscopy findings classification
Cancer High Risk
Intermidiate risk Low risk
No risk Clean colonoscopies
Colonoscopy findings
• Total number of colonoscopies: 162 • Unknown results: 6 • Total colonoscopies with known results: 156
• Total colonoscopies with findings: 66 • Clean colonoscopies: 90
(66)
(90)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1
Portion of colonoscopy results
Colonoscopies with findings Clean colonocopies
Medial EarlySign at a Glance
2009 2011
2010 2012
2016 2014
2015
2013
Development of first models Collection of Data
Initial model for CRC
2nd generation CRC Validation in UK
>1M Patient Records Initial model for Lung Cancer Models for real-time ICU
Predictor Engines Discovery workbench
Initial Models for Diabetes, Renal Failure Installation in MHS Peer-reviewed publication
EU Installations U.S., Asia validations GI Modelling >17M records
Company founding
for ColonFlag