phone as a sensor technology: mhealth and chronic disease
Post on 23-Dec-2014
93 Views
Preview:
DESCRIPTION
TRANSCRIPT
> slide to unlock
eric c. larson | eclarson.com
phone-as-a-sensor technology:
Assistant Professor Computer Science and Engineering
mhealth and chronic disease
MiPhone 9 Now with iColon.
> slide to unlock
Comp Sci. & Engr.
Comp Sci. & Engr.
databases & data mining
algorithms arch
AI & robotics
OS
networking
languages
symbolic computing
software
mobileComp Sci. & Engr.
databases & data mining
algorithms arch
AI & robotics
OS
networking
languages
symbolic computing
software
mobileComp Sci. & Engr.
databases & data mining
algorithms arch
AI & robotics
OS
networking
languages
symbolic computing
software
mhealth
mobileComp Sci. & Engr.
databases & data mining
algorithms arch
AI & robotics
OS
networking
languages
symbolic computing
software
mhealth
what is mhealth?
the promise of mhealth:
the promise of mhealth:
revolutionize medicine
the promise of mhealth:
revolutionize medicineeliminate doctor visits
the promise of mhealth:
revolutionize medicineeliminate doctor visitsremote / automatic diagnosis
the promise of mhealth:
revolutionize medicineeliminate doctor visitsremote / automatic diagnosisequalize developing countries
the promise of mhealth:
revolutionize medicineeliminate doctor visitsremote / automatic diagnosisequalize developing countries
the promise of mhealth:
revolutionize medicineeliminate doctor visitsremote / automatic diagnosisequalize developing countries
stress check
glucose buddyfitness trainer
heart rate
zombie run current mhealth
stress check
glucose buddyfitness trainer
heart rate
zombie run current mhealth43,000 apps for health on the app store
stress check
glucose buddyfitness trainer
heart rate
zombie run current mhealth43,000 apps for health on the app store 96% are for calorie counting & exercise
stress check
glucose buddyfitness trainer
heart rate
zombie run current mhealth43,000 apps for health on the app store 96% are for calorie counting & exercise4% are remote monitoring
consider physician’s needs
consider physician’s needsconnecting with patient
consider physician’s needsconnecting with patient tracking baselines
consider physician’s needsconnecting with patient tracking baselinespersonalized trending data
consider physician’s needsconnecting with patient tracking baselinespersonalized trending datamanaging chronic disease
consider physician’s needsconnecting with patient tracking baselinespersonalized trending datamanaging chronic disease
30% of all US healthcare spending is on chronic disease
mhealth and chronic disease management
mhealth and chronic disease management
compliance?cost?
doctor patient?data reliability?
compliance
compliance
baseline quantitysensor
phone as a sensorbaseline quantitysensor
phone as a sensorbaseline quantitysensor
embedded sensors
phone as a sensorbaseline quantitysensor
embedded sensors processing
estimated
accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
compliance++;cost--;dr_pat *= 10;
accelerometer gyroscope magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
compliance++;cost--;dr_pat *= 10;
data reliability?
what can the mobile phone sense with clinical accuracy?
future research
lung function jaundice
future research
lung function jaundice
spirometer lung function??
lung function
asthma COPD cystic fibrosis
evaluates pulmonary impairments
spirometer
device that measures amount of air inhaled and
exhaled.
using a spirometer
flow
volume
volum
e
time
using a spirometer
flow
volume
volum
e
time
using a spirometer
flow
volume
volum
e
time
volume-time graphvo
lume
time
volume-time graphvo
lume
time
volume-time graphvo
lume
time
FEV1
FVC
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
volume-time graphvo
lume
time1 sec.
FEV1
FVC
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
volume-time graphvo
lume
time1 sec.
FEV1
FVCFEV1% = FEV1/FVC
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
FEV1% = FEV1/FVC
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
FEV1% = FEV1/FVC
> 80% healthy60 - 79% mild40 - 59% moderate
< 40% severe
flow-volume graphflo
w
volume
flow-volume graphflo
w
volume
flow
volumeFEV1 FVC
1 sec.
PEF
PEF: Peak Expiratory Flow FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
flow-volume graph
flow
volume
normal
flow-volume graph
flow
volume
normalobstructive
flow-volume graph
obstructive diseases
!
resistance in air path leads to reduced air flow
obstructive diseases
!
resistance in air path leads to reduced air flow
restrictive diseases
!
lungs are unable to pump enough air and pressure
restrictive diseases
!
lungs are unable to pump enough air and pressure
flow-volume graphFlo
w
Volume
normalobstructive
flow-volume graphFlo
w
Volume
normal
restrictiveobstructive
clinical spirometry
home spirometry
home spirometry
!
faster detection rapid recovery
trending
home spirometry
high cost barrier patient compliance
less coaching limited integration
challenges with
SpiroSmart
availability cost portability more effective coaching interface integrated uploading
Using SpiroSmart
Using SpiroSmart
Using SpiroSmart
]
Using SpiroSmart
]
Using SpiroSmart
]
Using SpiroSmart
Using SpiroSmart
flow rate volume
lung functionairflow sensor
flow rate volume
lung functionairflow sensor
sound pressure microphone
flow rate volume
lung functionairflow sensor
sound pressure microphone processing
estimated
audio
audio
flow features
audio
flow features
measures regression
FEV1FVCPEF
0 1 2 3 40
5
10
15
Flow
(L/s
)
Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
0 1 2 3 40
5
10
15
Flow
(L/s
)Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
audio
flow features
measures regression
curve regression
FEV1FVCPEF
0 1 2 3 40
5
10
15
Flow
(L/s
)
Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
0 1 2 3 40
5
10
15
Flow
(L/s
)Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
audio
flow features
measures regression
curve regression
lung functionFEV1FVCPEF
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features estimation
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features estimation
vocal tractsource output
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features estimation
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking0 1 2 3 4 5 6 7
−1
−0.5
0
0.5
1lpc8raw
time(s)
amplitude
flow estimation features
vocal tractsource output 0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1lpc8raw
time(s)
amplitude
auto-regressive estimate
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
ground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
3.2
FEV1 features
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
3.2
0.120.17
FEV1 features
PEF featuresground truth
feature 1
feature 2
FEV1 features PEF features
measures regression
FEV1 features
bagged decision tree
PEF features
bagged decision tree
measures regression
FEV1 features
bagged decision tree
output
PEF features
bagged decision tree
output
measures regression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature N
window
ed machine
learning regression
......
curve regression
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature N
curve output
window
ed machine
learning regression
......
curve regression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
curve regression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
CRF
curve regression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
CRF
!
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
curve regression
study design
x 3
x 3
study enrollment
study enrollment
participants 5218-75 years old, mostly healthy
study a
study enrollment
participants 5218-75 years old, mostly healthy
study a
participants 1012-17 years old, mixed healthy/abnormal
study b
study enrollment
participants 5218-75 years old, mostly healthy
study a
participants 1012-17 years old, mixed healthy/abnormal
study b
participants 5610-69 years old, mostly abnormal
study c
enrolledby
pulmonologists
resultsmeasures regression
resultsmeasures regression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
resultscurves regression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
resultscurves regression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
resultscurves regression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
resultscurves regression
can SpiroSmart curves be used for diagnosis?
survey
• normal/abnormal subjects curves
survey
• 5 pulmonologists
• normal/abnormal subjects curves
survey
• 5 pulmonologists
• normal/abnormal subjects curves
• unaware if from SpiroSmart / spirometer
survey
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
one off 10%
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
one off 10%
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate false positive 14%
one off 10%
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
false negative 4%
false positive 14%
one off 10%
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
error 8%false negative
4%
false positive 14%
one off 10%
identical 64%
results
normal minimal obstructive mild obstructive moderate obstructive severe obstructive
restrictive
inadequate
error 8%false negative
4%
false positive 14%
one off 10%
identical 64%
!
!
abnormal vs normal 96%
appropriate for trending and screening
global health non-profit
global health non-profit
patient and doctors
global health non-profit
patient and doctors pharmaceutical drug trials
future research
lung function jaundice
future research
lung function jaundice
neonatal jaundice in the US
kernicterus: 21
hazardous jaundice: 1158
extreme jaundice: 2,317
severe jaundice: 35,000
phototherapy: 290,000
visible jaundice: 3.5 million
births/year: 4.1 million
Method Accuracy Disadvantages
TSB Gold standard (r=1.0)
Painful, costly, inconvenient, delayed
TcBAccurate
(r= 0.75 -0.93) Meter = $7000
tips $5 unavailable in most
physician offices
Visual assessment (provider or parent)
not accurate (r= 0.36 - 0.7), underestimates
severity
No standardization lighting, pigmentation
bilirubin level in blood
jaundice levelblood draw
bilirubin level in blood
jaundice levelblood draw
yellowness camera
bilirubin level in blood
jaundice levelblood draw
yellowness camera processing
estimated
bilicam
participants 48 newborns0-4 days old, collected in nursery
3 hospitals in Washington & Philadelphia
study A
participants 48 newborns0-4 days old, collected in nursery
3 hospitals in Washington & Philadelphia
study A
Color Linearization
• Camera Settings Adjustment
• Light Source Estimation
Image Segmentation
• Quality Control for Distance, Lighting, and Shadow
• Sternum, Forehead, Card Segmented
Color Calibration
• Dynamic Least Squares Regression
• Automatic Feature Selection
Neonatal Skin Response to Bilirubin
• Skin Independent Color Transformations Applied
• Multivariate Machine Learning Regression
bilicam
biliru
bin
level
bilicam estimation0
15
155 10
5
10
mg/dlr=0.91
bilicam initial results
20
20
biliru
bin
level
bilicam estimation0
15
155 10
5
10
mg/dlr=0.91
bilicam initial results
20
20
non whitewhite
biliru
bin
level
bilicam estimation0
15
155 10
5
10
mg/dlr=0.91
bilicam initial results
20
20
non whitewhite
TcB = 0.85
biliru
bin
level
bilicam estimation0
15
155 10
5
10
mg/dlr=0.91
bilicam initial results
20
20
non whitewhite
TcB = 0.85BiliCam = 0.84
bilicam future work
• near term: screeningbilicam future work
• near term: screening• medium term: more data
bilicam future work
• near term: screening• medium term: more data• long term: developing world
bilicam future work
• near term: screening• medium term: more data• long term: developing world
“in many resource poor nations,hyperbilirubinemia is the second or
third leading cause of infantmortality and disability”
bilicam future work
future research
lung function jaundice
future research
lung function jaundice
oxygen volume, VO2
future research
cardiac output and blood pressure
intra ocular pressure
intra ocular pressure
PressCam
eclarson.com eclarson@lyle.smu.edu @ec_larson> slide to unlock
Thank You!
eric c. larson | eclarson.com
eclarson.com eclarson@lyle.smu.edu @ec_larson> slide to unlock
phone-as-a-sensor technology:
Assistant Professor Computer Science and Engineering
mhealth and chronic disease
collaborators: !Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef
eric c. larson | eclarson.com
eclarson.com eclarson@lyle.smu.edu @ec_larson
phone-as-a-sensor technology:
Assistant Professor Computer Science and Engineering
mhealth and chronic disease
collaborators: !Joseph Camp Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Gaetano Boriello Mayank Goel Lilian DeGreef
top related