PROBABILISTIC FRAMEWORK FOR EEG-BASED DROWSINESS AND VIGILANCE MONITORING

Roman Rosipal1, Björn Peters2, Göran Kecklund3, Torbjörn Åkerstedt3

Georg Gruber4, Michael Woertz1, Peter Anderer4,5, Georg Dorffner1,4,6

1Austria Research Institute for Artificial Intelligence, Vienna, Austria 2Swedish National Road and Transport Research Institute, Linköping, Sweden

3Karolinska Institutet, Stockholm, Sweden4The Siesta Group Schlafanalyse GmbH, Vienna, Austria

5Department of Psychiatry, Medical University of Vienna, Vienna, Austria6Institute of Medical Cybernetics and Artificial Intelligence, Center for Brain

Research, Medical University of Vienna, Vienna, Austria

vti

Objectives Objectives

To develop probabilistic modelling framework for real-time monitoring of drowsiness, impaired vigilance or fatigue.

The framework should overcome the main drawback of the existing monitoring systems, which is their limited capability to deal with a wide range of information sources needed to cover many aspects influencing human behaviour (drowsiness, fatigue or vigilance).

Presented study adapts the methodology for the sleep process modelling developed in the SENSATION project.

The developed methods are applied to day-time electrophysiological recordings (EEG, EOG, EMG) focused on screening subject’s drowsiness (sleepiness) and vigilance.

Driving simulator- the third generation moving base driving simulator

- 45 shift workers

- drove during morning hours directly after a full night-shift with no sleep

Quatember-Maly clocktest- a computerized version of the Macworth et al (1957) vigilance test

- 15 subjects from a sleep related experiment (two nights in the sleep lab)

- the test was performed during two consecutive day-time periods in i) three 50-min ii) two 25-min long in time separated sessions

Two experiments Two experiments –– 1) driving simulator 2) vigilance test1) driving simulator 2) vigilance test

Karolinska Drowsiness Score (KDS; Gillberg, Kecklund & Åkerstedt,1996)- visual drowsiness scoring based on a single EEG channel (Oz–Pz), EOG & EMG (artifacts detection)

- the method was developed to score drowsiness of awake subjects andis based on the Rechtschaffen & Kales sleep scoring rules:

slow eye movements, changes in alpha & theta activity - 20-sec segments divided into 2-sec bins, each bin visually scored; KDS is the % of scored bins inside a 20-sec window; range 0-100

KarolinskaKarolinska Drowsiness Score (KDS)Drowsiness Score (KDS)

0 10 20 30 40 50 60 70 80 90 1000

1000

2000

3000

4000

5000

6000

7000

KDS

# of

seg

men

ts

Histogram of the KDS values for the 4-second segments used in the analysis.

Gaussian Mixture Model (GMM)Gaussian Mixture Model (GMM)

( ) ( ) ( ) ( )11/ 2/ 2 / 2| 2T

k k kd x xk kp x e μ μθ π

−−− − − Σ −= Σ

where ( )|

| (i k kk Ci

p x C p x | )α θ∈|

= ∑| |

1, 0k kk Ci

α α∈

= ≥∑

Parameters to estimate:- group priors; p(C1), p(C2), ... - number of components- mixing proportions; αi- means, covariances; μi , Σi

p(CK)p(C1) …p(C2)

αi, μi, ΣiC1

αi, μi, ΣiC2

αi, μi, ΣiCK

…

Structure of a Hierarchical GMMStructure of a Hierarchical GMM

( )1,2 1,2

( , ) ( | ) ( )i i ii i

p x p x C p x C p C= =

= =∑ ∑

( | ) ( )( | ) ( | ) ( )( )i i

i i ip x C p Cp C x p x C p C

p x= ∝

Bayes’ theorem to estimate class-posteriors:

where p(x) is the unconditional density:

First step: Two classes (cornerstones): KDS = 0 & KDS >= 50 i) Hierarchical GMM trained to discriminate classes ii) 4-second segments – total 10654 (5433/5221)iii) AR representation on each EEG (Fz-A1, Cz-A2, Oz-Pz) channeliv) 10 x 10-fold cross-validationConfusion matrix:

Second step: 30% of the cornerstones samples used for training the model. All 4-sec segment data were applied to the trained model. i) agreement with the KDS scores was visually checked ii) the nonparametric Spearman’s rank correlation coefficient (SC)

was computed to compare the smoothed KDS and predicted drowsiness curves.

DriverDriver’’s drowsiness models drowsiness model

0.78 0.22

0.23 0.77

Results: driverResults: driver’’s drowsiness models drowsiness model

0 400 800 12000

20

40

60

80

Subject : 5

Smoo

thed

KDS

/ PD

0 400 800 12000

20

40

60

80

KDS

0 400 800 12000

20

40

60

80

Subject : 11

Sm

ooth

ed

KDS

/ PD

0 400 800 12000

20

40

60

80

KDS

0 200 400 6000

20

40

60

80

Subject : 19

Smoo

thed

KDS

/ PD

0 200 400 6000

20

40

60

80

Time 4sec/point

KDS

0 400 800 12000

20

40

60

80

Subject : 34

Sm

ooth

ed

KDS

/ PD

0 400 800 12000

20

40

60

80

Time 4sec/point

KDS

In all but three subjects the Spearman’s rank correlation > 0.2; median 0.53, max. 0.85

0 200 400 600 800 1000 12000

20

40

60

80

Subject : 5

Time 4sec/point

Sm

ooth

ed

KDS

/ PD

5 11 19 340

0.20.40.60.8

SC

Subjects number

Top: The KDS (dash-dotted) and twenty predicted drowsiness (PD) (solid lines) curves. The thicker line represents the mean value of the PD curves.

Bottom: The boxplot of the Spearman rank coefficients (SC) for subjects 5,11,19 and 34.

Results: drowsiness model robustness analysis Results: drowsiness model robustness analysis

15 subjects used; each subject: 2 morning & 1 afternoon 50-min sessionsPre-trained driving simulator drowsiness model was used: EEG (Fz-A1, Cz-A2, Oz-Pz)The model was applied to all data and periods of low & high posteriors were selected (<0.2 / >0.8) The model was then re-trained using the selected segments from the first and last third time periodsReaction time (response) values were ‘smoothed’ using the cubic spline method

Analysis was done on 2.5-min segments where the mean of the predicted impaired vigilance and response times was computed (≈ 20 values).

Vigilance modelVigilance model

0 5 0 0 1 0 0 0 1 5 0 0 2 0 0 0 2 5 0 0 3 0 0 00 .2

0 .3

0 .4

0 .5

0 .6

0 .7

0 .8

0 .9

1

S u b je c t 1 5

t i m e 4 s e c / p o i n t

Pred

icte

d vi

gila

nce

/ sm

ooth

ed re

actio

n tim

es

s m o o t h e d R Tp r e d i c t e d v i g i l a n c em i s s e d s t i m u l u sn o s t i m i m u l u s r e s p o n s e

0 10 200

0.5

1Sub.1 R:0.00

0 10 200

0.1

Sub.2 R:0.13

0 10 200

0.5

1Sub.3 R:0.31

0 10 200

0.2

0.4Sub.4 R:0.3

0 10 200

0.4

Sub.5 R:-0.01

0 10 200

0.5

1Sub.6 R:0.57

0 10 200.4

0.7

1Sub.7 R:-0.32

0 10 200

0.5

1Sub.8 R:-0.15

0 10 200

0.1

Sub.9 R:0.45

0 10 200

0.05

0.1Sub.10 R:0.6

0 10 200

0.4

Sub.11 R:0.59

0 10 200

0.4

Sub.12 R:0.66

0 10 200

0.2

0.4

Sub.13 R:-0.18

0 10 200.4

0.7

1Sub.14 R:-0.02

0 10 200.4

0.7

1Sub.15 R:-0.08

0 10 200.2

0.3

0.4All subjects R:0.82

Results: vigilance modelResults: vigilance model

Mean predicted vigilance (red, dotted) and extrapolated reaction time values. Afternoon session

ConclusionsConclusions

A reasonably high level of correlation was observed between predicted drowsiness levels and the KDS values. This was despite the fact that the hGMM was applied to shorter data segments, no EOG information was used and, in contrast to the Karolinska scoring protocol, broadband spectral information from multi-electrode EEG setting was considered.

On vigilance task data the results were not so conclusive. High correlations were found on data recorded during the afternoon session only. However, it needs to be stressed that the presented analysis relates the continuously predicted vigilance and extrapolated reaction time values. This is new and in contrast to the approaches where global statistics computed from whole vigilance test experiments are analysed.

The computations associated with the presented approach are fast enough to build up a practical real-time drowsiness or vigilance monitoring system.