Passive sensing of circadian rhythms for individualized models

of cognitive performance

Julie Kientz, Tanzeem Choudhury

Saeed Abdullah, Elizabeth Murnane, Mark Matthews, Matt Kay

Cognitive capabilities vary over time

2

Alertness: basic building block of cognitive performance

3

fatigue and sleepiness =

alcohol intoxication

4

Fatigue is involved in 30% of all road accidents in US

NTSB. Safety report NTSB/SR-99/01

5

36% increase in serious medical errors

6

Williamson, Ann, et al. "The link between fatigue and safety." Accident Analysis & Prevention 43.2 (2011): 498-515.

Negative impact on learning and problem solving

7

8

Alertness

Chronotype

Sleep CircadianMisalignment

Stimulants

9

Circadian Rhythm: biological processes following a roughly 24-hour period

circa: about, diem: a day

10

Almost every neurobehavioral process displays circadian rhythms

11

Chronotype: Individual differences in temporal preference resulting from circadian rhythms (early and late types)

12

Circadian Rhythms and Alertness

• Internal time dictates optimal peak alertness period

• Alertness drops during mid-day dip

• Sleep is a crucial factor

13

Studying alertness beyond controlled lab environment

14

Continuous assessment of alertness based on in-situ data in a real-world setup

15

• How do body clock, time of day, and stimulant intake impact alertness?

• Do phone usage patterns reflect fatigue and sleepiness?

• Can we automatically assess alertness using passively sensed phone data?

16

Methodology

17

Population

• University-aged individuals

• Massive risk of circadian misalignment

• Largest and most habituated technology users

18

Participants & Procedure• 20 participants

• 7 male, 13 female

• 18-29 years old

• Android users

• 40 days

19

• Data

• Daily sleep diary

• 4-times-per-day alertness assessment (EMA)

• Phone use logs

• Interviews

Sleep Data

MSFSC = MSF −0.5(SDF −(5∗SDW +2∗SDF)/7)

1 2 3 4 5 6 7 8 9

35

0

5

10

15

20

25

30

Chronotype

% o

f Sam

ple

Larks Owls

extremeEarlytype

moderateEarlytype

slightEarlytype

Normaltype

slightLatetype

moderateLatetype

extremeLatetype

Chronotype:

20

21

Smartphone Toolkit

22

PVT

From PVT to Alertness

• Median response time from a PVT session

• Establish individual baseline across all session

• Alertness is departure from baseline

23

Results

24

Alertness varies across time

25

Early and late types have different performance pattern

26

Daylight saving time (DST)

• Social clock-shifting

• Known to cause circadian disruptions

• 70 countries observe DST, impacting 1.6 billion people

27

Negative impact of DST

28

Stimulant intake

• Positive stimulants

- Caffeine intake, napping, doing exercise, nicotine intake

• Negative stimulants

- Alcohol consumption, having meals, relaxation

29

Stimulant intake

• 5.1% increase after positive stimulants

• 1.37% drop after negative stimulants

• Statistically significant (t = 2.2, p = 0.03)

30

Self-Assessment of Alertness

• Self-assessment

- Tiredness, energy and concentration level

• Response time differs significantly between high and low self-ratings

- fatigued individuals are usually aware of reduced capability

32

230 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

2500

0

500

1000

1500

2000

App

licat

ion

Usa

ge E

vent

s

Entertainment Time & WeatherCommunicationProductivity BrowsingEmailSocial Media

Rhythms in App Use

33

Productivity vs. Entertainment

20

0

5

10

15

% o

f Usa

ge E

vent

s

EntertainmentProductivity

Mon Tues Wed Thu Fri Sat Sun

• Work days

• Free days

• Mid-week dip

34

Usage Differences by Chronotype

• Early Types

• 25% more productivity apps

• 18-28% fewer entertainment apps

• Late Types

• 22-68% more productivity apps

• 15-50% less entertainment apps

100

-100

-80-60-40-20

020

406080

Earl

y-La

te U

sage

Cha

nge

(%)

Morning(6AM-12PM)

Afternoon(12PM-6PM)

Evening(6PM-12AM)

Night(12AM-6AM)

EntertainmentProductivity

35

Internal Time

230 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

InT

Performance

6

-8

-7

-6

-5

-4

-3

-2

-10

1

2

3

4

5

Usage

Entertainment CMC

Alertness

Productivity

InT = ExT - MSFSC

“To wake myself up, I’ll have to look at things on the phone like Facebook or Tumblr.”

“In morning classes, I have less attention and am very tired so I’ll browse the phone. Using tactics like social media, I focus on the screen to try to keep my eyes open.”

———

“Every time before I go to bed, I play a card game until I feel sleepy.”

“I use my phone when falling asleep. Especially if I’m having trouble falling asleep, I’ll play a game or talk to my boyfriend until I fall asleep.”

36

App Use and Sleep

• Less sleep: less productivity (r=0.43), more entertainment apps (r=-0.19)

• Adequate sleep: 61% more productivity apps

• Inadequate sleep: 33% more entertainment apps

• Nightly use events reflect sleep interruptions

37

Predicting alertness from phone data

• PVT is not suitable for longitudinal deployment

• Passive inferring of alertness can enable a new suite of HCI applications

• Can data from mobile phone predict alertness?

38

Predicting Alertness• Stochastic Gradient Descent (SGD) with Huber loss

function

• Standardize all features to have zero mean and unit variance

• L1-norm as regularization term

- α = 10-8

- learning rate: γt = γ0 · t−1/4 (with γ0 = 0.01)

39

Predicting Alertness

• 10 fold cross-validation

• RMSE of 11.39 across all participants

• Accurate enough for scalable deployment

Internal Time

Avg. time between phone usage sessions

Short Session frequency

Phone usage duration

Relative sleep need

Top-ranking features for predicting alertness

40

Implications & Applications

41

Task Scheduling

• When to do what?

- based on cognitive demand and assessed alertness

• Better team collaboration

- grouping members with similar circadian characteristics

42

Learning and Education

• Circadian disruptions adversely affect memory and learning abilities

• Learning and memorization aligned with individual alertness rhythms in school

43

Accident Prevention

• Assistive systems for drivers

• Continuous monitoring to prevent industrial accidents

44

Future work: • Circadian-aware technology • Informatics tools & intervention studies

Contributions: • In-situ alertness sensing • Manifestations of biological rhythms in mobile use • Automated alertness prediction

45