towards individual and aggregate human behavior modeling...

122
Towards Individual and Aggregate Human Behavior Modeling from Data in the Real-World Nuria Oliver, PhD Scientific Director User, Data and Media Intelligence Telefonica Research

Upload: others

Post on 23-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Towards Individual and Aggregate Human Behavior Modeling from Data in the Real-World

Nuria Oliver, PhD

Scientific Director

User, Data and Media Intelligence

Telefonica Research

Page 2: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Page 3: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

• ~20 full time researchers

• Successful internship program with ~25

interns/year

• Successful stage program for undergraduate students

• Visiting professors and scholars

• Open innovation: Barca, Ferran Adria,

UNGP, MIT, Brno Univ, UCIrvine…• Hiring!!

Page 4: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

3. Big Data Analytics 4. Human Computer

Interaction and

Mobile Computing

1. Machine Learning

Personalization and

User Modeling (RecSys)

2. Multimedia Data

Analysis (Voice)

User, Data and Media Intelligence

LinasBaltrunas

AlexandrosKaratzoglou

Jose SanPedro

Aleksandar Matic

Jordi LuqueXavier Anguera

Enrique Frias

Martin PielotSouneil Park

Joan Serra

Carlos Segura

Page 5: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Machine Learning Approachesto Model Individual and

Aggregate Human Trait and Behavior from a variety of

Data sources: voice, mobile sensors, serviceusage data, mobile network

data...

We work on…

Page 6: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Human Behavior @Telefonica

Page 7: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Summary

• Built capabilities over the years on:• Data Science, Machine Learning and Data Analytics:

• Mobile data, voice, text

• Personalization and Recommendations

• Human-Computer Interaction and Mobile Computing

• User Modeling

• Generated over 40 patents

• Achieved international recognition: • 6 best paper awards and 5 best paper award

nominations

• 3 prestigious Marie Curie Fellowships and 2 EU projects

• Rising Talent Award, IEEE and ACM Senior Member Award, 10-year technical award (ICMI)

• External influence: GSMA white paper for African operators re Ebola; ITU, UN, MWC presentations

Page 8: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Our Work in the Media

Page 9: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Our Work in the Media

Page 10: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Can mobilephones be used

to model, understand and help their usersand the world

at large?

Page 11: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Page 12: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 13: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Page 14: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Page 15: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Detecting Boredom from

Mobile Phone Usage

Research

UbiComp ‘15, Osaka, Japan

Martin

Pielot

Telefonica

Research

Tilman

Dingler

University of

Stuttgart

Jose

San Pedro

Telefonica

Research

Nuria

Oliver

Telefonica

Research

Page 16: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0

War on Attention*

* http://www.forbes.com/sites/onmarketing/2012/10/19/the-attention-war/

Page 17: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0

The trade we make:

Our attention so Internet companies

can pay their bills

Page 18: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Our engagement is now defined by push-

driven notifications... We’re “hunting and pecking” through our app grid a lot less; the

apps that notify us are rewarded with our

engagement (and our dollars).

Page 19: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

The Deluge of Push-Driven Notifications

Page 20: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

‚Attention is a limited resource—a person has only so much of it ‘ [Matthew B. Crawford]

Attention Economy: treating human attention as a scarce commodity[Davenport and Beck, 2001]

times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0

Page 21: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Wild-West Land-Grab Phase

“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain

Page 22: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Wild-West Land-Grab Phase

“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain

If the trade

attention for free servicesis to be sustained

we need to better

protect mobile phone users

Page 23: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Could Boredom

be part of the solution?

Page 24: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Attention is not always scarce

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Page 25: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Attention is not always scarce

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Boredom displeasure caused by “lack

of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who

does not have anything to do; it’s

someone who is actively looking for

stimulation”

[Eastwood, 2002]

Page 26: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Attention is not always scarce

Boredom displeasure caused by “lack of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who does not have

anything to do; it’s someone who is actively looking for

stimulation”

[Eastwood, 2002]

Mobile phones are a commonly

used tool to kill time when bored

[Brown et al. 2014]

Page 27: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 28: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 29: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 30: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 31: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Attention is not always scarce

Mobile phones are a commonly used tool to

fill or kill time when bored [Brown et al. 2014]

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Boredom displeasure caused by “lack of stimulation”

[Fenichel, 1951]

“a bored person is not just someone who does not have anything

to do; it’s someone who is actively looking for stimulation”

[Eastwood, 2002]

If phones knew when their

users are killing time

maybe they could suggest a

better use of that time

Page 32: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Is it possible to

detect

boredom from

mobile phone

usagepatterns?

Page 33: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Borapp – Sensor-Data Collection

Always

collected

Only

collected

if phone in

use

Sensor Description

Battery Status Battery level ranging from 0-100%

Notifications Time and type (app) of notification

Screen Events Screen turned on, off, and unlocked

Phone Events Time of incoming and outgoing calls

Proximity Screen covered or not

Ringer Mode Silent, Vibration, Normal

SMS Time of receiving, reading, and sending SMS

Sensor Description

Airplane Mode Whether phone in airplane mode

Ambient Noise Noise in dB as sensed by the microphone

Audio Jack Phone connected to headphones or speakers

Cell Tower The cell tower the phone is connected to

Data Activity Number of bytes up/downloaded

Foreground app Package name of the app in foreground

Light Ambient light level in SI lux units

Screen Orient Portrait or Landscape mode

Wifi Infos The WiFi network the phone is connected to

Page 34: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Experience Sampling

“Right now, I feel bored”

[5-point Likert scale]

Min. 6 times per day

Preferably triggered when

phone in use

Borapp – Experience Sampling

Page 35: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

User Study: Data Collection

54 Participants

aged 21 – 46 (M = 30.6) years

11 female, 23male, 19 not disclosed

For two weeks in July 2014

Over 40M sensor log entries

4398 valid self-reports of boredom

Page 36: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

0

500

1000

1500

0 1 2 3 4

Fre

qu

ency

Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree

Boredom Ground Truth

Page 37: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

0

500

1000

1500

0 1 2 3 4

Fre

qu

ency

Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree

Absolute ground

truth

Bored: ratings 3, 4

446 (10.1%)

instances

Boredom Ground Truth

Page 38: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Absolute ground truth

Bored: ratings 3, 4

446 (10.1%) instances

Normalized ground

truth

Z-score per person

Bored: z > 0.25

1518 (34.5%) instances

0

400

800

1200

1600

2000

-2 -1 0 1 2

Fre

qu

en

cy

Normalized Subjective Boredom, (higher number = more bored than

usual)

Boredom Ground Truth

Page 39: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Category Example Feature Explanation

Context Semantic Location Home, work, other, unknown

Demographics Age, gender 38, female

Last Communication

Activity

Time last incoming

call

Time passed since somebody called the participants

Usage (intensity) Bytes received Number of bytes downloaded in the last 5 minutes

Usage (externally

triggered)

Number of

notifications

Number of notifications received in the last 5 minutes

Usage (idling) Number of apps Number of apps launched in the last 5 minutes

Usage (type) Most used app App used for the most time in the last 5 minutes.

35 Features, 7 Categories

Page 40: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

RQ1: how well can phones detect

killing-time/boredom events from

these usage patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough to

be useful?

Page 41: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

• Supervised machine learning classification: L2-

regularized logistic regression, linear SVMs and

Random Forests

• 5-fold cross validation

• Performance evaluation:

• Average Precision, Recall

• AUROC: area under the ROC curve information

about the ability of the model to rank users by their

probability to be bored

• Absolute and normalized boredom as ground

truth

Modeling Approach

Page 42: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

Page 43: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

Primary data set

Page 44: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

74.6%

82.9%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

normalized

absolute

Model Performance |

Random Forest (AUCROC)

Primary data set

62.4% precision for

50% recall of boredom events

Page 45: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Boredom can be detected

from phone-usage patterns

with an accuracy of ca.

75% to 83% AUCROC

Take Away #1

Page 46: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

RQ1: how well can phones detect

killing-time boredom events from

these usage patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough to

be useful?

Page 47: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recency of

communication activity

i.e., time since last

incoming or

outgoing

communication;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Page 48: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day,

ambient light

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Page 49: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics,

i.e., gender and age;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Page 50: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics, i.e., gender and age;

General usage

intensity i.e, phone out of

pocket, or time since

last phone use …;

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Page 51: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recency of communication activity i.e., time since last incoming or

outgoing communication;

Phase of the dayi.e., hour of the day, ambient light

Demographics, i.e., gender and age;

General usage intensity i.e, phone out of pocket, or time

since last phone use …;

Intensity of recent

usage i.e. # of unlocks, or

# of apps launched

in last 5 minutes, …

Feature Import Correlation The more bored, the ..

time_last_outgoing_call 0.0607 -0.143 less time passed

time_last_incoming_call 0.0580 0.088 more time passed

time_last_notif 0.0564 0.091 more time passed

time_last_SMS_received 0.0483 0.053 more time passed

time_last_SMS_sent 0.0405 -0.090 less time passed

time_last_SMS_read 0.0388 -0.013 more time passed

light 0.0537 -0.010 darker

hour_of_day 0.0411 0.038 later

proximity 0.0153 -0.186 less covered

gender (0=f, 1=m) 0.0128 0.099 more male (1)

age 0.0093 n.a. +20s/40s, -30s

num_notifs 0.0123 0.061 more notifs

time_last_notif_cntr_acc 0.0486 -0.015 less time passed

time_last_unlock 0.0400 -0.007 less time passed

apps_per_min 0.0199 0.024 more apps per minute

num_apps 0.0124 0.049 more apps

bytes_received 0.0546 -0.012 less bytes

bytes_transmitted 0.0500 0.039 more bytes

battery_level 0.0268 0.012 the higher

battery_drain 0.0249 -0.014 the lower

Page 52: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Apps

Co-occur with being

bored

Co-occur with

NOT bored

… and uncategorized apps

Page 53: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Boredom is related toRecency of communication

Phase of the day

Demographics

Intensity and type of phone

usage

Type of used apps

Take Away #2

Page 54: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

RQ1: how well can phones

detect killing-time boredom

events from these usage

patterns?

RQ2: which usage patterns are

related to killing time with the

phone?

RQ3 is the model good enough

to be useful?

Page 55: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Borapp2

Model running on Mobile

Phone

Using primary data set

Constantly infers the

user’s boredom state on

the fly

Page 56: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Suggests Reading Buzzfeed Articles

Page 57: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

User Study 2: Data Collection

16 Participants (different from 1st

study)

aged 18 – 51(M = 39) years

13 male, 2 female, rest did not disclose

For two weeks in Feb 2015

941 Buzzfeed recommendations

48% when predicted bored

Page 58: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Measure 1: Click-ratioFraction of times people

clicked on notification

(Mdn)

8% when not bored

20.5% when bored

(as inferred by the model)

Difference significant

z = -2.102, p = .018

Large effect

r = -.543

Page 59: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Measure 2: Engagement-ratio

Fraction of times people

spent more than 30 sec

reading (Mdn)

4% when not bored

15% when bored

(as inferred by the model)

Difference significant

z = -2.102, p = .018

Large effect

r = -.511

Page 60: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

When inferred to be bored,

participants were …

More likely to clickMore likely to read

for > 30 seconds

Page 61: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

The generic model was

powerful enough to

create significant, large

effects on click- and

engagement-ratios

Take Away #3

Page 62: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Impact in the Press…

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Page 63: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Application Scenarios

Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

2.0

Page 64: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recommend

content to

alleviate

boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful but not

necessarily boredom-curing activities

Encourage embracing

boredom

Page 65: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recommend content to

alleviate boredom

Shield user from

non-important

interruptions

during non-bored

times

Suggest useful but not necessarily boredom-curing

activities

Encourage embracing boredom

Page 66: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful

but not

necessarily

boredom-curing

activities

Encourage embracing

boredom

Page 67: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful

but not

necessarily

boredom-curing

activities

Encourage embracing

boredom

Page 68: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Being bored

is good for

you

Why don’t

you turn me

off?

Recommend content to

alleviate boredom

Shield user from non-important

interruptions during non-bored

times

Suggest useful but not

necessarily boredom-curing

activities

Encourage

embracing

boredom

Page 69: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Relevant Publications

http://doi.acm.org/10.1145/1864349.1864371

http://www.youtube.com/watch?v=_p7n_pn7xaE

TEDx Las Ramblas, Feb 2012

""When Attention is not Scarce: Detecting Boredom from Mobile Phone

Usage"

Pielot, M., Dingler, T., San Pedro, J. and Oliver, N.

Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)

Best paper award!

"Boredom-Computer Interaction: Boredom Proneness and SmartPhone Use"

Matic, A., Pielot, M. and Oliver, N.

Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)

Page 70: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Page 71: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

MobiScore: Credit Score Inference

With J. San Pedro, D. Proserpio & J. Gonzalez

Page 72: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 73: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Credit Scores, a number that represents an assesment or

the likelihood that a person will repay his or her debt, are

widely spread in the world enabling the growth of

consumer credit and transactional operations…

… They are calculated upon applicants’ creditand financial history…

Page 74: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

… however, because of the data that is used to

compute credit scores, not everybody is

scorable…

Thin-file and no-hit:Not only a situation

for Emergent

Economies

2.5B

UNBANKED UNDERBANKED

8M

IMMIGRANTS

1.1M/year

NEW CLIENTS

No Positive Information

STUDENTS&YOUNG WORKERS

Page 75: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Page 76: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Page 77: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Would it be possible to build

an alternative score using

mobile behavioral data?

Page 78: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Credit

Score

Model

Probability of Default @30 and @90 days

Consumption

Social

Mobility

Mobile Behavioral

Data (CDR) MobiScore

Customer Relation

Management Data (CRM)

Demographic

Product

Socio-economic

Page 79: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 80: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Typical Mobile Behavioral Data• CDR

• SMS

Consumption Social Network Mobility

Call duration In/Out Degree Radius of gyration

N. Events Delta w.r.t time window Travelled distance

Lapse between events Unique Calls per day Rate of popular antennas

Reciprocated events Unique SMS per dayRegularity of popular

antennas

… … …

HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD CD_ERB CD_CCC QT_DUR

20:05:31 XXX YYY 3 11 20140519 2 1562 568 33

… … … … … … … … … …

HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD QT_TRFG

15:53:54 XXX ZZZ 3 25 20140506 2 1

… … … … … … … …

Page 81: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Customer Relation Management (CRM) Data

• Demographic information: age, gender

• Socio-economic• Derived from home address

• Payment variables: late payments

• Product features: device brand, device OS, device type, line type, line status, line quantity

• Time since activation

Page 82: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Credit default reports, i.e. pending

balance that is considered to be

uncollectible

More than 30 days in arrears

More than 90 days in arrears

Ground Truth

Page 83: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Credit

Score

Model

Probability of Default @30 and @90 days

Consumption

Social

Mobility

Mobile Behavioral

Data (CDR) MobiScore

Customer Relation

Management Data (CRM)

Demographic

Product

Socio-economic

Page 84: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Building Credit Scores from Data

• Mobile phone usage logs• 3-month period (Jan-March 2014)

• Fully anonymized

• Over 35 million call events and 11 million SMS events

• Customer Relation Management Data

• Credit default reports, i.e. pending balance that is considered to be uncollectible

• More than 30 days in arrears

• More than 90 days in arrears

Of over 60,000 individuals in a Latin American country

Mo

bile

Da

taFin

an

cia

l Info

Page 85: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Modeling Approach

• Supervised machine learning: L2-regularized logistic regression, linear SVMs and Gradient Boosted Trees

• 5-fold cross validation

• Performance evaluation:• Average Precision

• AUROC: area under the ROC curve information about the ability of the model to rank customers according to their probability of default

• Default @30 days and @90 days

Page 86: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

2 weeks 1 month 3 months 2 weeks 1 month 3 months

GBT 63.0 64.5 67.5 68.5 69.4 71.6

@30 LR 62.2 64.0 66.4 67.6 68.8 70.7

SVM 62.1 63.9 66.7 67.9 68.8 70.6

GBT 63.1 64.4 67.5 70.2 70.8 72.5

@90 LR 62.4 64.5 67.4 68.7 70.5 72.1

SVM 63.1 64.1 67.2 69.7 70.3 72.1

Classification Performance (AUROC)

• GBTs outperform other classifiers thanks to their higher degree of complexity and flexibility

CRM + Voice CDRs CRM + Voice+SMS CDRs

Page 87: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Comparison with State of the Art

Page 88: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Implications

• MobiScore leverages passively collected mobile data to accurately infer default risk

• CDR + CRM features achieve significantly better performance than state-of-the-art models

• MobiScore opens the door to alternative credit score models that would enable millions of people to get access to credit

Page 89: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Relevant Publications

"MobiScore: Towards Universal Credit Scoring from

Mobile Data"

Proserpio, D., San Pedro, J. and N. Oliver

Proc. of Int. Conf on User Modeling (UMAP 2015)

"Prediction of Socioeconomic Levels using Cell Phone

Records", Victor Soto and Vanessa Frias-Martinez and

Jesus Virseda and Enrique Frias-Martinez, International

Conference on User Modeling, Adaptation and

Personalization, UMAP'11, Industrial Track, Girona, Spain,

2011

Page 90: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling:• Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Page 91: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

6.8 billion subscribers

96% of world’s population (ITU)

Mobile penetration of 120% to 89% of population (ITU)

Emerging and developed regions

More time spent on our phones than watching TV or with our with

our partner (US and UK)

Page 92: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Cell Phones as Sensors of Human Activity

May 19, 2011, 7:06 pm The Sensors Are Coming!By NICK BILTON

Telecom / WirelessNEWSCellphones for ScienceScientists want to put sensors into everyone's hands

Digital footprints enable large-scale

analysis of human behavior

Page 93: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 94: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Sensors of aggregated human activities used to

monitor citizens’ interactions and mobility patterns (Song et

al. 2010; Dong et al. 2011)

understand individual spending behaviors (Singh et al.,

2013) and financial responsibility (San Pedro et al, 2015)

predict socio-economic indicators of territories (Eagle et al.,

2010; Soto et al., 2011; Smith-Clarke et al., 2014)

model spreading of malaria (Wesolowski et al., 2012) and

H1N1 (Frias-Martinez et al., 2011)

infer people’s traits (deOliveira et al. 2010, Staiano et al.,

2012; Chittaranjan et al. 2013; de Montjoye et al., 2013) and

states (Bogomolov et al., 2013)

Phones as Social Sensors

Page 95: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 96: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 97: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,
Page 98: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Big Data for Social Good

Crime Prediction

Analysis of impact of floods

http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver

Page 99: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Crime

Work with Bogomolov, A., Lepri, B., Staiano, J., Pianesi, F., Pentland, A.

Page 100: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Affects quality of life and economic

development both at the national and local level

Several studies explore relationships between

crime and socio-economic variables: education,

income, unemployment, ethnicity, …

Several studies have shown significant

concentrations of crime in small geographical

areas: crime hotspots

Crime

Page 101: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

T1: Natural surveillance as key deterrent for

crime: people moving around are eyes on the

street (Jacobs, 1961)

high diversity among the population and

high number of visitors -> less crime

T2: Defensible space theory (Newman, 1972)

high mix of people -> more crime

Crime and Urban Environment

Page 102: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

People-centric perspective vs Place-centric perspective

people-centric perspective used for

individual or collective criminal profiling

place-centric perspective used for

predicting crime hotspots

Crime Prediction

Page 103: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Data-driven and place-centric approach to

crime prediction

Multimodal approach: people dynamics

derived from mobile network data and

demographics

European metropolis: London

Prediction of crime hotspots and not criminals

profiling

Our Approach

Page 104: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Smartsteps Dataset:

for each of the Smartsteps cells a variety of demographic

and human dynamics variables were computed every

hour for 3 weeks (from December 9 to December 15, 2012

and from December 23, 2012 to January 5, 2013)

Criminal Cases Dataset: criminal cases for December 2012 and for January 2013

London Borough Profiles Dataset:

open dataset containing 68 metrics about the population

of a particular geographic area

Multimodal Approach: Data

Page 105: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

• Footfall count: Shows the trend in footfall in a

specified area hourly, daily, weekly and

monthly. Provides a basic profile of the crowd.

• Catchment area: Shows which postal sectors

are your customers coming from by hour, day,

week and month. Shows the “battleground”

for two sites.

• Transport mode: Shows flows of crowds from

any two points, segmented by road, air, train,

etc.

SmartSteps

Page 106: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

For each cell and for each hour the dataset contains:

an estimation of how many people are in the cell

the percentage of these people at home, at work or

just visiting the cell

the gender splits (male vs. female)

the age splits (0-20 years, 21-30 years, 31-40 years, …)

SmartSteps Data

Page 107: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Crime geolocation for 2 months (December 2012

– January 2013)

All reported crimes in UK specifying month and

year and not specific day/time

Median crime value (=5) used as threshold

Spatial granularity of borough profiles is at LSOA

levels: LSOA are small geographical areas

defined by UK Office for National Statistics (mean

population: 1500)

Crime Data

Page 108: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

68 metrics about the population of a specific

geographical area: demographics, households,

migrant population, employment, earnings, life

expectancy, happiness levels, house prices, etc.

Spatial granularity of borough profiles is at LSOA

levels:

LSOA are small geographical areas defined by UK

Office for National Statistics (mean population: 1500)

London Borough Profiles Data

Page 109: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

From Smartsteps data we extract

1st order features (mean, median, min.,

max., entropy, etc.)

2nd order features on sliding windows of

variable length (1 hour, 4 hours, 1 day,

etc.) to account for temporal patterns

Feature Extraction

Page 110: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Feature Selection

Mean decrease in Gini coefficient of

inequality

the feature with maximum mean decrease

in Gini coefficient is expected to have the

maximum influence in minimizing the out-of-

the-bag error

The feature selection process produced a

reduced subset of 68 features (from an initial

pool of about 6000 features)

Page 111: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Classification Approach

Binary classification task: high crime area vs low

crime area

10-fold cross-validation approach

Classifier: Random Forest (RF)

RF overcomes logistic regression, support vector

machines, neural networks, decision trees

Page 112: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Smartsteps-based classifier significantly outperforms baseline

majority and borough profiles-based classifiers

Experimental Results

Page 113: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

ground-truth

Experimental Results

~70% accuracy in predicting crime hotspots

predictions

Page 114: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Features encoding daily dynamics have morepredictive power than features extracted on amonthly basis

Relevance of high number of residents to predictcrime areas

increased ratio of residents -> more crime (incontrast with Newman’s thesis)

Entropy-based features are useful for predicting thecrime hotspots

high diversity of functions (home vs work) and highdiversity of people (gender and age) act as eyeson street decreasing crime (in line with Jacobs’thesis)

Relevant Features

Page 115: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Only 6 out of 68 features in the joint model areLondon Borough features, namely

%working population claiming out of workbenefits

Largest migrant population

% overseas nationals entering the UK

% resident population born abroad

Relevant Features

Page 116: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Our method captures the dynamics of a

place rather than making extrapolations from

previous crime histories. We can use it in areas

where people are less inclined to report crimes

Our method provides new ways of describing

geographical areas: novel risk-inducing or risk-

reducing features of geographical areas

Implications

Page 117: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Relevant Publications“Moves on the street: Predicting Crime Hotspots using aggregated

anonymized data on people dynamics” - A. Bogomolov, B. Lepri, J. Staiano, Leouze, E., N. Oliver, F. Pianesi, A. Pentland Journal of Big Data (Big Data Journal 2015)

“Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data” - A. Bogomolov, B. Lepri, J. Staiano, N. Oliver, F. Pianesi, A. Pentland 16th ACM International Conference on Multimodal Interaction (ICMI 2014)

"Flooding through the Lens of Mobile Phone Activity"Pastor-Escuredo, D., Torres Fernandez, Y., Bauer, J.M., Wadhwa, A., Castro-Correa, C., Romanoff, L., Lee, J.G., Rutherford, A., Frias-Martinez, V., Oliver, N., Frias-Martinez, E. and Luengo-Oroz, M. Proceedins of IEEE Global Humanitarian Technology Conference, GHTC 2014, Silicon Valley, CA, Oct 2014

Talk at WIRED 2013. London UK

http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver

Page 118: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Outline

• Brief Introduction

• Mobile Phones as Human Behavioral Sensors

• Individual modeling: • Boredom Inference

• MobiScore

• Aggregate modeling: Big Data for Social Good

• Crime prediction

• Conclusions

Page 119: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Conclusions

•Mobile Phones have huge potential to help their users

• Individual behavior modeling

• Persuasive computing, new services

• Mobile Phones have huge potential to help the world

• Mobile phones as sensors of aggregate human behavior

• Big Data for Social Good

Page 120: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

A Few Challenges

• Representativeness of the data, generalization

• Combination of data from multiple sources

• Real-time analysis and prediction

• Lack of ground truth intervention to validate and attribute causality

• HCI challenges when designing IntelligentAssistants

• Regulatory and legal barriers

• Potential privacy risks and unintendedconsequences

Page 121: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

Thanks!

[email protected]

@nuriaoliver

Page 122: Towards Individual and Aggregate Human Behavior Modeling ...simd.albacete.org/caepia15/wp-content/media-ftp/... · Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15,

We are hiring!

http://tidhiring.pdi.tid.es/