towards individual and aggregate human behavior modeling...
TRANSCRIPT
Towards Individual and Aggregate Human Behavior Modeling from Data in the Real-World
Nuria Oliver, PhD
Scientific Director
User, Data and Media Intelligence
Telefonica Research
Outline
• Brief Introduction
• Mobile Phones as Human Behavioral Sensors
• Individual modeling: • Boredom Inference
• MobiScore
• Aggregate modeling: Big Data for Social Good
• Crime prediction
• Conclusions
• ~20 full time researchers
• Successful internship program with ~25
interns/year
• Successful stage program for undergraduate students
• Visiting professors and scholars
• Open innovation: Barca, Ferran Adria,
UNGP, MIT, Brno Univ, UCIrvine…• Hiring!!
3. Big Data Analytics 4. Human Computer
Interaction and
Mobile Computing
1. Machine Learning
Personalization and
User Modeling (RecSys)
2. Multimedia Data
Analysis (Voice)
User, Data and Media Intelligence
LinasBaltrunas
AlexandrosKaratzoglou
Jose SanPedro
Aleksandar Matic
Jordi LuqueXavier Anguera
Enrique Frias
Martin PielotSouneil Park
Joan Serra
Carlos Segura
Machine Learning Approachesto Model Individual and
Aggregate Human Trait and Behavior from a variety of
Data sources: voice, mobile sensors, serviceusage data, mobile network
data...
We work on…
Human Behavior @Telefonica
Summary
• Built capabilities over the years on:• Data Science, Machine Learning and Data Analytics:
• Mobile data, voice, text
• Personalization and Recommendations
• Human-Computer Interaction and Mobile Computing
• User Modeling
• Generated over 40 patents
• Achieved international recognition: • 6 best paper awards and 5 best paper award
nominations
• 3 prestigious Marie Curie Fellowships and 2 EU projects
• Rising Talent Award, IEEE and ACM Senior Member Award, 10-year technical award (ICMI)
• External influence: GSMA white paper for African operators re Ebola; ITU, UN, MWC presentations
Our Work in the Media
Our Work in the Media
Can mobilephones be used
to model, understand and help their usersand the world
at large?
Outline
• Brief Introduction
• Mobile Phones as Human Behavioral Sensors
• Individual modeling: • Boredom Inference
• MobiScore
• Aggregate modeling: Big Data for Social Good
• Crime prediction
• Conclusions
6.8 billion subscribers
96% of world’s population (ITU)
Mobile penetration of 120% to 89% of population (ITU)
Emerging and developed regions
More time spent on our phones than watching TV or with our with
our partner (US and UK)
6.8 billion subscribers
96% of world’s population (ITU)
Mobile penetration of 120% to 89% of population (ITU)
Emerging and developed regions
More time spent on our phones than watching TV or with our with
our partner (US and UK)
Detecting Boredom from
Mobile Phone Usage
Research
UbiComp ‘15, Osaka, Japan
Martin
Pielot
Telefonica
Research
Tilman
Dingler
University of
Stuttgart
Jose
San Pedro
Telefonica
Research
Nuria
Oliver
Telefonica
Research
times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0
War on Attention*
* http://www.forbes.com/sites/onmarketing/2012/10/19/the-attention-war/
SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0
The trade we make:
Our attention so Internet companies
can pay their bills
Our engagement is now defined by push-
driven notifications... We’re “hunting and pecking” through our app grid a lot less; the
apps that notify us are rewarded with our
engagement (and our dollars).
The Deluge of Push-Driven Notifications
‚Attention is a limited resource—a person has only so much of it ‘ [Matthew B. Crawford]
Attention Economy: treating human attention as a scarce commodity[Davenport and Beck, 2001]
times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0
Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain
Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain
If the trade
attention for free servicesis to be sustained
we need to better
protect mobile phone users
Could Boredom
be part of the solution?
Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
2.0
Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
2.0
Boredom displeasure caused by “lack
of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who
does not have anything to do; it’s
someone who is actively looking for
stimulation”
[Eastwood, 2002]
Attention is not always scarce
Boredom displeasure caused by “lack of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who does not have
anything to do; it’s someone who is actively looking for
stimulation”
[Eastwood, 2002]
Mobile phones are a commonly
used tool to kill time when bored
[Brown et al. 2014]
Attention is not always scarce
Mobile phones are a commonly used tool to
fill or kill time when bored [Brown et al. 2014]
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
2.0
Boredom displeasure caused by “lack of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who does not have anything
to do; it’s someone who is actively looking for stimulation”
[Eastwood, 2002]
If phones knew when their
users are killing time
maybe they could suggest a
better use of that time
Is it possible to
detect
boredom from
mobile phone
usagepatterns?
Borapp – Sensor-Data Collection
Always
collected
Only
collected
if phone in
use
Sensor Description
Battery Status Battery level ranging from 0-100%
Notifications Time and type (app) of notification
Screen Events Screen turned on, off, and unlocked
Phone Events Time of incoming and outgoing calls
Proximity Screen covered or not
Ringer Mode Silent, Vibration, Normal
SMS Time of receiving, reading, and sending SMS
Sensor Description
Airplane Mode Whether phone in airplane mode
Ambient Noise Noise in dB as sensed by the microphone
Audio Jack Phone connected to headphones or speakers
Cell Tower The cell tower the phone is connected to
Data Activity Number of bytes up/downloaded
Foreground app Package name of the app in foreground
Light Ambient light level in SI lux units
Screen Orient Portrait or Landscape mode
Wifi Infos The WiFi network the phone is connected to
Experience Sampling
“Right now, I feel bored”
[5-point Likert scale]
Min. 6 times per day
Preferably triggered when
phone in use
Borapp – Experience Sampling
User Study: Data Collection
54 Participants
aged 21 – 46 (M = 30.6) years
11 female, 23male, 19 not disclosed
For two weeks in July 2014
Over 40M sensor log entries
4398 valid self-reports of boredom
0
500
1000
1500
0 1 2 3 4
Fre
qu
ency
Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree
Boredom Ground Truth
0
500
1000
1500
0 1 2 3 4
Fre
qu
ency
Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree
Absolute ground
truth
Bored: ratings 3, 4
446 (10.1%)
instances
Boredom Ground Truth
Absolute ground truth
Bored: ratings 3, 4
446 (10.1%) instances
Normalized ground
truth
Z-score per person
Bored: z > 0.25
1518 (34.5%) instances
0
400
800
1200
1600
2000
-2 -1 0 1 2
Fre
qu
en
cy
Normalized Subjective Boredom, (higher number = more bored than
usual)
Boredom Ground Truth
Category Example Feature Explanation
Context Semantic Location Home, work, other, unknown
Demographics Age, gender 38, female
Last Communication
Activity
Time last incoming
call
Time passed since somebody called the participants
Usage (intensity) Bytes received Number of bytes downloaded in the last 5 minutes
Usage (externally
triggered)
Number of
notifications
Number of notifications received in the last 5 minutes
Usage (idling) Number of apps Number of apps launched in the last 5 minutes
Usage (type) Most used app App used for the most time in the last 5 minutes.
35 Features, 7 Categories
RQ1: how well can phones detect
killing-time/boredom events from
these usage patterns?
RQ2: which usage patterns are
related to killing time with the
phone?
RQ3 is the model good enough to
be useful?
• Supervised machine learning classification: L2-
regularized logistic regression, linear SVMs and
Random Forests
• 5-fold cross validation
• Performance evaluation:
• Average Precision, Recall
• AUROC: area under the ROC curve information
about the ability of the model to rank users by their
probability to be bored
• Absolute and normalized boredom as ground
truth
Modeling Approach
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance |
Random Forest (AUCROC)
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance |
Random Forest (AUCROC)
Primary data set
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance |
Random Forest (AUCROC)
Primary data set
62.4% precision for
50% recall of boredom events
Boredom can be detected
from phone-usage patterns
with an accuracy of ca.
75% to 83% AUCROC
Take Away #1
RQ1: how well can phones detect
killing-time boredom events from
these usage patterns?
RQ2: which usage patterns are
related to killing time with the
phone?
RQ3 is the model good enough to
be useful?
Recency of
communication activity
i.e., time since last
incoming or
outgoing
communication;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower
Recency of communication activity i.e., time since last incoming or
outgoing communication;
Phase of the dayi.e., hour of the day,
ambient light
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower
Recency of communication activity i.e., time since last incoming or
outgoing communication;
Phase of the dayi.e., hour of the day, ambient light
Demographics,
i.e., gender and age;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower
Recency of communication activity i.e., time since last incoming or
outgoing communication;
Phase of the dayi.e., hour of the day, ambient light
Demographics, i.e., gender and age;
General usage
intensity i.e, phone out of
pocket, or time since
last phone use …;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower
Recency of communication activity i.e., time since last incoming or
outgoing communication;
Phase of the dayi.e., hour of the day, ambient light
Demographics, i.e., gender and age;
General usage intensity i.e, phone out of pocket, or time
since last phone use …;
Intensity of recent
usage i.e. # of unlocks, or
# of apps launched
in last 5 minutes, …
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower
Apps
Co-occur with being
bored
Co-occur with
NOT bored
… and uncategorized apps
Boredom is related toRecency of communication
Phase of the day
Demographics
Intensity and type of phone
usage
Type of used apps
Take Away #2
RQ1: how well can phones
detect killing-time boredom
events from these usage
patterns?
RQ2: which usage patterns are
related to killing time with the
phone?
RQ3 is the model good enough
to be useful?
Borapp2
Model running on Mobile
Phone
Using primary data set
Constantly infers the
user’s boredom state on
the fly
Suggests Reading Buzzfeed Articles
User Study 2: Data Collection
16 Participants (different from 1st
study)
aged 18 – 51(M = 39) years
13 male, 2 female, rest did not disclose
For two weeks in Feb 2015
941 Buzzfeed recommendations
48% when predicted bored
Measure 1: Click-ratioFraction of times people
clicked on notification
(Mdn)
8% when not bored
20.5% when bored
(as inferred by the model)
Difference significant
z = -2.102, p = .018
Large effect
r = -.543
Measure 2: Engagement-ratio
Fraction of times people
spent more than 30 sec
reading (Mdn)
4% when not bored
15% when bored
(as inferred by the model)
Difference significant
z = -2.102, p = .018
Large effect
r = -.511
When inferred to be bored,
participants were …
More likely to clickMore likely to read
for > 30 seconds
The generic model was
powerful enough to
create significant, large
effects on click- and
engagement-ratios
Take Away #3
Impact in the Press…
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
2.0
Application Scenarios
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
2.0
Recommend
content to
alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not
necessarily boredom-curing activities
Encourage embracing
boredom
Recommend content to
alleviate boredom
Shield user from
non-important
interruptions
during non-bored
times
Suggest useful but not necessarily boredom-curing
activities
Encourage embracing boredom
Recommend content to
alleviate boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful
but not
necessarily
boredom-curing
activities
Encourage embracing
boredom
Recommend content to
alleviate boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful
but not
necessarily
boredom-curing
activities
Encourage embracing
boredom
Being bored
is good for
you
Why don’t
you turn me
off?
Recommend content to
alleviate boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not
necessarily boredom-curing
activities
Encourage
embracing
boredom
Relevant Publications
http://doi.acm.org/10.1145/1864349.1864371
http://www.youtube.com/watch?v=_p7n_pn7xaE
TEDx Las Ramblas, Feb 2012
""When Attention is not Scarce: Detecting Boredom from Mobile Phone
Usage"
Pielot, M., Dingler, T., San Pedro, J. and Oliver, N.
Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)
Best paper award!
"Boredom-Computer Interaction: Boredom Proneness and SmartPhone Use"
Matic, A., Pielot, M. and Oliver, N.
Proc of ACM Int Conf on Ubiquitous Computing (Ubicomp 2015)
Outline
• Brief Introduction
• Mobile Phones as Human Behavioral Sensors
• Individual modeling: • Boredom Inference
• MobiScore
• Aggregate modeling: Big Data for Social Good
• Crime prediction
• Conclusions
MobiScore: Credit Score Inference
With J. San Pedro, D. Proserpio & J. Gonzalez
Credit Scores, a number that represents an assesment or
the likelihood that a person will repay his or her debt, are
widely spread in the world enabling the growth of
consumer credit and transactional operations…
… They are calculated upon applicants’ creditand financial history…
… however, because of the data that is used to
compute credit scores, not everybody is
scorable…
Thin-file and no-hit:Not only a situation
for Emergent
Economies
2.5B
UNBANKED UNDERBANKED
8M
IMMIGRANTS
1.1M/year
NEW CLIENTS
No Positive Information
STUDENTS&YOUNG WORKERS
6.8 billion subscribers
96% of world’s population (ITU)
Mobile penetration of 120% to 89% of population (ITU)
Emerging and developed regions
More time spent on our phones than watching TV or with our with
our partner (US and UK)
6.8 billion subscribers
96% of world’s population (ITU)
Mobile penetration of 120% to 89% of population (ITU)
Emerging and developed regions
More time spent on our phones than watching TV or with our with
our partner (US and UK)
Would it be possible to build
an alternative score using
mobile behavioral data?
Credit
Score
Model
Probability of Default @30 and @90 days
Consumption
Social
Mobility
Mobile Behavioral
Data (CDR) MobiScore
Customer Relation
Management Data (CRM)
Demographic
Product
Socio-economic
Typical Mobile Behavioral Data• CDR
• SMS
Consumption Social Network Mobility
Call duration In/Out Degree Radius of gyration
N. Events Delta w.r.t time window Travelled distance
Lapse between events Unique Calls per day Rate of popular antennas
Reciprocated events Unique SMS per dayRegularity of popular
antennas
… … …
HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD CD_ERB CD_CCC QT_DUR
20:05:31 XXX YYY 3 11 20140519 2 1562 568 33
… … … … … … … … … …
HR_ORG TLFN_A TLFN_B CD_GEO_A CD_GEO_B DT_ORG CD_SNTD QT_TRFG
15:53:54 XXX ZZZ 3 25 20140506 2 1
… … … … … … … …
Customer Relation Management (CRM) Data
• Demographic information: age, gender
• Socio-economic• Derived from home address
• Payment variables: late payments
• Product features: device brand, device OS, device type, line type, line status, line quantity
• Time since activation
Credit default reports, i.e. pending
balance that is considered to be
uncollectible
More than 30 days in arrears
More than 90 days in arrears
Ground Truth
Credit
Score
Model
Probability of Default @30 and @90 days
Consumption
Social
Mobility
Mobile Behavioral
Data (CDR) MobiScore
Customer Relation
Management Data (CRM)
Demographic
Product
Socio-economic
Building Credit Scores from Data
• Mobile phone usage logs• 3-month period (Jan-March 2014)
• Fully anonymized
• Over 35 million call events and 11 million SMS events
• Customer Relation Management Data
• Credit default reports, i.e. pending balance that is considered to be uncollectible
• More than 30 days in arrears
• More than 90 days in arrears
Of over 60,000 individuals in a Latin American country
Mo
bile
Da
taFin
an
cia
l Info
Modeling Approach
• Supervised machine learning: L2-regularized logistic regression, linear SVMs and Gradient Boosted Trees
• 5-fold cross validation
• Performance evaluation:• Average Precision
• AUROC: area under the ROC curve information about the ability of the model to rank customers according to their probability of default
• Default @30 days and @90 days
2 weeks 1 month 3 months 2 weeks 1 month 3 months
GBT 63.0 64.5 67.5 68.5 69.4 71.6
@30 LR 62.2 64.0 66.4 67.6 68.8 70.7
SVM 62.1 63.9 66.7 67.9 68.8 70.6
GBT 63.1 64.4 67.5 70.2 70.8 72.5
@90 LR 62.4 64.5 67.4 68.7 70.5 72.1
SVM 63.1 64.1 67.2 69.7 70.3 72.1
Classification Performance (AUROC)
• GBTs outperform other classifiers thanks to their higher degree of complexity and flexibility
CRM + Voice CDRs CRM + Voice+SMS CDRs
Comparison with State of the Art
Implications
• MobiScore leverages passively collected mobile data to accurately infer default risk
• CDR + CRM features achieve significantly better performance than state-of-the-art models
• MobiScore opens the door to alternative credit score models that would enable millions of people to get access to credit
Relevant Publications
"MobiScore: Towards Universal Credit Scoring from
Mobile Data"
Proserpio, D., San Pedro, J. and N. Oliver
Proc. of Int. Conf on User Modeling (UMAP 2015)
"Prediction of Socioeconomic Levels using Cell Phone
Records", Victor Soto and Vanessa Frias-Martinez and
Jesus Virseda and Enrique Frias-Martinez, International
Conference on User Modeling, Adaptation and
Personalization, UMAP'11, Industrial Track, Girona, Spain,
2011
Outline
• Brief Introduction
• Mobile Phones as Human Behavioral Sensors
• Individual modeling:• Boredom Inference
• MobiScore
• Aggregate modeling: Big Data for Social Good
• Crime prediction
• Conclusions
6.8 billion subscribers
96% of world’s population (ITU)
Mobile penetration of 120% to 89% of population (ITU)
Emerging and developed regions
More time spent on our phones than watching TV or with our with
our partner (US and UK)
Cell Phones as Sensors of Human Activity
May 19, 2011, 7:06 pm The Sensors Are Coming!By NICK BILTON
Telecom / WirelessNEWSCellphones for ScienceScientists want to put sensors into everyone's hands
Digital footprints enable large-scale
analysis of human behavior
Sensors of aggregated human activities used to
monitor citizens’ interactions and mobility patterns (Song et
al. 2010; Dong et al. 2011)
understand individual spending behaviors (Singh et al.,
2013) and financial responsibility (San Pedro et al, 2015)
predict socio-economic indicators of territories (Eagle et al.,
2010; Soto et al., 2011; Smith-Clarke et al., 2014)
model spreading of malaria (Wesolowski et al., 2012) and
H1N1 (Frias-Martinez et al., 2011)
infer people’s traits (deOliveira et al. 2010, Staiano et al.,
2012; Chittaranjan et al. 2013; de Montjoye et al., 2013) and
states (Bogomolov et al., 2013)
Phones as Social Sensors
Big Data for Social Good
Crime Prediction
Analysis of impact of floods
http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver
Crime
Work with Bogomolov, A., Lepri, B., Staiano, J., Pianesi, F., Pentland, A.
Affects quality of life and economic
development both at the national and local level
Several studies explore relationships between
crime and socio-economic variables: education,
income, unemployment, ethnicity, …
Several studies have shown significant
concentrations of crime in small geographical
areas: crime hotspots
Crime
T1: Natural surveillance as key deterrent for
crime: people moving around are eyes on the
street (Jacobs, 1961)
high diversity among the population and
high number of visitors -> less crime
T2: Defensible space theory (Newman, 1972)
high mix of people -> more crime
Crime and Urban Environment
People-centric perspective vs Place-centric perspective
people-centric perspective used for
individual or collective criminal profiling
place-centric perspective used for
predicting crime hotspots
Crime Prediction
Data-driven and place-centric approach to
crime prediction
Multimodal approach: people dynamics
derived from mobile network data and
demographics
European metropolis: London
Prediction of crime hotspots and not criminals
profiling
Our Approach
Smartsteps Dataset:
for each of the Smartsteps cells a variety of demographic
and human dynamics variables were computed every
hour for 3 weeks (from December 9 to December 15, 2012
and from December 23, 2012 to January 5, 2013)
Criminal Cases Dataset: criminal cases for December 2012 and for January 2013
London Borough Profiles Dataset:
open dataset containing 68 metrics about the population
of a particular geographic area
Multimodal Approach: Data
• Footfall count: Shows the trend in footfall in a
specified area hourly, daily, weekly and
monthly. Provides a basic profile of the crowd.
• Catchment area: Shows which postal sectors
are your customers coming from by hour, day,
week and month. Shows the “battleground”
for two sites.
• Transport mode: Shows flows of crowds from
any two points, segmented by road, air, train,
etc.
SmartSteps
For each cell and for each hour the dataset contains:
an estimation of how many people are in the cell
the percentage of these people at home, at work or
just visiting the cell
the gender splits (male vs. female)
the age splits (0-20 years, 21-30 years, 31-40 years, …)
SmartSteps Data
Crime geolocation for 2 months (December 2012
– January 2013)
All reported crimes in UK specifying month and
year and not specific day/time
Median crime value (=5) used as threshold
Spatial granularity of borough profiles is at LSOA
levels: LSOA are small geographical areas
defined by UK Office for National Statistics (mean
population: 1500)
Crime Data
68 metrics about the population of a specific
geographical area: demographics, households,
migrant population, employment, earnings, life
expectancy, happiness levels, house prices, etc.
Spatial granularity of borough profiles is at LSOA
levels:
LSOA are small geographical areas defined by UK
Office for National Statistics (mean population: 1500)
London Borough Profiles Data
From Smartsteps data we extract
1st order features (mean, median, min.,
max., entropy, etc.)
2nd order features on sliding windows of
variable length (1 hour, 4 hours, 1 day,
etc.) to account for temporal patterns
Feature Extraction
Feature Selection
Mean decrease in Gini coefficient of
inequality
the feature with maximum mean decrease
in Gini coefficient is expected to have the
maximum influence in minimizing the out-of-
the-bag error
The feature selection process produced a
reduced subset of 68 features (from an initial
pool of about 6000 features)
Classification Approach
Binary classification task: high crime area vs low
crime area
10-fold cross-validation approach
Classifier: Random Forest (RF)
RF overcomes logistic regression, support vector
machines, neural networks, decision trees
Smartsteps-based classifier significantly outperforms baseline
majority and borough profiles-based classifiers
Experimental Results
ground-truth
Experimental Results
~70% accuracy in predicting crime hotspots
predictions
Features encoding daily dynamics have morepredictive power than features extracted on amonthly basis
Relevance of high number of residents to predictcrime areas
increased ratio of residents -> more crime (incontrast with Newman’s thesis)
Entropy-based features are useful for predicting thecrime hotspots
high diversity of functions (home vs work) and highdiversity of people (gender and age) act as eyeson street decreasing crime (in line with Jacobs’thesis)
Relevant Features
Only 6 out of 68 features in the joint model areLondon Borough features, namely
%working population claiming out of workbenefits
Largest migrant population
% overseas nationals entering the UK
% resident population born abroad
Relevant Features
Our method captures the dynamics of a
place rather than making extrapolations from
previous crime histories. We can use it in areas
where people are less inclined to report crimes
Our method provides new ways of describing
geographical areas: novel risk-inducing or risk-
reducing features of geographical areas
Implications
Relevant Publications“Moves on the street: Predicting Crime Hotspots using aggregated
anonymized data on people dynamics” - A. Bogomolov, B. Lepri, J. Staiano, Leouze, E., N. Oliver, F. Pianesi, A. Pentland Journal of Big Data (Big Data Journal 2015)
“Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data” - A. Bogomolov, B. Lepri, J. Staiano, N. Oliver, F. Pianesi, A. Pentland 16th ACM International Conference on Multimodal Interaction (ICMI 2014)
"Flooding through the Lens of Mobile Phone Activity"Pastor-Escuredo, D., Torres Fernandez, Y., Bauer, J.M., Wadhwa, A., Castro-Correa, C., Romanoff, L., Lee, J.G., Rutherford, A., Frias-Martinez, V., Oliver, N., Frias-Martinez, E. and Luengo-Oroz, M. Proceedins of IEEE Global Humanitarian Technology Conference, GHTC 2014, Silicon Valley, CA, Oct 2014
Talk at WIRED 2013. London UK
http://www.wired.co.uk/news/archive/2013-10/17/nuria-oliver
Outline
• Brief Introduction
• Mobile Phones as Human Behavioral Sensors
• Individual modeling: • Boredom Inference
• MobiScore
• Aggregate modeling: Big Data for Social Good
• Crime prediction
• Conclusions
Conclusions
•Mobile Phones have huge potential to help their users
• Individual behavior modeling
• Persuasive computing, new services
• Mobile Phones have huge potential to help the world
• Mobile phones as sensors of aggregate human behavior
• Big Data for Social Good
A Few Challenges
• Representativeness of the data, generalization
• Combination of data from multiple sources
• Real-time analysis and prediction
• Lack of ground truth intervention to validate and attribute causality
• HCI challenges when designing IntelligentAssistants
• Regulatory and legal barriers
• Potential privacy risks and unintendedconsequences