objective monitoring of the obesogenic behaviour: from ... · location unit is the geohash or...

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Objective Monitoring of the Obesogenic Behaviour: From smart Raw Data to Privacy Preserving Statistics

1

Christos Diou, Anastasios Delopoulos*

Multimedia Understanding GroupInformation Processing LaboratoryDepartment of Electrical and Computer EngineeringAristotle University of ThessalonikiGreece

Workshop on trusted smart statistics, Wiesbaden, Jan 30-31, 2018

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

H2020 funding→ 2016-2020

13 organizations→Universities

→Schools

→Obesity clinics

→Technical companies

→Telecommunications provider

→Public Health Authorities

5 countries

Big Data Agains Childhood Obesity

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Obesity is a threat for health and economy

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Big data for evidence-based policies

Thousands of children→Schools→Clinics

Behavioural data →Personal Behavioural Patterns →Behavioural Risk Factors

Local Environment Conditions from relevant areas

BigO is built around the “citizen-scientist” model, which relies on individuals sharing their behavioral data

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

BigO Community

Reaching out to more than 23.000 school children to become BigO citizen scientists and share their behavioural data

Engage ~7.000

Engaging more than 2.000 children at 3 obesity clinics

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Need of multi-level approaches

Obesity risk depends on:→ The way we eat

→ What we eat

→ How we move

→ The way we sleep

These decompose into a long list of personal behavioral patterns

Highly correlated, in a causal way, with the conditions of localurban, social, regulatory and economic environment Based on Davison KK, Birch LL. Childhood overweight: a contextual model and

recommendations for future research. Obesity reviews. 2001 Aug 1;2(3):159-71

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

BigO Data collection

Photos - Food & Ads

Location – GPS Physical Activity

Self-reporting

Sleep

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

The BigO System

BigO Cloud Storage + Analytics Platform

Public HealthOfficials

Healthcare professionals

School

BigO community

Children

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Measurable quantities that provide information about an individual’s behaviour

• Objective measurements or timed self-reports that more accurate than questionnaires

Behavioural indicators

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Activity counts in time interval

• Steps in time interval

• Activity type

• Transportation mode used

• Visited POI and POI type

• Bite sequence during a meal

• Meal occurrence (self-reported with pictures, or detected from smartwatch)

• Sleep start and stop times

Base behavioural indicators

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Activity counts (computed from a filtered version of the acceleration vector)

• Steps (Gu et al., 2017)

• Activity type (our implementation)

Physical activity indicators

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Detect visited locations and means of transportation between them

Location and transportation

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Modified DBSCAN algorithm (Luo et al., 2017) for POI detection

• Detection of home and school

• Cross-reference with Foursquare and Google places for the rest

• Trip detection: Based on average speed

• Transportation mode detection based on accelerometer, using SVM models

POI, trip and transportation mode detection

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Measures of behavior: Analyze meal microstructure

Detect bites during meals

In the wild

Smartwatch captures accelerometry + gyroscope data

Signal Processing + Deep Learning

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Identify eating cycles using the smartwatch: Model hand micromovements

Measures of behavior: Analyze meal microstructure

Kyritsis K, Diou C, Delopoulos A. Food Intake Detection from Inertial Sensors Using LSTM Networks. In International Conference on Image Analysis and Processing. 2017.

Kyritsis, K., Diou, C., & Delopoulos, A. Modeling Wrist Micromovements to Measure In-Meal Eating Behavior from Inertial Sensor Data. IEEE journal of biomedical and health informatics, 2019

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Using the base behavioural indicators, we extract a variety of derived behavioural indicators, such as• Fast food visit frequency

• Frequency of eating at home

• Eating schedule adherence

• Average physical activity level at work

• Average physical activity level in the afternoon

• Frequency of visits to gym or sports facilities

• Number of exercise sessions per week

• Frequency of walking/cycling to work

• Distribution of use of different modes of transportation

• Average sleep duration

• …

Derived behavioural indicators

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Behavioural profile (1)

• Adopt a list of certain POI Types

• For each transition

• Transition probabilities between POI types

• Probability distribution of behaviours for

• nodes

• edges

• Summary of the individual’s behaviour

• Can be used to compare the behaviours of individuals

• Different profile for workdays and non-workdays

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Behavioural profile (2)

Transportation distributions:

Transportation mode

Activity level

…

Duration

POI distributions:

Activity level

Meal occurrence

Eating speed

Sleep occurrence

…

Duration

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Behavioural profile similarity map (t-SNE)

Behaviour profile similarity was computed using the eigenvectors of the

transition matrix

Useful for clustering Behavioural Profiles

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Privacy by design: Geohash votes

Pseudonymization→ Real names out of the system→ Analytics on Geohashes not on

persons

Innovative handling of location data

→ votes to elements of {geohashes} x {behaviours}○ Cecilia was walking fast on

Odengatan street of Stockholm at 9:15 am

○ increase votes(u6sce5, ‘walk fast’, 9) by one

→ k-anonymity○ Cast the vote to all subareas of u6sce

if less than k votes

6 digit geohashes in central Stockholm

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Aggregation functions to extract statistics

• For example:

Average: 𝑓1 𝑖, 𝑗 = 𝐸 𝑈,𝐺,𝑇,𝐵𝑖{𝐵𝑖|𝐺 = 𝑗}

• 𝑈: Users

• 𝑇: Time slots

• 𝐺: Geohashes

• 𝐵𝑖: Values of the 𝑖-th behavioural indicator

• 𝑖, 𝑗: Indicators of the 𝑖-th behavioural indicator and 𝑗-th geohash, respectively

Aggregating geohash votes

21

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Privacy by design: Timelines

22

• Capture individual behaviour, without revealing exact location

• Types of POIs visited, along with transportation modes between POIs• Not POI location!

• Granularity in privacy control• Behavioural indicators for each POI

type

• Can include short geohash (to designate a broad POI area)

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Heatmap - behaviour

23

Activity counts

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Local environment conditions (LECs)

• Examples• Average number of supermarkets and grocery stores in 100, 1000 and 5000

meters

• Open spaces or public parks in neighbourhood

• Average distance to public transportation

• School exercise programs

• Unemployment rate

• Education level

• Exposure to food advertising in the urban environment

• … many more

Location profile

24

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Heatmap - location

25

Number of fast food outlets

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Location unit is the geohash or region and is represented as a vector

𝒙 = 𝑥1, … , 𝑥𝑝𝑇

where each 𝑥𝑖 is the value of a LEC

Behaviour is a scalar, 𝑦, representing an aggregated value of a behaviouralindicator across subjects in a region, or a value associated with the behavioural profile of subjects (future work).

Learning goal: 𝑦 = 𝑓(𝒙)

Learning models for prediction and inference

26

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Types of learning problems

27

Observed overall behaviour of people living at a geohash

Observed behaviourof people visiting a geohash

Which LECs play a role and how much

What is the estimated average behaviour for locations with no measurements

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Linear and generalized linear models:• Hypothesis tests [LEC 𝑥𝑖 does not affect the measured behaviour 𝑦]

• Confidence intervals [LEC 𝑥𝑖 determines the output with a factor in [𝑤𝑖0, 𝑤𝑖1] with 95%] probability

• Effect size measures [𝑅2 and adjusted 𝑅2, Cohen’s 𝑑, odds ration, risk ratio]

• Inference using ensemble methods (random forests, EXTRA trees etc)• Breiman et al, assessing variable importance through out-of-bag error

• Also through the average impurity decrease

Inference

28

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

• Predict the new average behavioural indicator 𝑦 + Δ𝑦 from its current value, 𝑦, when the location characteristics change from 𝒙 to 𝒙 + Δ𝒙.• The “new” location is actually a modified version of the previous (change in

one LEC)

Prediction

29

Multimedia Understanding Group, Aristotle University of Thessaloniki, Greece

Thank you!

Prof. Anastasios DelopoulosDept. of Electrical & Computer Engineering

Aristotle University of ThessalonikiGreece

adelo@eng.auth.gr

Dr. Christos DiouDept. of Electrical & Computer Engineering

Aristotle University of ThessalonikiGreece

diou@auth.gr