a context and user aware smart notification system

A Context and User Aware

Smart Notification

System

Fulvio Corno

Luigi De Russis

Teodoro Montanaro*

http://jol.telecomitalia.com/j

olswarm/

http://elite.polito.it/

2

Outline

1. Context and Motivation

2. Goal

3. Architecture

4. Prototype

5. Preliminary results

6. Conclusion

7. Future work

3

Context

Context

Infographic from "The Connectivist": growing of IoT connected devices

(http://www.theconnectivist.com/2014/05/infographic-the-growth-of-the-internet-of-things/)

4

Motivation

Motivation

5

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

6

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

7

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

8

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

The number of notifications is growing

9

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

The number of notifications is growing

10

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

Nowadays the same notification is replicated on all available devices

The number of notifications is growing

11

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

Nowadays the same notification is replicated on all available devices

The number of notifications is growing

12

Motivation

Motivation

IoT devices can generate, receive and show different kinds of notifications

Nowadays the same notification is replicated on all available devices

The number of notifications is growing

The benefit of displaying the same

notification on all available devices

could put user patience to a hard test

13

Analyze how machine learning approach can improve

IoT notification user experience

Goal

Goal

14

Analyze how machine learning approach can improve

IoT notification user experience

Goal

Goal

15

Analyze how machine learning approach can improve

IoT notification user experience

Goal

Goal

Develop a system able to filter incoming notifications

depending on:

• Notification information

• Environment status

• User context

• User habits

16

Analyze how machine learning approach can improve

IoT notification user experience

Goal

Goal

Develop a system able to filter incoming notifications

depending on:

• Notification information

• Environment status

• User context

• User habits

17

Analyze how machine learning approach can improve

IoT notification user experience

Goal

Goal

Develop a system able to filter incoming notifications

depending on:

• Notification information

• Environment status

• User context

• User habits

Evaluate machine learning approach

18

We propose:

Architecture

Architecture

19

We propose: A modular architecture

Architecture

Architecture

20

We propose: A modular architecture

Architecture

Architecture

21

We propose: A modular architecture

Architecture

Architecture

22

We propose: A modular architecture aware of

Architecture

Architecture

23

We propose:

Environment status

(e.g., weather information,

current date and time)

A modular architecture aware of

Architecture

Architecture

24

We propose:

Environment status

(e.g., weather information,

current date and time)

User context (e.g.,

location, status, current

activity),

A modular architecture aware of

Architecture

Architecture

25

We propose:

Environment status

(e.g., weather information,

current date and time)

User context (e.g.,

location, status, current

activity),

User habits

A modular architecture aware of

Architecture

Architecture

26

We propose: A modular architecture

Architecture

Architecture

27

We propose:

Decision maker: Machine Learning

algorithm makes decisions (best devices

+ best modes + best moment).

Architecture

Architecture

28

Architecture: example

Architecture

29

Architecture: example

Architecture

Mario is in a

meeting

30

Architecture: example

Architecture

Mario is in a

meeting

31

Architecture: example

Architecture

Every meeting lasts at least 2

hours

Mario is in a

meeting

32

Architecture: example

Architecture

Every meeting lasts at least 2

hours

Mario is in a

meeting

33

Architecture: example

Architecture

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

34

Architecture: example

Architecture

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

35

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

36

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

37

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

38

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Notify:

• At 18:10

• on his personal

smartphone

• Sound

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

39

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Notify:

• At 18:10

• on his personal

smartphone

• Sound

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

40

Architecture: example

Architecture

Notification: the toilet

paper has just finished

Notify:

• At 18:10

• on his personal

smartphone

• Sound

Time: 17:00

Meeting started at

16:00

Wife is at home

Every meeting lasts at least 2

hours

Mario is in a

meeting

41 Prototype

Prototype implementation

42 Prototype

Prototype implementation Aim: evaluate machine learning approach to decide

• who should receive an incoming notification;

• the best moment to show the notification;

• the best device(s)

• the best mode to notify the incoming notification

43 Prototype

Prototype implementation Aim: evaluate machine learning approach to decide

• who should receive an incoming notification;

• the best moment to show the notification;

• the best device(s)

• the best mode to notify the incoming notification

44 Prototype

Prototype implementation

Preliminary version of

Aim: evaluate machine learning approach to decide

• who should receive an incoming notification;

• the best moment to show the notification;

• the best device(s)

• the best mode to notify the incoming notification

45 Prototype

Prototype implementation

Preliminary version of

Aim: evaluate machine learning approach to decide

• who should receive an incoming notification;

• the best moment to show the notification;

• the best device(s)

• the best mode to notify the incoming notification

46 Prototype

Prototype implementation

47 Prototype

Prototype implementation

Preliminary version of

48 Prototype

Prototype implementation

Preliminary version of

Dataset

49 Prototype

Prototype implementation

Preliminary version of

Dataset Algorithms

50

Prototype implementation

Prototype

Dataset

51

Prototype implementation

Prototype

94 people over 9 months

monitored through

smartphones in 2004:

• Sender

• Receiver

• Type of notification

• Timestamp of receipt

• User current location

Dataset

52

Prototype implementation

Prototype

94 people over 9 months

monitored through

smartphones in 2004:

• Sender

• Receiver

• Type of notification

• Timestamp of receipt

• User current location

Dataset

Synthetic data:

• User current

activity

• Available devices

for the user

• Target device.

53

Prototype implementation

Prototype

94 people over 9 months

monitored through

smartphones in 2004:

• Sender

• Receiver

• Type of notification

• Timestamp of receipt

• User current location

Dataset

Synthetic data:

• User current

activity

• Available devices

for the user

• Target device.

Real + synthetic dataset:

165,289 samples, almost one per

each hour of the day

(the missing samples are related

to hours in which users turned

off their smartphones)

54

Prototype implementation

Prototype

94 people over 9 months

monitored through

smartphones in 2004:

• Sender

• Receiver

• Type of notification

• Timestamp of receipt

• User current location

Dataset

Synthetic data:

• User current

activity

• Available devices

for the user

• Target device.

Real + synthetic dataset:

165,289 samples, almost one per

each hour of the day

(the missing samples are related

to hours in which users turned

off their smartphones)

Information collected by Decision Maker in previous example

{

“notification“:{

“senderName“:“mySmartHome“,

“type“:“smart Home Notification“,

“receiptTimestamp“:“1447347600“

},

“userStatus“: {

“senderId“: “359“,

“currentActivity“:“STILL“,

“currentActivityConfidence“:“50%“,

“availableDevices”:[“deviceId”:”23”]

},

“deviceStatus“:{

“deviceId“:”23”,

“category“:”Smartphone”,

“currentStatus“:”On”,

“currentMode“:”Ring”,

“wifiStatus“:” Connected through MOBILE”,

“batteryLevel“:” 57%”,

“batteryStatus“:”BATTERY_STATUS_NOT_CHARGING”

}

}

55

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

56

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

57

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

58

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

Training dataset: 80% of data

Tests dataset: 20% of data

59

Prototype implementation

Prototype

Simplified version of the Decision

maker:

• only one device as receiver;

• only one available mode for each

device;

• no decision about the best time

to deliver the notification;

• not aware of environment

context

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

Training dataset: 80% of data

Tests dataset: 20% of data

60

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

Training dataset: 80% of data

Tests dataset: 20% of data

61

Prototype implementation

Prototype

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

Training dataset: 80% of data

Tests dataset: 20% of data

62

Prototype implementation

Prototype

Three machine learning

algorithms:

1. Support Vector Machine

2. Gaussian Naïve Bayes

3. Decision Trees.

Machine learning algorithms

Dataset

Real + synthetic data (165,289

samples)

Training dataset: 80% of data

Tests dataset: 20% of data

63

Preliminary results

Preliminary results

64

Preliminary results

Preliminary results

96,10%

83,40%

93,90%

0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

Support

Vector

Machine

Gaussian

Naive Bayes

Decision Trees

Percentage of corrected predictions obtained

with used algorithms

65

Preliminary results

Preliminary results

CPU time (in seconds) for a training phase with

33058 samples

96,10%

83,40%

93,90%

0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

Support

Vector

Machine

Gaussian

Naive Bayes

Decision Trees

5801,1

12,9 13,9

1

10

100

1000

10000

Support

Vector

Machine

Gaussian

Naive Bayes

Decision Trees

Percentage of corrected predictions obtained

with used algorithms

66

Preliminary results

Preliminary results

Average CPU time (in milliseconds) for each

notification classification

Support Vector Machine 40,22 ms

Gaussian Naive Bayes 0,31 ms

Decision Trees 0,001 ms

96,10%

83,40%

93,90%

0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

Support

Vector

Machine

Gaussian

Naive Bayes

Decision Trees

Percentage of corrected predictions obtained

with used algorithms

CPU time (in seconds) for a training phase with

33058 samples

5801,1

12,9 13,9

1

10

100

1000

10000

Support

Vector

Machine

Gaussian

Naive Bayes

Decision Trees

67

Conclusion Obtained results demonstrated that our system uses a promising technique to

manage the problem of overwhelming notifications.

Specifically, the machine learning approach was tested through 3 different

algorithms and SVM and DT seem to be the most promising one.

Conclusion

Future work:

Define a new dataset to include all the needed real information

Development of a system to collect real data and real notifications

Careful evaluation of the machine learning algorithms

Enhancement of prototype to include unconsidered blocks

68

Thank you

Future work

Notification Collector (beta):

Android app to collect real data

https://goo.gl/pLMWSG

To contribute: download it! Requirement: Android 5 (Lollipop)

We collect (anonymously):

• Incoming notification info (no

content)

• User current activity

• User current location

• Device status

• User feedback