Sensing, Tracking and Contextualizing Entities in Ubiquitous Systems

(Sensoriamento, Rastreamento e Contextualização de Entidades em Computação Ubíqua)

Antonio Alfredo F. Loureiro loureiro@dcc.ufmg.br

Department of Computer Science Universidade Federal de Minas Gerais, Brazil

III Workshop do PPgCC

• Demography • 50,000 students

• 3,000 faculty members

• Campus • 10,000 football fields

• Other facts • More than 300 undergraduate and graduate courses

• 27 libraries with a collection of 1 million items

2

3

UFMG Campus Central area

New Computer Science Building

4

UFMG Campus Central area

5

Pampulha Lake Same region of UFMG Campus

6 Savassi Region Liberty Square

7

Serra do Curral Southern part of BH

8

Mercado Central

9

Administrative Center of the State Government Buildings designed by Niemayer

Two buildings: Minas (front) and Gerais (back)

Tiradents Palace (Governor’s Palace) Building is hanged by the ceiling

UFMG in the media

Latin-american university with the highest number of patents in the European Patent Office

10

Evolution of Computing Systems

11

Mobile Computer?

Pervasive Computing/Ubiquitous Computing

Emerging technologies are poised to personalize

the user experience radically—in real time and almost everywhere.

It’s not too early to prepare!

Processing Communication “Building Blocks” + +

Entities

Context Sensing

Elements

have

Broad spectrum

Cloud

Logical Physical

is obtained through

classified as sense

stored in

are of Different Types

must fit

12

Outline

• Sensing

• Context

• Localization and tracking

• Processing

• Concluding remarks

13

Fog Computing

Devices with sensing capability

Notification Data

Physical

entities

Logical

entities

Social sensors: • Person: social sensing • Information: origin,

evolution, dissemination • …

Physical sensors: • Objects • People • Environment • …

Virtual sensors: • Events given by a

predicate • ...

Application

Content

Communication

Finance

Event

Notification

Collaboration

Database

Runtime

Queue

Identity

Platform

Object

Storage

Cluster

Block Storage

Infrastructure

Cloud Computing

Complexity

Communication infrastructure Internet, WiFi, 3G, 4G, Bluetooth, Zigbee, ...

14

WSNs IoT

MANETs PSN VANETs Mobile Devices

...

Sensing in a broader context

Notificação Dado Notification Data

Heterogeneous data sources in the fog

HDF2 HDFn ... HDF1

WSNs IoT

MANETs PSN VANETs Mobile Devices

...

Sensing as a Service

Services & Applications

Heterogeneous data sources

In the cloud

HDC2 HDCn ... HDC1

Services & Applications

Fog #1

Fog #2

Fog #3

Fog #…

. . .

Sensing in a broader context

15

IoT

16

Participatory Sensor Networks

17

Notification Data

Physical

entities

Logical

entities

Application

Content

Communication

Finance

Event

Notification

Collaboration

Database

Runtime

Queue

Identity

Platform

Object

Storage

Cluster

Block Storage

Infrastructure

Complexity

Internet, WiFi, 3G, 4G, Bluetooth, Zigbee, ...

Some fundamental building blocks: •Heterogeneous data fusion •Localization & Tracking •Context awareness •Privacy •Cloud offloading • ….

Sensing as a Service

Fontes de dados heterogêneos

DS2 DSn ... DS1

Services and Applications

Theory

Observations/Experiments

18

WSNs IoT

MANETs PSN VANETs Mobile Devices

...

Sensing in a broader context

19

Sensing as a service

•Heterogeneous data sources

•Heterogeneous data fusion

Sensing in a broader context

What time should we go to the airport to catch our flight at 6pm?

• Location

• Traffic condition

• Weather condition

• Unexpected conditions

• Parking availability

• Flight/Airplane

20

Sensing in a broader context

Entities

• Technical name for “thing”

• Different classes with different properties • User

• Software

• Hardware

• ...

• Depending on the set of entities, we can have Internet of things, Web of things, PSNs, …

“Entities” have a key role

21

Context

• “Characterizes” a given entity • State, properties, data, …

• Classified as • physical

• logical

• Depends on the entity

22

Physical context

• Typically measured by a physical sensor

• Example: entity is a person • Define the person’s physical state

• It might depend on the person’s location (e.g., home, hospital)

23

Logical context

• There aren’t many sensors • Social “sensors” but others not currently available

• Example: entity is a person • Define the person’s logical state

• It might depend on people’s perception

24

Understanding the challenges

A broad spectrum

Challenge: treatment of individual sources and combination of them

25

A fundamental challenge

• We have a good idea of how to do information fusion in traditional sensor networks

• However, in a heterogeneous scenario we are far from there

Physical entities

Logical entities

Information fusion for physical + logical contexts

26

Information fusion in ubiquitous computing

• Entity can have different types of sensed data

• Sensed data has spatio-temporal attributes

• Information fusion becomes a dynamic process because of • mobility

• context change

• prediction

• ...

27

What do we need

• Take as an example, integrated circuit design

• For most of the fundamental building blocks in ubiquitous computing, we still need to establish the principles

28

Principles Tools Techniques Methodology

Ubiquitous computing and some fundamental building blocks • Localization and tracking (L&T) – possibly the very first one

• Communication

• Cloud computing

• Information fusion

• Mobility and topology information

• Security

• Cloud offloading

• ...

Challenge: provide useful services

29

Belo Horizonte, Brazil New York, USA

PSN Applicability: City Image

Belo Horizonte, Brazil New York, USA

School

Nightclub

Typical Monday

Bar

Work

Restaurant

Are transitions random?

Belo Horizonte, Brazil New York, USA

School

Nightclub

Typical Monday

Bar

Work

Restaurant

There are more favorable transitions than others

30

Mobility models for social communication or the city DNA

31

Next location Next location

Rejection Indiference Favoring

Mobility models for social communication or the city DNA

32

Next location Next location

Rejection Indiference Favoring

Mobility models for social communication or the city DNA • Example: checkins in Foursquare work as social sensors

33

PSN coverage

High coverage Some common geographic aspects Besides the economical aspect, cultural differences?

34

Sensing seasonality

Foursquare dataset 35

Sensing seasonality

Foursquare dataset 36

Smartphones and sensing

37

28% of American Adults use mobile and social location-based services

http://pewinternet.org/Reports/2011/Location/Report/Smartphones.aspx

Best sensing devices available in the market

L&T: Motivation

• Location awareness plays a key role in different networks

• Different entities require or can take advantage of some sort of location information: • Routing

• Data dissemination

• Applications

• Services

• Many others

Different requirements

38

Dimensions of L&T

• Types of entities

• Techniques: internal vs. external

• Roles

• QoS requirements

• Privacy

• …

39

What types of entities to L&T?

• Different possibilities depending on the scenario • User

• Application

• Service

• Protocol

40

Localization techniques

Different capabilities and possibilities

Different solutions

Interesting research/practical challenges

41

L&T: Roles

• Applications/services and protocols can benefit from location information

• Location and tracking can be used as: • Main role • Support role

• Beyond the location information, tracking techniques can be used to: • Detect and predict trajectories of single or multiple targets (basic service) • Provide customized services for users (will probably happen all time)

42

L&T: Roles

• Main role • L & T techniques are themselves the goals • For instance, driving or walking in an unknown terrain

• Support role • L & T techniques provide information for other entities • For instance, data dissemination for users, applications, …

Lots of possibilities/opportunities

43

Cooperative Target Tracking (CTT)

• Entities cooperate to perform the tracking task

• Target tracking techniques can be applied to augment the entities’ perception of the surrounding context

• Results can be used to actuate on the entity, surrounding environment, etc

44

Energy efficient GPS sensing with cloud offloading

Joint work with Microsoft Research, Redmond Best paper at ACM IPSN’12

45

Entities

Context Sensing

Elements

have

Broad spectrum

Cloud

Logical Physical

is obtained through

classified as sense

stored in

are of Different Types

must fit

How to process all these pieces of information?

46

Autonomic computing

The ability to learn and use that experience for future actions

47

Architecture of an autonomic element

• Fundamental part of the architecture • Managed elements

• Autonomic manager

• Responsible for: • providing its service

• managing its own behavior in accordance with policies

• interacting with other autonomic elements

48

Monitor Execute

Analyze Plan

Knowledge

Managed Element

Sensors Effectors

Autonomic Manager

Autonomic Element

Some perspectives

• Open data platform

• Personalized applications and services

• We need advance the state of the art in several different areas

• We need to better prepare our students!

49

Conclusion Combining the building blocks • Fusion different sensing sources

• Topology modeling, L&T

• Processing them

• Services for different wireless networks

50

Physical Sources

Logical Sources

Sensing Info Fusion L& T Context + +

Notificação Dado

Physical

entities

Logical

entities

Cloud Computing

Complexity

Fog Computing

Notification Data

Sensing Devices

Physical sensors: • Objects • People • Environment • …

Virtual sensors: • Events given

by a predicate • ...

Social sensors: • Person: social sensing • Information: origin,

evolution, dissemination • …

Application

Content

Communication

Finance

Event

Notification

Collaboration

Database

Runtime

Queue

Identity

Platform

Object

Storage

Cluster

Block Storage

Infrastructure

Sensing Actuation

Integration + Reasoning

Integration + Reasoning

Conclusion Integration & Reasoning

51

Conclusion

• Lots of opportunities!!

• This is the area of the “ice cream problem”

52

Algumas questões antes de terminarmos

• Qual é a principal função de uma universidade? • Formar pessoas

• Qual é o principal objetivo da Pós-Graduação? • Formar doutores

• Nos Estados Unidos, principal país de inovação das TICs, para onde vão os doutores formados? • Nove em 10 vão para a indústria!

• No Brasil, existe essa perspectiva? • Sim, sem inovação não teremos oportunidades como nação

• Minha sugestão: • Invista no seu doutorado!

53

Algumas questões antes de terminarmos

• O que é um doutorado? • Processo pelo qual você aprende a fazer pesquisa que gera inovação

• Em que área você deve fazer seu doutorado? • Escolha o seu “sorvete” favorito neste momento

• No futuro, você certamente experimentará outros sabores!

• Onde fazer o doutorado? • Em um bom programa

• UFMG é um deles (possivelmente o melhor do Brasil no momento)

54

• Envolve todos esses aspectos

• Seis professores

• 50+ alunos

55

Thank you!

Antonio Alfredo F. Loureiro loureiro@dcc.ufmg.br

Department of Computer Science Universidade Federal de Minas Gerais, Brazil