Towards Application Development for the Internet of ThingsPankesh Patel, Animesh Pathak, Thiago Teixeira, and Valérie Issarny

ARLES project-teamINRIA Paris-RocquencourtLisbon, December 12, 2011

``Things’’: Salient Features

• May have sensors attached.

• May have actuator attached.

• Can communicate with other Things.

• Can be involved in the information

exchange between real/physical and

``digital/virtual’’ world. [IoTRoadMap]

[IoTRoadMap] De Saint-Exupery, Internet of things, strategic research roadmap, Research report 2009. (URL: http://ec. europa. eu/information—society/policy/rfid/documents/in—. cerp. Pdf )

2

Internet of Things (IoT)

``A global network infrastructure that connects physical and virtual things” [CASAGRAS Project].

[CASAGRAS Project] : http://www.rfidglobal.eu/userfiles/documents/CASAGRAS26022009.pdfFigure credit : http://www.leaderstudio.net/iot/images/internet-of-things.jpg

3

- 4

IoT– Application Domains

• Intelligent buildings

• Automatic energy metering

• Home automation

• Wireless monitoring

• Medical technology, Healthcare,

• Retail, Logistics, Supply chain management

• Environment monitoring

• …

- 5

Outline

• Motivation

• Related work

• Initial research contributions

• Behaviours in IoT applications

• IoT domain model

• Related work in IoT domain model

• Concluding Remarks

• Future work

- 6

Motivation

• Domain experts have to deal with • low-level system issues

• low-level design of distributed protocols

• Requires considerable CS background

• Shifts their focus from application logic

Previously addressed in:Wireless Sensor Network (WSN)

• Hundreds to thousands of largely similar devices

• Typically deployed in limited geographical area

• Focus mostly on scale

7

Figure credit : http://www.cs.ucsb.edu/~nchohan/rrc/, http://www.ece.cmu.edu/~agr/projects/saga/

- 8

Previously addressed in:Pervasive/ Ubiquitous computing• Focus mostly on heterogeneous devices.

Figure credit : http://203crjameshowe.blogspot.com/2008/10/203cr-designing-for-usability-2-studio.html

Origin of Challenges in Internet of Things

• Heterogeneity • Diverse devices, many manufactures, different data types

• Large scale• Millions of devices , very large amount of data, millions

of users

9

- 10

Goal of Our Research

``Provide programming abstractions that allow domain experts in the IoT to specify applications involving rich interactions among a large number of heterogeneous devices’’.

First step : Important Initial Questions

1. What are the key concepts in the domain ?2. What are the associations (or relationships)

among identified concepts?

Answer : Domain Model (DM)[OldeldDM]

11

[OldeldDM] P. Oldeld. Domain Modelling. Technical report, Appropriate Process Group, 2002.

- 12

DM Benefits : Creates Common Understanding

• IoT terms - with different meaning by different people • leads to confusion and hinders scientific discourse

[Haller2010, IoTA]

• DM provides common lexicon • Useful to domain experts, programmers , researchers.

[Haller2010] S. Haller. The Things in the Internet of Things. IoT 2010 Poster Session. Tokyo, Japan, November, 2010. [IoTA] IoT-A Project: http://www.iot-a.eu

- 13

DM Benefits :Enables Modular Design

• DM clearly identifies capabilities of each concepts.• Application requirements tend to arrive in terms of

behaviour [OldeldDM]. They can be broken down into concepts.

Sense the temeprature of a room and keep it steady

Sensor Physical entityComputational and Actuation

DM Benefits : Models Invariant Properties

• Models invariant associations [IoTA, SENSEI] – not change from one application to other.

14

[IoTA] IoT-A Project: http://www.iot-a.eu/ [SENSEI] SENSEI project: http://www.sensei-project.eu/

Device: Smart Phone

Device: sunSPOT

Sensor: Light Sensor

Sensor: Temperature

Sensor

Hosts

Hosts

1 *

1 *

Application 1

Application 2

Invariant Association

- 16

Our approach of Domain Model Construction

App 3

App 6

App 1

App 8

App 10

App 2

App 5

App 9

App 11

App 4

App 7

App 12Applications

Classification of applications into behaviours

Behaviour1

Behaviour2

Behaviour3

c1 c2

c3 c4 c6

c5

c7

Our Domain Model

Identification of concepts

- 17

Behaviours in the Internet of Things

- 18

Data Collection

• Smart things interact with user by stating information about

themselves.

• Periodically

• on event

• on demand

• Found in• Patient monitoring [Niyato2009]

• Food supply chain [Dada2008]

• …[Niyato2009] D. Niyato, E. Hossain, and S. Camorlinga. Remote patient monitoring service using heterogeneous wireless access networks: architecture and optimization. Selected Areas in Communications, IEEE Journal on, 27(4):412-423, may 2009.[Dada2008] A. Dada and F. Thiesse. Sensor applications in the supply chain: The example of quality-based issuing of perishables. In Proceedings of the 1st international conference on The internet of things, pages 140-154. Springer-Verlag, 2008.

Example: Talking with Plants [Botanicalls]

[Botanicall] http://www.botanicalls.com/[PresAg] http://www.precisionagriculture.com.au/

19

• Plant notifies a user on his cell phone about whether it needs water.

• Moisture sensor, Botanicalls device, Botanicalls service, Mobile phone, Twitter service, User

• Also, precision agriculture [PresAg]

Sense - Compute - Actuate

• Smart things interact with each other at either in the local network or through the Internet.

• They take corrective actions [Mattern2010] with no human originator, recipient or intermediary.

• It may prompt users as required.• Found in

• Optimizing power consumption costs [Buckl2009]

• Work place safety [SENSEI]

• …[Mattern2010] F. Mattern and C. Floerkemeier. From the Internet of Computers to the Internet of Things. From active data management to event-based systems and more, Springer pages 242- 259, 2010.[Buckl209] C. Buckl, S. Sommer, A. Scholz, A. Knoll, A. Kemper, J. Heuer, and A. Schmitt. Services to the eld: An approach for resource constrained sensor/actor networks. In International Conference onAdvanced Information Networking and Applications Workshops, pages 476-481. IEEE, 2009.

20

- 21

Sense-Compute-Actuate Example: maintain temperature in room.[ParaHVAC]

[ParaHVAC] M. Feldmeier and J. Paradiso. Personalized HVAC control system. In Internet of Things (IOT), pages 1 -8, 29-dec., 2010.

User enters Preferences

Temperature node

Heater

- 22

Intermittent Sensing

• Centred around tag • RFID• (1D) Barcode • QR- code

• Mostly in applications where things have an information shadow[ValhoIoT] on the Internet

• Found in • Tourist information[RellTourist ] • Mobile ticketing[BroPerci]

• ...[ValhoIoT] C. A. Valhouli. The Internet of Things: Networked Objects and Smart Devices. Technical report, the hammersmithgroup, February 2011.[RellTourist D. Reilly, M. Welsman-Dinelle, C. Bate, and K. Inkpen. Just point and click?: using handhelds to interact with paper maps. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, pages 239-242. ACM, 2005.[BroPerci] G. Broll, E. Rukzio, M. Paolucci, M. Wagner, A. Schmidt, and H. Humann. Perci: Pervasive service interaction with the Internet of things. Internet Computing, IEEE, 13(6):74- 81, 2009.

- 23

Intermittent Sensing example: Reviewing consumer’s product [my2cents]

[my2cents] S. Karpischek and F. Michahelles. my2cents-Digitizing consumer opinions and comments about retail products. In Internet of Things (IOT), 2010, pages 1-7. IEEE, 2010.

1

2 3

User scans barcode through mobile device

When user wants to write comments, writes and sends to storage service

When user wants to read other users’ comments, scan barcode, queries to storage service.

Domain model of IoT

24

Basic IoT Interaction

30

Entity of Interest

User

real world objects.e.g. room, book, plant

Sensing

31

Entity of Interest Phenomenon

1

1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

Sensor measurement

generates

1

1..*

*

Consists-of1 1..*

1

User

Observable property of entity of interest e.g. temperature

Representation of sensor observation, without no explicit meaning. E.g. 25

Attach metadata to raw data.e.g. unit of measurement, time of sensing

Actuation

32

Entity of Interest Phenomenon

Action

affects

*

1..*

ActuatorPerfoms

1

Actuator driver

actuated-by

11

Command

consumes1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

1

1

Sensor measurement

generates1

1

1..*

*

Consists-of1 1..*

1

1

User

Translates actuation command and triggers the actuator.

instruction that describes a desired outcome.e.g. Switch ON heater

Sensing and Actuation

33

Entity of Interest Phenomenon

Action

affects

*

1..*

ActuatorPerfoms

1

Actuator driver

actuated-by

11

Command

consumes1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

1

1

Sensor measurement

generates1

1

1..*

*

Consists-of1 1..*

1

1

Information

Extends

Extends

Resource

Extends

Extends

User

It is a conceptual representation of a sensor or an actuator.

It is any data that is meaningfulby itself.

Role of Device as Resource Host

34

Entity of Interest Phenomenon

Action

affects

*

1..*

ActuatorPerfoms

1

Actuator driver

actuated-by

11

Command

consumes1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

1

1

Sensor measurement

generates1

1

1..*

*

Consists-of1 1..*

1

1

Information

Extends

Extends

Resource

Extends

Extends

User

DeviceHosts1

*

Provides the ability of communication.e.g. mobile phone, tag readers, personal computer

Role of Device as software host

36

Entity of Interest Phenomenon

Action

affects

*

1..*

ActuatorPerfoms

1

Actuator driver

actuated-by

11

Command

consumes1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

1

1

Sensor measurement

generates1

1

1..*

*

Consists-of1 1..*

1

1

Information

Extends

Extends

Resource

Extends

Extends

DeviceHosts1

*

User

Runs-on 1..*

1

Software Component

Software Components

37

Software Component

Encapsulate system’s functionalities ,Provides interface

End-user Application

Storage Service

Computational service

Driver

Extends

Extends ExtendsExtends

Sensor Driver

ActuatorDriverUser Store

Provides access to

Interacts with

1

1..*1

1

Communicates-with

1*

Information

consumes

generates

1 1..*

11

ExtendsExtends

Traditional Internet concepts

``Things’’- orientedconcepts

Our IoT Domain Model

38

Entity of Interest Phenomenon

Action

affects

*

1..*

ActuatorPerfoms

1

Actuator driver

actuated-by

11

Command

consumes1

Sensor

Observes

Raw dataProduces

1 1

Sensor driver

Accessed-by

1

1

Sensor measurement

generates1

1

1..*

*

Consists-of1 1..*

1

1

Information

Extends

Extends

Resource

Extends

Extends

DeviceHosts1 *

User

Software Component

Runs-on 1..*

1

End-user Application

Storage Service

Store

Computational Service Driver

Communicates-with

extends

Related Work on IoT Domain Model

• Haller [Haller2010] has defined the relationship among

things, devices, resources, and services.

• Aim: common understanding

• We add an explicit “user”, traditional internet concepts.

• A more detailed model is in [IoTA2011]

• Direct association between user and physical entity

• We add software components between them

39

[Haller2010] S. Haller. The Things in the Internet of Things. IoT 2010 Poster Session. Tokyo, Japan, November, 2010. [IoTA2011] www.iot-a.eu/

- 41

Concluding Remarks

• Developed a better understanding of IoT challenges

• Identified IoT application behaviours

• Captured the concepts (physical world and traditional

internet) and associations of IoT domain

• Domain Model benefits

• Captured terminology of IoT concepts.

• Modeled invariant properties of IoT domain

• Modular design

- 42

Future work

• Make the domain model more complete

• Expand the scope of the application survey

• Explore sub-class of ``communicate-with’’ association:

Publish/subscribe, procedure call, notification, etc.

• Explore effect of large scale

• Propose IoT Programming Abstractions

Merci, thank you, obrigado, शु�क्रि�या�, σας ευχαριστώ, grazie, 謝謝 , dank u, danke, gracias, شما از تشکر ありがとう ,با , நன்றி�, આભા�ર.

Thanks for listening me

• This work was supported in part by the European Commission FP7 CHOReOS and NESSOS projects, and the ANR Murphy project.

• The authors are grateful to the reviewers for their helpful comments.

email: pankesh.patel@inria.fr