Center for Ubiquitous Computing, University of Oulu, Finland

Teemu Leppänen, Iván Sánchez Milara, Jilin Yang, Joonas Kataja and Jukka Riekki

Enabling User-centered Interactionsin the Internet of Things

Leppänen, T., Sanchez Milara, I., Yang, J., Kataja, J. and Riekki, J. “Enabling User-centered Interactions in the Internet of Things”, In: IEEE SMC 2016, pp. 1537-1543, October 9-12, Budapest, Hungary, 2016.

Cloud infrastructure

Current IoT architectures

2 SMC 2016

3 SMC 2016

What is the role of humans in IoT ???

Just users of cloud-based services???

Just data observers???

Cloud infrastructure

Current IoT architectures

4 SMC 2016

Humans in IoT

5 SMC 2016

Humans as communication

nodes

Humans as processing nodes

Humans as actuators

D. Nunes, P. Zhang, and J. Silva, “A survey on Human-in-the-Loop applications towards an Internet of All”, IEEE Commun. Surv. Tuts., vol. 17, no. 2, pp. 944-965, February 2015.

Icons designed by Madebyoliver from Flaticon

6 SMC 2016

What should the role of humans in IoT

???

CC Attribution license. Postscapes and Harbor Research. http://www.postscapes.com/what-exactly-is-the-internet-of-things-infographic/

Interactions in IoT

7 SMC 2016

Thing to Thing Human to Thing Human to Human

A. M. Ortiz, D. Hussein, S. Park, S. N. Han, and N. Crespi, “The Cluster Between Internet of Things and Social Networks: Review and Research Challenges,” IEEE Internet Things J., vol. 1, no. 3, pp. 206–215, Jun. 2014.

8 SMC 2016

How can we build those interactions ?

Mobile Agents for IoT

9 SMC 2016

IoT SOFTWARE FRAMEWORK Mobile agents operate as multi-agent system

System components modeled as resources and accessed with Web protocols: HTTP & CoAP, providing interoperability

T. Leppänen, M. Liu, E. Harjula, A. Ramalingam, J. Ylioja, P. Närhi, J. Riekki, and T. Ojala, “Mobile Agents for Integration of Internet of Things and Wireless Sensor Networks,” in 2013 IEEE SMC2013, pp. 14–21.

– Mobile agents are software agents with mobility: migrate between computers in networked systems

– Agents are autonomous, intelligent and social– A distibuted application design and implementation paradigm

NFC

10 SMC 2016

NEAR FIELD COMMUNICATION– Short range wireless technology (~5cm)– Data stored permanently in the tag– Readers embedded in mobile phones

Reader/Writer P2P Card Emulation

User interactions in IoT

11 SMC 2016

Traditional approach• Central coordination usually in

cloud platform• To execute a task it might

involve multiple actions, all the load for the user.

• Human-to-human interaction not considered

• Conflicts between users solved by themselves with no information about current state of system or external components

Mobile agents approach

• Decentralized, autonomously operated by mobile agents

• Agent performs actions to execute user-defined tasks

• Mobile agents can assist users to take decisions.

• Agents migrate based on human social relationships

Mobile agents interactions (I)

12 SMC 2016

Thing to ThingMobile agent

Mobile agents interactions (II)

13 SMC 2016

Human to Thing

Mobile agent

Mobile agent

Mobile agent

Mobile agent

Mobile agents interactions (III)

14 SMC 2016

Human to Human Mobile agent

Mobile agent

Mobile agent

Mobile agent

Mobile agent

System architecture

15 SMC 2016

Migration throughCOAP message

Web Service

Migration throughHTTP request

Migration through NFC (social interaction)

Agent requests data / actuates component

DeviceSensor/ Actuator

DeviceSensor/ Actuator

Migration throughNFC

Mobile agent architecture

16 SMC 2016

• Task code is application-specific• Task resources accessed through

URLS with HTTP methods• State exposes (selected) task

results in the system

• HW, OS and programming language independent

• Current implementations for embedded systems (binary code) and smartphones (Python, JavaScript)

Mobile agent encapsulated in a COAP message:• Enables standardized communication

for embedded networked systems• Integrates disparate systems• Minimal protocol overhead• Reduces the resources needed for

protocol translation

Mobile agent encapsulation

17 SMC 2016

Resources addresses + utility data

Task code (precompiled to node native language)

COAP Message

COAP Message embedded in one or several NDEF records

Summary

18 SMC 2016

• Humans are part of the IoT– It is necessary consider thing-to-thing, human-to-thing, human-

to-human interactions– Move away from classical person-as-observer paradigm

• We present an homogenous interaction method for the IoT– Best suited for scenarios where users should be in control– Technology enablers: Mobile agents and NFC

• We implement a real world prototype to verify the feasibility of the system