architecting internet of things systems with prismtech vortex

Angelo Corsaro, PhDChief Technology Officer

[email protected]

Architecting IoT Systems

mailto:[email protected]

Cop

yrig

ht P

rism

Tech

, 201

4

Internet of Things Flavours

Wikipedia: Interconnection of

uniquely identifiable embedded

computing-like devices within the

existing Internet infrastructure

Internet of Things (IoT) is the term used to describe any kind of application that connected and made “things” interact through the internet

It is now clear that there are at least two kinds of IoT, Consumer IoT (CIoT) and Industrial IoT (IIoT)

The CIoT and IIoT follow the [Collect | Store | Analyse | Share] architecture, yet they have some key differences that is important to understand

https://twitter.com/acorsaro










http://www.linkedin.com/in/corsaro










http://slideshare.net/angelo.corsaro









Cop

yrig

ht P

rism

Tech

, 201

4

The Consumer Internet of Things (CIoT) represents the class of consumer-oriented applications where:

Devices are consumer devices, such as smart appliances, e.g. refrigerator, washer, dryer, personal gadgets such as, fitness sensors, google glasses, etc.

Data volumes and rates are relatively low

Applications are not mission or safety critical, e.g., the failure of fitness gadget will make you, at worse, upset, but won’t cause any harm

CIoT applications tend to be “consumer centric”

Consumer Internet of Things (CIoT)






























Cop

yrig

ht P

rism

Tech

, 201

4

The Industrial Internet of Things (IIoT) represents industry-oriented applications where:

Devices are machines operating in industrial, transportation, energy or medical environment1

Data volumes and rates tend to be from sustained to relatively high

Applications are mission and or safety critical, e.g. the failure of a smart grid has severe impact on our life and economy, the misbehaving of a smart traffic system can threaten drivers

IIoT applications tend to be “system centric”

Industrial Internet of Things (IIoT)

1 The list of application domains is supposed to give examples and is not exhaustive






























Cop

yrig

ht P

rism

Tech

, 201

4

Efficient and scalable Data Sharing is a key requirement of practically any IoT system

The degree of performance and fault-tolerance required by the data sharing platform varies across Consumer and Industrial Internet on Things applications

Fog Computing support is key for IIoT

CIoT/IIoT Data Sharing RequirementsHigh Individual Data Rates

High Aggregated Data Volumes

Low Latency

Temporal Determinism

Device-2-Device (D2D) Comms

Device-2-Cloud (D2C) Comms

Cloud-2-Cloud (C2C) Comms

Bandwidth Efficiency

Fault-Tolerance

Transport-Level Security

Data-Level Security

0,00 0,25 0,50 0,75 1,00

CIoT IIoT

[Ref: A Comparative Study of Data-Sharing Standards for the Internet of Things, Cutter Journal, Dec 2014 ]

Relative Importance






























Cop

yrig

ht P

rism

Tech

, 201

4

Connectivity to the Cloud is a pre-requisite of cloud computing.

- Some IoT systems need to be able to work even when connection is temporarily unavailable or under degraded connection

Cloud computing assumes that there is enough bandwidth to collect the data

- That can become an overly strong assumptions for Industrial Internet of Things applications

Cloud computing centralises the analytics thus defining the lower bound reaction time of the system

- Some IoT applications won’t be able to wait for the data to get to the cloud, be analysed and for insights to get back

Cloud Computing Limitations

[source: http://on.wsj.com/1saV1xU]






























http://on.wsj.com/1saV1xU%5D

Cop

yrig

ht P

rism

Tech

, 201

4

Analytics, traditionally computed on a cloud, are increasingly moving on the edge to improve scalability and responsiveness

The Data Pipeline is executed at different scales,

Cloud and Fog Computing

Fog Computing

Cloud Computing

Fog Computing

Fog Computing

Device-to-Cloud Communication

Device-to-Device Communication

Fog-to-Cloud Communication

Cloud-to-Cloud Communication


Collect | Store | Analyse | Share


































VORTEX The IoT Data Sharing Platform

Cop

yrig

ht P

rism

Tech

, 201

4

VORTEX is a ubiquitous data sharing platform for the Internet of Things providing scalable end-to-end seamless, efficient, secure and timely data sharing for IoT supporting device, edge, gateways and cloud

VORTEX platform coverage, performance and scalability make it the only viable choice from consumer to demanding business critical, industrial, real time, IoT applications

VORTEX simplifies IT/OT integration, and is the only platform that holistically addresses IT and OT requirements

Introducing Vortex

Sensors/ Things/ Devices

Connectivity/ Transport

Edge Management/ Telemetry

Application Platform

Enterprise Systems

Ubiquitous Data Sharing peer-to-peer, device-to-cloud, cloud-to-cloud

Cop

yrig

ht P

rism

Tech

, 201

4

Specialised device implementations optimally addressing requirements of OT and IT platforms

VORTEX can readily deal with data ingestion seamlessly integrating with other protocols, e.g. MQTT, CoAP, etc.

VORTEX leverages the DDS standard for interoperability and uniquely extends it with support for Internet Scale systems, mobility and Web 2.0 applications

The VORTEX Platform

PaaS/MaaS

Cop

yrig

ht P

rism

Tech

, 201

4

VORTEX supports both the Cloud and the Fog Computing Paradigm

VORTEX natively supports:

- Device-to-Device Communication

- Device-to-Cloud Communication

Cloud, Fog and Edge Computing

Fog Computing

Cloud Computing

Fog Computing

Fog Computing





































Cop

yrig

ht P

rism

Tech

, 201

4

Abstracting Connectivity

Fog Computing

Cloud Computing

Fog Computing

Fog Computing



































Cop

yrig

ht P

rism

Tech

, 201

4

Vortex provides a Distributed Data Space abstraction where applications can autonomously and asynchronously read and write data

Its built-in dynamic discovery isolates applications from network topology and connectivity details

DDS’ Data Space is completely decentralised

High Level Abstraction

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS






























Cop

yrig

ht P

rism

Tech

, 201

4

DDS supports the definition of Common Information Models.

These data models allow to naturally represent physical and virtual entities characterising the application domain

DDS types are extensible and evolvable, thus allowing incremental updates and upgrades

Data Centricity






























Cop

yrig

ht P

rism

Tech

, 201

4

A Topic defines a domain-wide information’s classA Topic is defined by means of a (name, type, qos) tuple, where

• name: identifies the topic within the domain

• type: is the programming language type associated with the topic. Types are extensible and evolvable

• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.

Topic

TopicTypeName

QoS

struct TemperatureSensor { @key long sid; float temp; float hum;}






























Cop

yrig

ht P

rism

Tech

, 201

4Vortex “knows” about application data types and uses this information provide type-safety and content-based routing

Content Awarenessstruct TemperatureSensor { @key long sid; float temp; float hum;}

sid temp hum101 25.3 0.6507 33.2 0.7913 27,5 0.55

1307 26.2 0.67

“temp > 25 OR hum >= 0.6”

sid temp hum101 25.3 0.6507 33.2 0.7

1307 26.2 0.67

Type

TempSensor






























Cop

yrig

ht P

rism

Tech

, 201

4

For data to flow from a DataWriter (DW) to one or many DataReader (DR) a few conditions have to apply:

The DR and DW domain participants have to be in the same domain

The partition expression of the DR’s Subscriber and the DW’s Publisher should match (in terms of regular expression match)

The QoS Policies offered by the DW should exceed or match those requested by the DR

Quality of ServiceDomain

Participant

DURABILITY

OWENERSHIP

DEADLINE

LATENCY BUDGET

LIVELINESS

RELIABILITY

DEST. ORDER

Publisher

DataWriter

PARTITION

DataReader

Subscriber

DomainParticipant

offered QoS

Topicwrites reads

Domain Idjoins joins

produces-in consumes-from

RxO QoS Policies

requested QoS






























Your First Vortex App!

Cop

yrig

ht P

rism

Tech

, 201

4

Writing Data in Scalaimport dds._import dds.prelude._import dds.config.DefaultEntities._

object SmartMeter { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dw = DataWriter[Meter](topic) while (!done) { val meter = readMeter() dw.write(meter) Thread.sleep(SAMPLING_PERIOD) } }}

enum UtilityKind { ELECTRICITY, GAS, WATER

}; struct Meter {

string sn; UtilityKind utility; float reading; float error;

}; #pragma keylist Meter sn






























Cop

yrig

ht P

rism

Tech

, 201

4

Reading Data in Scalaimport dds._import dds.prelude._import dds.config.DefaultEntities._

object ChatLog { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dr = DataReader[Meter](topic) dr listen { case DataAvailable(_) => dr.read.foreach(println) } }}


}; struct Meter {
































Cop

yrig

ht P

rism

Tech

, 201

4

Writing Data in C++#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Meter> topic(“SmartMeter”); Publisher pub(dp); DataWriter<Meter> dw(pub, topic);

while (!done) { auto value = readMeter() dw.write(value); std::this_thread::sleep_for(SAMPLING_PERIOD); }

return 0; }


}; struct Meter {
































Cop

yrig

ht P

rism

Tech

, 201

4

Reading Data in C++#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Meter> topic(”SmartMeter”); Subscriber sub(dp); DataReader<Meter> dr(dp, topic);

LambdaDataReaderListener<DataReader<Meter>> lst; lst.data_available = [](DataReader<Meter>& dr) { auto samples = data.read(); std::for_each(samples.begin(), samples.end(), [](Sample<Meter>& sample) { std::cout << sample.data() << std::endl; } } dr.listener(lst); // Print incoming data up to when the user does a Ctrl-C std::this_thread::join(); return 0; }


}; struct Meter {
































Deployment Models

Cop

yrig

ht P

rism

Tech

, 201

4

Fog + Cloud

Fog Computing Fog Computing

Fog Computing


Peer-to-Peer (Brokerless)


Device communicate peer-to-peer within a fog-domain and through Cloud across fog-domains

Some device concurrently communicate with peers and the cloud






























Cop

yrig

ht P

rism

Tech

, 201

4

Fog + Cloud-Link + CloudDevice communicate peer-to-peer within a fog-domain

A Cloud-Link controls which data is exchanged with the could


Fog Computing




Cloud-LinkCloud-Link






























Cop

yrig

ht P

rism

Tech

, 201

4

Federated Fog


Fog Computing



Cloud-Link

Cloud-Link

Fog domain are federated by Cloud-Link instances

A Cloud-Link controls which data is exchanged with the could






























Architectures

Cop

yrig

ht P

rism

Tech

, 201

4

device-to-cloud

device-to-cloud

device-to-cloud

device-to-cloud

device-to-device

device-to-device

device-to-device

device-to-device

Query

Storage

Compute

Analytics

VORTEX Enables

Cop

yrig

ht P

rism

Tech

, 201

4

device-to-cloud

device-to-cloud

device-to-cloud

device-to-cloud

device-to-device

device-to-device

device-to-device

Query

Storage

Compute

Analytics

VORTEX Enables

Cop

yrig

ht P

rism

Tech

, 201

4

device-to-cloud

device-to-cloud

device-to-cloud

device-to-cloud

device-to-device

device-to-device

device-to-device

device-to-device

Query

Storage

Compute

Analytics

VORTEX Enables

Performance

Cop

yrig

ht P

rism

Tech

, 201

4

Industry leading performance and scalability

VORTEX DEVICE

• Peer-to-Peer Intra-core latency as low as 8 µs

• Peer-to-Peer latency as low as 30 µs

• Point-to-Point throughput well over 2.5M msg/sec

VORTEX CLOUD

• Routing latency as low as 4 µs

• Linear scale out

• 44K* msgs/sec with a single router, 4x times more the average Tweets per second in the world (~6000 tweets/sec)!

*2048 bytes message payload

VORTEX Security

Cop

yrig

ht P

rism

Tech

, 201

4

Boundary Security


Fog Computing





TLS

TLS Boundary security support is enabled by Cloud-Link

Cloud-Link separates security concerns at different scales and also allows to control what information to expose






























Cop

yrig

ht P

rism

Tech

, 201

4

The DDS Security defines a framework for interoperable security with pluggable:

- Authentication

- Access Control

- Crypto

- Logging

- Tagging

DDS Security






























Cop

yrig

ht P

rism

Tech

, 201

4

Vortex & DDS Security


Fog Computing


• Mutual Authentication • Secure Multicast • Encryption options:

• User Data • User Data + Protocol

Header • Message Authentication

• Nothing • Hash-based MAC


S-DDSI

DDS Security makes it possible for Vortex to:

Securely interoperate with other DDS implementation

Plug user preferred authentication, access, control, and crypto plugin

Provide support for Symmetric and Asymmetric authentication as well as secure multicast

Separate security domains through Vortex Cloud-Link

S-DDSI






























Cop

yrig

ht P

rism

Tech

, 201

4


Fog Computing





Edge Analytics Edge Analytics

Cloud Analytics

Edge Analytics

Different Networks Different Partitions too.

Cloud Deployed on Linode UK






























Cop

yrig

ht P

rism

Tech

, 201

4

Specialised device implementations optimally addressing requirements of OT and IT platforms

VORTEX can readily deal with data ingestion seamlessly integrating with other protocols, e.g. MQTT, CoAP, etc.

VORTEX leverages the DDS standard for interoperability and uniquely extends it with support for Internet Scale systems, mobility and Web 2.0 applications

The VORTEX Platform

PaaS/MaaS

Cop

yrig

ht P

rism

Tech

, 201

4

architecting internet of things systems with prismtech vortex

Software