example architectures leidenschaft technik internet of...

www.tugraz.at ◼

WISSEN

TECHNIK

LEIDENSCHAFTExample Architectures –

Internet of Things &

Kubernetes

Software Architecture VO/KU (706.706/706.707) – 2019-10-16

Heimo Gursch & Roman Kern – ISDS

www.tugraz.at ◼

Agenda

• Internet of Things (IoT)

• communication

• protocols

• student lab at the ISDS

• sensors platforms

• architecture

• virtualisation and containers

• Kubernetes



www.tugraz.at ◼

Internet of Things (IoT)

History and Background

• problem

• connect sensors and actuators to digital networks

• enable control by computer systems

• automate (industrial) processes

• general term: Supervisory Control and Data Acquisition (SCADA)

• yesterday’s solutions

• point-to-point links (e.g., RS-232)

• fieldbuses (e.g., Modbus, CAN, PROFIBUS)

• Many started as proprietary by a single company but are now standards



www.tugraz.at ◼


Gamechangers

• IP became the dominant communication protocol

• integrated computing power in sensors and actuators became powerful

enough to do IP-based communication

• Ethernet and variants of it became the dominate networking

technologies

• Industrial Ethernet

• WiFi

• Power over Ethernet (PoE)



www.tugraz.at ◼


The Gamechanger's Consequences

• use Ethernet (a variant) almost everywhere

• use IP-based communication almost everywhere

• with IP also TCP or UDP became the dominant

• Layer 1 to 4 in the ISO OSI model are covered by standard technologies

• every device can communicate with every other device

• remote communication via the Internet is possible

• relatively high data rates are achievable (if necessary)

• standardised and cheap network components can be used



www.tugraz.at ◼

Don’t take only my word for it…



source: Eclipse Foundation IoT developer survey 2019

www.tugraz.at ◼

How to transmit data on IoT?



source: Eclipse Foundation IoT developer survey 2019

www.tugraz.at ◼


HTTP – HyperText Transfer Protocol

• properties

• widely available: implementations in different languages

• easy to use: simple communication model

• almost never blocked and routed everywhere

• designed for something completely different: client-server hosting of

(static) content

• usage

• very common



www.tugraz.at ◼


WebSocket



• properties

• WebSocket is a full protocol on its own and not an add-on to HTTP

• BUT: WebSocket keeps combability to HTTP wherever possible

• offers full-duplex communication over single TCP connection

• Server can send messages to clients without prior requests by clients

once a connection has been established

• standardized by Internet Engineering Task Force (IETF)

• usage

• gaining popularity as we speak

www.tugraz.at ◼


MQTT – Message Queue Telemetry Transport

• properties

• simple on clients but requires central broker

• implementation in many languages exist

• standardised by the Organization for the Advancement of Structured Information Standards (OASIS)

• name is confusing, since there is no message queue in the protocol, its a historic reminiscence to the original creation at IBM

• usage

• version 3.1.1 from 2014 commonly used

• version 5 standardised in 2019



www.tugraz.at ◼


MQTT Publish and subscribe model



broker

sensoring

equipment

publish to topic

• publish-subscribe

communication

• sender publishes a

messages to a so called

topic

• every client subscribed to

the topic gets the

messages relayed by the

broker

www.tugraz.at ◼

What have we got in the ISDS student lab?

Sensor Platform – Tinkerforge

• complete sensor platform

• OpenSource hard- and software

• API to many programming languages

• offers embedded Linux system (RED Nod) and Raspberry PI integration

• MQTT enabled communication over Ethernet

• https://www.tinkerforge.com/

• Rolled out in six rooms collecting room climate data



https://www.tinkerforge.com/

www.tugraz.at ◼


Sensor Platform – WaspMote & Meshlium

• WaspMote nodes and Meshlium base station

• nodes are based on Arduino, e.g., no operating

system

• Meshlium runs Linux and forwards data to other

services, e.g, via MQTT

• http://www.libelium.com/



http://www.libelium.com/

www.tugraz.at ◼


Sensor Platform – Power Meter

• Proprietary solution

• Ethernet interface

• Measurements are parsed from the web

interface

• http://energenie.com/



http://energenie.com/

www.tugraz.at ◼

Serverserver01 (10.0.100.248)



tinkerforge-01 – 08

(10.0.100.220-227)egm-pwm-lan01 – 13

(10.0.100.200-212)

http://openweathermap.org

WiFi + BT Scanmeshlium01

(10.0.100.245)

MQTT Brokerserver01:1883

server01:5432

SQL to RESTTap

http://server01:8080

光Grafana

http://server01:8085

https://server01:8443

Performance

ZAMG TAWEShttp://www.zamg.ac.at/ogd/

Student Lab Software Architecture• Devices

• Hardware

• Software

• Docker

• Communication

• MQTT

• HTTP

• SQL

• Bash

www.tugraz.at ◼

Student Lab Software Architecture

Key Take-Away Messages

• there is no one-fits all solution

• network and communication protocols

• devices and operating systems

• requirement might differ on parts of the architecture

• real time constraints

• reliability

• IoT, microservices and containers go well with each other

• independent modules in hardware or hard- and software

• connected by defined network interfaces



www.tugraz.at ◼

Virtualisation



host OS + bins / libs

hardware

applic

ation 1

applic

ation 3

applic

ation 2

host OS

hardware

hypervisor

gu

est

OS

+

bin

s / lib

s

guest

OS

+

bin

s / lib

s

guest

OS

+

bin

s / lib

s

applic

ation 1

applic

ation 3

applic

ation 2

host OS

hardware

container hypervisor

bin

s / lib

sapplic

ation 1

applic

ation 3

ap

plic

ation

2

host OS

hardware

container hypervisor

ap

plic

atio

n 1

+

bin

s / lib

s

applic

ation 3

+

bin

s / lib

s

applic

ation 2

+

bin

s / lib

s

bin

s / lib

s

bin

s / lib

s

no virtualisation virtual machines container

virtualisation

unikernel

www.tugraz.at ◼

Virtualisation

When There is no Virtualisation

• no virtualisation overhead

• separation of applications is done by the OS

• allocation of resources are based on users and groups

• resources are files, CPU time, memory, network bandwidth

• binaries and libraries are shared via the OS

• package management

• combine common binaries and libraries in packages

• problems occurs with version mismatch:

• application 1 needs version x of package A

• application 2 needs version y of package A



www.tugraz.at ◼

Virtualisation

Virtual Machines

• hypervisor re-creates the behaviour of hardware, i.e., hardware level virtualisation

• each virtual machine is completely separated from each other

• resources are allocated by the hypervisor to the virtual machines

• inside the virtual machines a full operating system is running

• the OS in the virtual machine runs the application with the required binaries and libraries

• most virtualisation overhead of all solutions

• Hypervisors

• VirtualBox, VM Ware Fusion / Player / Workstation / Server, Proxmox VE, KVM, Microsoft Hyper-V, Parallels Workstation, Xen, …



www.tugraz.at ◼

Virtualisation

Container Virtualisation

• hypervisor re-creates the behaviour of OS, i.e., OS virtualisation

• CPU & memory management

• filesystem

• network

• container engine might offer additional separation over normal OS

• user separations

• devices

• inside the containers on a minimal OS with kernel, required binaries and libraries to run the application is available

• considerable less virtualisation overhead than full virtualisation

• most common product: Docker

• alternatives: Linux Containers (LXC), Solaris Containers, FreeBSD jail, …



www.tugraz.at ◼

Virtualisation

Unikernels Virtualisation

• objective: achieve same separation like containers but decrease the

virtualisation overhead

• solution: include library operating systems directly into the application so

that it can run on a hypervisor directly

• lowest possible virtualisation overhead

• separation of running containers still done by the hypervisor

• examples: MirageOS, ClickOS, UniK, Rumprun



www.tugraz.at ◼

Docker Containers

How to Run and Manage Containers

• directly, i.e. Docker CLI: too cumbersome if you’ve got hundreds running

• Docker Compose: can only manage singe hosts set-ups

• Docker Swarm: hardly used in the industry

• Kubernetes:

• currently the tool of choice in the industry

• handles deployment, scaling, and management

• designed to handle multi-host setups, i.e., clusters

• https://kubernetes.io/



https://kubernetes.io/

www.tugraz.at ◼

Kubernetes Architecture



www.tugraz.at ◼

Kubernetes Terminology

• nodes: hosts or hardware

• pod: one or more containers running on the same node

• pods get IP address and volumes assigned

• service: set of pots creating an application

• ConfigMaps and Secrets:

• configuration input to the instances running of the containers

• can be anything from individual properties up to JSON / XML files

• available as environment variables or on the container’s file system



www.tugraz.at ◼


Master

• master / slave architecture

• distributed architecture

• ready for automated commission, running and decommission of services

• master controllers the cluster

• master offers API to control the complete setup either by other services (e.g., built automatisation) or by users / developers

• master runs the controller manager service

• more or less the central core or kernel of a Kubernetes Architecture

• scheduler: knows witch nodes are available and what in running where



www.tugraz.at ◼


Nodes

• nodes are the slaves in the architecture

• Kube-Proxy handling the network Communication with other nodes

• Kubelet does what ever the master tells it on the node

• cAdvisor: monitoring

• Pods: run the actual software that proving the service to the users



www.tugraz.at ◼

Virtualisation and Kubernetes

Key Take-Away Messages

• virtualisation in any of its forms is now common practice in running software systems

• containers offer a good compromise between overhead and separation

• Kubernetes is currently to tool of choice to manage large distributed container-based software systems

• Kubernetes is designed to manage large and distributed setups

• reason for its inherent complexity

• only pays of in distributed setups

• Kubernetes are provided by may cloud provides like Amazon AWS or Microsoft Azure



www.tugraz.at ◼

WISSEN

TECHNIK

LEIDENSCHAFTExample Architectures –

Internet of Things &

Kubernetes



example architectures leidenschaft technik internet of...

Documents