d2.2 iot work final-v1.0 1 - pdfs.semanticscholar.org · omg dds the omg data-distribution service...

01/06/2011 D2.2 –Bootstrapping Architecture IoT@Work/WP2/D2.2/ of 84

Category: Report Status: Final Availability: Public

IoT@Work WP 2 – COMMUNICATION NETWORKS

D2.2 –BOOTSTRAPPING ARCHITECTURE

Reference: IoT@Work/WP2/D2.2

Category: Report

Deliverable Responsible: SIEMENS

Author(s): A. M. Houyou, H.-P. Huth, G. Völksen, S. Fries,

K. Fischer, J. Gessner, A. Schattleitner (Siemens)

J. Claessens (EMIC)

M. Ippolito (CRF)

D. Rotondi, R. Tarli, G. Di Vito, A. Guarneri (TXT e-solutions)

C. Kloukinas (City)

J. Jasperneite, H. Trsek (inIT)

Due Date: 30/04/2011

Deliverable Date: 01/06/2011

Project Start Date: 01/06/2010

Project Duration: 36 Months

Status: Final

Availability: Public



Executive Summary

This deliverable represents a snapshot in the IoT@Work architecture design process. It defines the way application and scenario requirements are transformed into system requirements, while focusing on only one task of the whole system architecture, namely the bootstrapping task. This deliverable designs the functionalities that will enable a Plug&Work bootstrapping. The bottom-up approach explained throughout the text explores possible implementations and extensions to state of the art networking technologies mostly. The deliverable stops right before specifying the final components of the architecture and their dependencies, since the next task that needs to be addressed is focused on an autonomous resource management. The options and decisions made in this work and their basis will also be evaluated further in future work based on the defined system requirements and proposed architecture principles.

Document History

Version History

Version Status Date Editor(s)

0.1 Draft March 2011 A. Houyou (Siemens)

0.2 Draft March 2011 A. Houyou (Siemens)

0.3 Draft April 2011 H.-P. Huth, A. Houyou (Siemens)

0.4 Draft April 2011 H.-P. Huth (Siemens), D. Retondo (TXT)

0.5 Draft April 2011 A. Houyou (Siemens), H. Trsek (inIT)

0.6 Draft Mai 2011 H.-P. Huth (Siemens), H. Trsek (inIT)

0.7 Draft Mai 2011 A. Houyou (Siemens), D. Retondo (TXT)

0.8 Draft Mai 2011 A. Houyou (Siemens), H. Trsek (inIT)

0.9 Draft June 2011 A. Houyou (Siemens)

1.0 Final June 2011 A. Houyou (Siemens)



Summary of Changes

Version Section(s) Synopsis of Change

0.1 Initial ToC Document Structure

0.2 All Detailed Document structure

0.3 Chapters 2, 3, 5 Architecture Views, Requirements

0.4 Chapters 2, 4 SotA Assumptions, Scenarios

0.5 Chapters 3, 5 Requirements, Functionalities

0.6 Chapters 2, 6 Application and Full System View

0.7 Chapter 1, 4 Introduction, Final Assumptions

0.8 Chapter 5, 6 Directory Role, System View

0.9 Chapter 1, 6, 7 Final Version

1.0 All Reviewed Final Version



Contents

1 Introduction............................................................................................11

1.1 WP2 Networks and Communication Services............................................. 11

1.2 Task 2.1 Definition of Advanced Communication Services ......................... 12

1.3 Task 2.3 Techniques for Realisation of Advanced Communication Services – Network Bootstrapping......................................................................................... 12

1.4 Scope of this Deliverable............................................................................ 13 1.4.1 Chosen Methodology and Deliverable Structure ............................................... 13 1.4.2 Basic Assumptions............................................................................................. 14

2 Architecture Overview and Approach..................................................15

2.1 Overview .................................................................................................... 15

2.2 Phases in the Bootstrapping process ......................................................... 17 2.2.1 Offline Configuration .......................................................................................... 17 2.2.2 Power On ........................................................................................................... 19 2.2.3 Basic Connectivity.............................................................................................. 19 2.2.4 Middleware and Subsystem internal Helper Services ....................................... 20 2.2.5 Applications........................................................................................................ 25

2.3 View Points of the Architecture................................................................... 26 2.3.1 View Points ........................................................................................................ 26 2.3.2 Layer View ......................................................................................................... 27 2.3.3 Functional View.................................................................................................. 28 2.3.4 IoT Resource View............................................................................................. 29 2.3.5 Slice View .......................................................................................................... 31

3 Revised System Requirements – Network View .................................32

3.1 Network View on Requirement Analysis ..................................................... 32

3.2 Advanced Communication Services and Industrial Automation Needs - QoS Requirements in the Factory ................................................................................ 38

3.3 Naming and Addressing Needs of Automation ........................................... 41 3.3.1 Directory entries and data-base management .................................................. 41 3.3.2 Dealing with Address Changes.......................................................................... 41 3.3.3 Remote Access.................................................................................................. 41 3.3.4 Cross-Checking Device Identity......................................................................... 42

3.4 Types of Device IDs ................................................................................... 42

4 System Bootstrapping Procedures and Assumptions .......................44

4.1 Plug&Work Scenario Boards ...................................................................... 44 4.1.1 Scenario 1: Software Update with minimal Impact on Production..................... 44 4.1.2 Scenario 2: Bootstrap Energy Usage Assessment Devices and Application .... 45 4.1.3 Scenario 3: Plugging in/out factory modules ..................................................... 46

4.2 Assumed Scenario Board Description ........................................................ 46 4.2.1 Initial Bootstrapping Scenario ............................................................................ 47 4.2.2 Current Drawbacks ............................................................................................ 48 4.2.3 IoT@Work Future Scenario ............................................................................... 48 4.2.4 Event Collection and Notification Service .......................................................... 49 4.2.5 Energy Management Service Bootstrapping ..................................................... 51

4.3 Engineered application process & application programming....................... 51

4.4 Network Resource Configuration and Planning .......................................... 53



4.5 Pre-requisites of Running Network Services............................................... 54 4.5.1 Overview ............................................................................................................ 54 4.5.2 Basic Connectivity Boot Sequences .................................................................. 55 4.5.3 Ethernet Start Up ............................................................................................... 56 4.5.4 IP Layer Start-Up ............................................................................................... 57 4.5.5 Network Supported Remote Boot ...................................................................... 58 4.5.6 Network Basic Services ..................................................................................... 58

5 Device Bootstrapping – Process Description .....................................60

5.1 Bootstrapping Functional Components....................................................... 60 5.1.1 Device Directories.............................................................................................. 61 5.1.2 Naming and Addressing Components ............................................................... 64 5.1.3 Device Security and Context Functional Components ...................................... 65

5.1.3.1 Authentication Function ................................................................. 66

5.1.3.2 Location and Context Function ...................................................... 66

5.1.3.3 Integrity and Authentication of data................................................ 66

5.1.3.4 Data Confidentiality ....................................................................... 67

5.1.3.5 Secure (Device) Inventory ............................................................. 67

5.1.3.6 Pre connect Network Access Control (NAC).................................. 67

5.2 Plug&Work Device Bootstrapping Process Description .............................. 67 5.2.1 PHASE I: Establishing Basic Connectivity......................................................... 68

5.3 Bootstrapping Higher Layers ...................................................................... 70 5.3.1 Phase II: Pairing "Engineered ID" vs. "Self-Configured ID”".............................. 70 5.3.2 Phase III: Instantiating Pre-Engineered Application Details .............................. 70

6 Configuration Engineering and Bootstrapping ...................................72

6.1 Factory Automation Applications ................................................................ 72

6.2 How to Engineer and Trigger Bootstrapping ............................................... 73 6.2.1 Main Concepts of Model-based Development................................................... 74

6.2.1.1 Platform Independent Model.......................................................... 75

6.2.1.2 Platform Specific Model ................................................................. 76

6.3 Future steps towards a plug and work bootstrapping in a modular manufacturing system.......................................................................................... 77

6.3.1 A realistic case study: the Lemgoer Model Fabric............................................. 77 6.3.2 Future implementations and extensions ............................................................ 78

7 Conclusions and Next Steps.................................................................80

8 References..............................................................................................82



List of Figures

01/06/2011 D2.2 – Bootstrapping Architecture IoT@Work/WP2/D2.2/ of 84


Figure 2.1: Bootstrapping Phases .......................................................................... 16

Figure 2.2: Bootstrapping Supporting services ........................................................ 22

Figure 2.3: Top level log and audit namespace structure ....................................... 23

Figure 2.4: Bottom level log and audit namespace structure ................................... 24

Figure 2.5: Architecture – IoT Resource View ......................................................... 29

Figure 2.6: Architecture – Bootstrapping ................................................................. 31

Figure 2.7: Slice and Layer View............................................................................ 31

Figure 3.1: The automation pyramid and corresponding requirements .................... 38

Figure 3.2: Network Requirements of Factory Automation Applications .................. 40

Figure 4.1: Current usual Energy Monitoring configuration ..................................... 47

Figure 4.2: Siemens SENTRON configuration (source Siemens SENTRON Powermanager brochure)........................................................................................ 48

Figure 4.3: ENS based Energy Monitoring configuration ....................................... 49

Figure 4.4: Energy Monitoring Hierarchical namespace ........................................ 50

Figure 5.1 The First Functional Components for a Plug&Work Architecture ............ 61

Figure 5.2: The Device Directory Actors .................................................................. 63

Figure 5.3: The Device Directory Reference Model ................................................. 64

Figure 5.4: Sequence Chart of Bootstrapping a single Device ................................. 68

Figure 6.1: Platform independent model for an automation system ......................... 75

Figure 6.2: Platform specific model for an automation system................................. 76

Figure 6.3: Real setup of the Lemgoer Model Fabric (LMF)..................................... 77

Figure 6.4: Agile manufacturing scenario ................................................................ 78

List of Tables

Table 3.1 – QoS requirements in control and field level communication.................. 39

Table 4.1 Basic Connectivity Boot Steps ................................................................ 55

Table 7.1 – Follow-up Activities Based on This Deliverable..................................... 81



List of Acronyms

ARP Address Resolution Protocol

Is a protocol used in local area networks to translate IP addresses to Ethernet addresses

BIOS Basic Input/Output System

BOOTP Bootstrap Protocol

is a network protocol used by clients to obtain an IP address from a configuration server. The DHCP is a more advanced protocol that has superseded the use of BOOTP

CA Certification Authority

DCS Distributed Control System

Is a process control system using a network infrastructure to communicate with a set of sensors, controllers, operator terminals and actuators

DD Device Directory

DHCP Dynamic Host Configuration Protocol

is a protocol used on IP networks for configuring network devices. There are specific versions of DHCP for IPv4 and IPv6

EB Event Based

EDD Electronic Device Description

ENS Event Collection and Notification Service

ERP Enterprise Resource Planning

GPRS General Packet Radio Service

GSDML General Station Description Markup Language

HMI Human-Machine-Interface

HW Hardware

ICMP Internet Control Message Protocol

IEC International Electrotechnical Commission

IEEE Institute of Electrical and Electronics Engineers

The Institute of Electrical and Electronics Engineers is an international non-profit, professional organization for the advancement of technology related to electricity and electronics.

IETF Internet Engineering Task Force

It is an open standards organization, based on volunteers, that develops and promotes Internet standards, in particular the ones related to the TCP/IP and Internet protocol suite.

The IETF is formally a part of the Internet Society and is overseen by the Internet Architecture Board (IAB)

IoT Internet of Things

IP Internet Protocol



LDAP Lightweight Directory Access Protocol

LLDP Link Layer Discovery Protocol

MAC Media Access Control

MDA Model Driven Architecture

MES Manufacturing Execution System

MW Middleware

NAC Network Access Control

OMG Object Management Group

OMG DDS The OMG Data-Distribution Service for Real-Time Systems (DDS) is an international middleware standard for real-time and embedded systems.

OO Object Oriented

OOMW Object Oriented Middleware Architecture

OPC-UA OPC-UA

A new set of specifications that are not based on Microsoft COM that will provide standards based cross-platform capability. It is managed by the OPC Foundation.

ORB Object Request Broker

in Common Object Request Broker Architecture (CORBA ORBs manage the transformation (called marshalling/serialization and un-marshalling/de-serialization) of in-process data structures to and from the byte sequence transmitted over the network

OSI Open Systems Interconnection

PDP Policy Decision Point

PLC Programmable Logic Controller

A programmable microprocessor-based device used in discrete manufacturing to control assembly lines, machinery or other types of mechanical, electrical and electronic equipment on the shop floor

PROFINET PROFINET

is an open industrial Ethernet standard for automation. PROFINET uses TCP/IP and IT standards and supports real-time Ethernet communications

QoS Quality of Service

RADIUS Remote Authentication Dial In User Service

SNMP Simple Network Management Protokol

SSID Service Set Identity, an ID for IEEE 802.11 WLAN stations

SSH Secure Shell

SSL Secure Socket Layer

STP Spanning Tree Protocol

TCP Transmission Control Protocol

TPM Trusted Platform Module

UDP User Datagram Protocol

UID Unique Identifier



URI Uniform Resource Identifier

VLAN Virtual Local Area Network

WLAN Wireless Local Area Network

XML Extensible Markup Language



1 Introduction

1.1 WP2 Networks and Communication Services

The Plug&Work approach of this project aims to simplify the integration of devices in automation systems, relying on self-organizing properties of IoT technologies and protocols. This WP focuses on developing middleware services and components that allow taking advantage of the self-organizing IoT, in order to simplify access to the IoT resources by automation applications. This WP is in particular concerned with networking and distributed protocols that manage the mentioned resources. Both during a setup and an operation phase, allocation of network resources connecting and supporting applications will have to occur in a manner transparent to the application, supporting the idea of invisible networks. The resulting Plug&Work architecture components and functionalities, which enable an automatic configuration of the system, are explained in this deliverable. We refer here to system bootstrapping.

Although we define different stages for bootstrapping the full system, in this deliverable we are mainly concerned with the dependencies and interactions that should be common to all components of the IoT. We then plan on addressing other resource management functionalities in a later stage of the project. These include composition and orchestration of bootstrapped resources to meet the different types of applications addressed in this project. WP2 in particular is concerned with the definition of applications needs for reliable and timely communication, which we refer to as “advanced communication services”. These are parts of the IoT resources, which we call communication and networking resources, that are offered to automation and manufacturing applications. The different architecture components, whether concerned with hiding network complexity or security functional blocks, will be part of a final middleware infrastructure. Developing such a middleware will be carried out in an iterative and adaptive manner (agile development), and in close synchrony with the updates and findings of WP1’s architecture and system requirements and the security components and security audit of the developed architecture, carried out in WP3. The final result of WP2 will be a validated and integrated framework supporting Plug&Work of IoT-based automation systems (developed in two parallel tasks – Task 2.5 and Task 3.6). Objectives addressed by the WP include:

• Supporting Plug&Work at all levels of automation systems, from the application level down to the plug and play network.

• Providing communication services for industrial environments on-demand and in an adaptive manner, over a shared heterogeneous communication infrastructure, and catering for very different application needs.

• Developing self-organization and self-configuration concepts and protocols to provide the required robustness, reliability and real-time support for industrial environments. Only a minimal network configuration effort should be required at the communication network nodes for setting up a fully functional industrial communication network.

• Enhancing the flexibility of the advanced communication infrastructure by supporting different industrial services and applications with different requirements.

• Enabling automated network management with the least amount of manual configuration possible.

• Comparing centralised and decentralised network management approaches.



• Taking care of mobility requirements in industrial environments by introducing reliable, robust and real-time wireless links using the newest wireless network technologies.

Essentially, the approach followed in WP2 (partially reported herein) is a bottom-up exploration of the functional components that will be part of the complete IoT@Work architecture, which is addressed in WP1 in a top-down manner (this will be reported in D1.2). Ideally, these components will be based on existing internet standards and protocols, which will be tested against the project requirements and innovatively combined to offer specifically Plug&Work functionalities.

1.2 Task 2.1 Definition of Advanced Communication Services

The second phase of the task will specify the interaction mechanisms between the higher layers and the definition of advanced communication services, by specifying how to establish communication primitives from application layer data models or specific application requests (either through data-flow descriptions, or interaction descriptions generated from engineering stations), and how to translate this into a list of communication primitives. Because these primitives will provide an abstraction from concrete communication protocols, it will become possible to develop network-agnostic applications. Deliverable D2.2 presents the early results of this task, which are the specification of the communication needs and requirements in the automation domain. These requirements are presented in Chapter 3.

1.3 Task 2.3 Techniques for Realisation of Advanced Communication Services – Network Bootstrapping

The orchestration sublayer is defined in the WP1 architecture, in terms of assumptions and interface. This sublayer realises the advanced communication services, by using the appropriate mechanisms to (i) discover the network resources, (ii) configure and structure networks, (iii) allocate the network resources with different levels of guarantee, determinism, security, reliability, etc., and (iv) adapt resources in a Plug&Work manner.

The objective of this task is the definition of the bootstrapping procedure based on Plug&Work mechanisms, so as to be able to support the orchestration sublayer of WP1. The investigation of existing self-organizing protocols towards finalizing a bootstrapping procedure is one that will involve development and testing (based on simulation) of different alternatives. A final recommendation for the configuration bootstrapping procedure should result here. The results of this subtask will be provided in deliverable D2.2.

• Sketch of general bootstrapping architecture (WP2/WP3)

• Network bootstrap: integration of network bootstrapping and auto-configuration protocols in a network view of connectivity.

• Device bootstrap: influence of device interfaces and communication capabilities on bootstrapping procedure. This is based on the procedure for naming and addressing designed in Task 2.2.

• Application bootstrap: influence of application structure and meta-data on structuring networking.

• Communication Service initialisation: as a separate procedure to identify required resources in structuring networks with the help of routing mechanisms and (possibly inspired by wireless ad-hoc networks).



1.4 Scope of this Deliverable

1.4.1 Chosen Methodology and Deliverable Structure

The deliverable represents a snapshot of both architectural assumptions and interpretation of system requirements that have a direct impact on the bootstrapping process. The final architecture is not targeted at this stage - only the view points that are important for defining the bootstrapping process are explained. These view points, found in Chapter 2, include a user point of view and the definition of the bootstrap process in the IoT@Work scenarios. The other view points are mostly related to the impact of a layered and sliced architecture on the modularity, decentralization and dependencies among the functional components. The device view point, including assumptions on device types and capabilities, is also addressed. The architecture provides a top-down guideline on separation of concerns and on the implementation of the proposed functionalities. This deliverable is concerned with the bootstrapping process, and therefore it is important to keep in mind that the resulting architecture components are not complete yet. They represent a subset of the functionalities required in the IoT@Work middleware.

The targeted system requirements and, application needs and constraints, especially those related to factory automation, are revisited in Chapter 3. The advanced communication services are formulated as requirements on the network management and orchestration in general, while focusing on system configuration and start-up phases.

Besides some of the already defined architecture principles, like layering, decentralization, and self-organization, the bootstrapping process is also defined along the line of some scenarios, which are directly generated according to the scenario families introduced in Deliverable D1.1. Bootstrapping scenarios mostly fit within the agile manufacturing family, where adaptivity and agility of the system is targeted. In Chapter 4, the selected scenario is analysed. The example is taken from the planned demonstrator at the FIAT research center laboratories, related to an energy monitoring and control application of the production system. The chosen scenario is used to discuss how the bootstrap process should work in IoT@Work, from “plugging in” a new device into an existing running system, until starting all processes that enable a full integration into a distributed SCADA application.

The assumptions about the running system are important to understand the pre-requisites and dependencies in carrying out the bootstrap process. To some extent several technologies already exist for dealing with parts of the bootstrap process and these technologies are also defined in Chapter 4. In order to enable a full Plug&Work without interruption of the production processes, some extensions to existing technology are needed to lead to the IoT@Work bootstrap architecture.

The abstract definition of the required functional blocks is considered in Chapter 5. This chapter also indicates in which direction the chosen technologies might be used or extended, and defines clear dependencies between different layers. The relationship between device context, layers, and security and trust systems are also defined there.

Chapter 6 defines the possible impact of IoT@Work bootstrapping on existing standardization efforts and the tool set found especially in the factory automation field. An important aspect is the hierarchy that is found in automation systems. These are considered alongside the current trends for making automation cells modular, agile, and extensible, given the importance of these standards for IoT@Work. Finally, Chapter 7 concludes this deliverable.



1.4.2 Basic Assumptions

At the architectural level, we assume that the IoT application components will interact with each other both through the use of services and events. Thus it is important to support both paradigms – service-based systems and event-based ones. Given the domain, real-time constraints cannot be ignored and the communication services should try to support these in their interfaces for either paradigm. In a Service-Oriented or Object-Oriented (OO) system, component interaction needs the establishment of common interfaces (offered by a server and used by a client) and the dissemination of the points that offer these (e.g., WS address or remote object reference). In an Event-Based (EB) system, interaction needs the establishment of common event types (produced by publishers and consumed by subscribers) – what OMG’s DDS calls domains. A domain is essentially equivalent to an interface. In an OO system a client sends a server a message that conforms to the server’s interface, to ask the server to perform the actions linked with that message’s method and receive any results. In an EB system publishers emit event messages that subscribers react to. The main differences between the two (OO & EB) is that in OO interaction is mainly synchronous, 1-to-1, and the interacting elements know each other, causing problems when one fails, is unavailable or needs to be replaced. On the other hand, in EB interaction is mainly asynchronous, N-to-M and the interacting elements do not know each other, thus easing the replacement of elements and making it easier to mask failures.

At the technical level, we assume that IoT devices will be powerful enough to run complex services and offer support mechanisms (e.g., cryptographic functions), without major constraints with respect to memory and/or computation power (within reason). The reason for this is that these devices will be generally large enough to be IP-enabled (not tiny sensors or RFID enabled objects), working in an environment where power and connectivity is not a big problem.



2 Architecture Overview and Approach

2.1 Overview

Wikipedia1 defines:

"Bootstrapping or booting refers to a group of metaphors that share a common meaning: a self-sustaining process that proceeds without external help"

In our specific project we use the term "bootstrapping" for a process which changes the state of a device, a subsystem, a network (or parts thereof), or an application from "not operational" to "operational". Bootstrapping has prerequisites, i.e. pre-configured device properties coming from a planning/commissioning process, and it has post conditions, e.g. for running a web based application, a naming service must be made available to allow the use of symbolic names.

As in the Wikipedia definition, bootstrapping is self-sustained in the sense that the process is triggered by some local event (i.e. power on), then runs as an autonomous process until it reaches its final stage. Autonomous is an important property as in many cases external source of information or controllers may not yet be reachable at some stages. On the other hand, it is important to have a reasonable amount of external control on bootstrapping processes, so as to be able to influence them if needed. Therefore we divide the whole bootstrapping process into different "phases", which form a sequence of independent processes, but with clearly defined points in time marking the beginning and/or ending of phases. "Phases", in the IoT@Work context, are clearly related to layers (as in OSI protocol layers for distributed systems or in operating systems layers for HW, drivers and more), but the terms phases and layers are not identical. Section 2.2 identifies and briefly describes these phases and layers.

Next issue to consider is the question of what is bootstrapped. We have to distinguish between single devices (end devices or network nodes), applications, middleware components or combinations thereof. In the general case, we also have a combination of multiple entities to boot. A device typically is a hardware plus firmware and operating system but it also contains services (e.g., an automation SW or, a management agent), and it may contain middleware components (e.g. a local part of a messaging service). Moreover we will have to cover cases where multiple devices and services form one logical entity that needs to be booted as well. Section 2.3.4 introduces the more generic term "things" for devices or compounds.

1 http://en.wikipedia.org/wiki/Bootstrapping



Figure 2.1: Bootstrapping Phases

A "subsystem" is a set of associated devices, resources, services and properties which may need a bootstrapping process to start operation. Note, resources in this paragraph are passive and describe either physical and limited assets like hard-disk space and bandwidth, or they describe a certain set of data, for example the "resource file" of an application. Services in this context are active components (they "do something"), which use one or more resources to provide their service.

In order to identify and describe subsystems that are bootstrapped, we introduce in section 2.3 some top-down architectural assumptions that define the role of the IoT system model and its impact on manufacturing systems. We then analyse the IoT@Work adopted system model by means of different viewpoints on the overall architecture. The goal of this deliverable is to define the architecture components that enable a Plug&Work bootstrap, which is achieved through a bottom-up approach.

The bootstrapping process of the device-local computing system and its applications is typically controlled by the operating system (or BIOS/firmware) and sequences or dependencies are specified in a configuration of the controlling OS. Subsystems may have a central controlling entity (e.g., the PLC placed in the network controlling slave devices such as I/O modules) or they may boot by having their components co-ordinate in a decentralized manner. In either case, components must have a basic connectivity before a controller can issue commands or another component informs it of its new state. Controlling a local boot can be achieved by various means. For example the Linux Variant Ubuntu uses an event-based approach, while others may use simpler sequential boot scripts [30].

In all cases dependencies and race conditions may arise if a local service needs access to external components such as a network time service. In the case where a race condition occurs or some stage fails, the systems typically use some default values (e.g., for the current time) or simply fail to boot up completely. In such cases the systems may not have the required state at the end of the bootstrap process. Nevertheless, higher layer services should be able to continue their boot into a

pre-configuration

basic connectivity

middleware booting

power on application /

service boot

commissioning of initial parameters;

offline

HW initialisation

firmware boot

Ethernet/IP

boot

start of local services

discovery of external components

application start

t

main IoT@Work bootstrapping scope (blue)

manufacturing

commissioning

Operating System boot

device boot

external control, e.g. DHCP

external control, e.g. discovery

app specific external control



restricted state, while the system is updated as soon as the above conflict has been resolved.

Next, changes in the overall system may require changes in the state of a device. To avoid deadlocks, slow-downs from timeouts, and to allow re-configuration in the live system, we have the following requirements for an IoT@Work enabled device2:

• In case of failures within a boot stage, the stage must be initialised by means of default values to bring it to a well-defined state.

• If external services needed for the bootstrapping of a sub-system are not available, reasonable timeouts must ensure a non-blocking operation.

• If possible, a certain subsystem must be able to change its state in the live system (re-boot), without requiring other systems to stop and re-boot. As an example, a device may boot off-line, but if the network interface comes up later on, the services associated with this device or affected by the presence of external services (layer2/3, eventually system clock, etc.) should be restarted.

• Independent stages should be able to boot concurrently.

2.2 Phases in the Bootstrapping process

2.2.1 Offline Configuration

For some systems bootstrapping begins on a green field with an identification step without any security credentials. These systems have very few means against security compromises.

Therefore, most specifications presume the existence of initial imprinted security credentials, e.g., in the form of a trust anchor, a key store or a Trusted Platform Module (TPM).

Although strictly speaking the configuration of a device before it connects to the production environment is not part of the IoT@work bootstrapping process, this is an important rollout step to bring initial parameters into a subsystem configuration.

Basic device credentials may be stored on the device during manufacturing or during the initial phase of device installation. These credentials are independent of the later installation location of the device (production environment) – they are only manufacturer/device specific. Using these basic credentials, the security credentials of subsequent operational phases may be securely negotiated and exchanged.

The following credentials may be provided during manufacturing:

� Device identity.

� Device certificate and corresponding private key.

� Root certificate, for example of the Manufacturer Certification Authority (CA).

� Network interface addresses, typically preconfigured by a manufacturer. This covers, e.g., the IEEE 802 MAC address (Ethernet, WLAN) or the SSID (Service Set Identity) of a WLAN. It is possible to change these IDs in a running system.

Some requirements for these basic credentials are:

2 Note, these requirements are not new and are in general state of the art in common devices but they

must be met in new IoT@Work components as well.



� Support unique identification of the device.

� Secure storage of “private” credentials, e.g., in a TPM (Trusted Platform Module) or a flash memory.

� Secure Platform (integrity) attestation (the credentials must be bound to the hardware in a secure manner).

Specific types of imprinted keys in the automation environment:

• Trust anchor

In cryptography a trust anchor is defined as the root of a chain of trust. This root (e.g. a certification authority) has to be trusted and is typically represented by a root certificate containing the public key of the CA. This trust anchor can be used to verify digital signatures, e.g. for server authentication to the device and to establish an encrypted Channel (SSL, SSH). The trust anchor must be protected from unauthorized exchange.

Digital Signature verification is typically used in automation applications for authentication of firmware or software updates, configuration changes and (remote) administration.

Small embedded devices with highly restricted memory sometimes may only want to store a public key, without the additional certificate bytes (e.g., an X.509 certificate may need several Kbytes, depending on the context and key length). In that case the public key used for signature verification is only implicitly bound to the signer’s identity and the validity of the signature cannot be verified securely. Furthermore, there is no means for key revocation or for limiting the key validity time period. In this case, concepts for access control to the trust anchor should be complemented by concepts that renew the trust anchor on a regular basis.

• Key store with device specific public/private key pair or secret keys

The device specific public/private key pair enables mutual authentication and encryption between devices in conjunction with the root certificate.

They are more difficult to protect from unauthorized access in the automation context than in the office world, where the user provides the authentication factors through a combination of knowledge, credentials possession and biometrics.

In the automation world, where no user interaction is desired, these authentication factors have to be presented by the machine itself. To provide protection from unauthorized access to the key store, current solutions often resort to obfuscation of the access key to the key store, even though security of the solution cannot be verified scientifically.

Besides obfuscation, security dongles or chip cards may be considered as suitable authentication mechanisms, which are attached to or inserted in the machine. Nevertheless, the software utilizing this security HW must be appropriately bound to the security device.

• Cryptographic Hardware Modules, e.g. Trusted Platform Module (TPM)

Dedicated cryptographic modules provide definitely higher protection of security credentials. TPM chips compliant to the TCG specification provide a public/private key pair embedded in the device’s hardware and cryptographic functions, to generate derived keys, as well as digital signatures and verification functions for these keys. A special function called “sealing” allows



encryption of arbitrary device data, with only the device being able to decrypt them.

The TPM is meant to verify the platform integrity.

The TPM specification was in general planned to improve the security of Personal Computers. Some so called TPM chips use common cryptographic chips and realize necessary functions in software to be TCG-compliant. Such solutions are usually not suitable for automation devices that don’t support a standard Operating System.

• Physical Unclonable Function (PUF)

If biometrics could be applied to automation devices, an easy solution for device authentication would be at hand. Key stores could be accessed securely by the device, providing a unique, unclonable function result. Some companies claim to have found such PUFs, but are just beginning to present their solutions to the public.

It is clear, that if biometrics has restrictions by false positives and false negatives, so may have a PUF. The usefulness of these new techniques in the automation context is still to be proven.

2.2.2 Power On

If a device is powered on (or does a reset), it will typically initialize the internal SW to a point where - in case of networked devices - the "Basic Connectivity" phase can start. This is typically very platform specific and beyond the scope of this project.

The result of this phase is a device where a clock is running (but may have wrong values), self-tests of the HW have been completed, and eventually adjustments and configurations of the HW (i.e. processor speed, memory cycles and so on) have been applied. Relevant parts of the OS bootstrap after power-on will be discussed in the respective paragraphs.

2.2.3 Basic Connectivity

Basic connectivity is a state where a device is able to communicate. We consider only IP based systems, which may be combined with non-IP based real-time services (i.e. Profinet RT) but do not consider pure field-bus systems. At power on, a device typically starts its firmware and/or operating system. At this phase, first all Ethernet interfaces must boot, then the IP stack must be started. Unless stated otherwise, we do not distinguish between IPv6 or IPv4 for this discussion. Other media, i.e. wireless LAN or specific industrial Ethernet variants may differ on the physical and MAC layer. We concentrate here on the most important case of wired Ethernet (IEEE 802.3). For this discussion, differences on the physical layer and partially on the MAC layer can be neglected.

This approach attempts to re-use and adapt the bootstrapping processes from Enterprise networks, where we already find a rich environment for secure Plug&Play of devices and services. IoT@Work scenarios have (in this respect) two fundamental differences to Enterprise networks:

• There is a clear need to plan, engineer and configure resource usage in automation networks in order to meet the strict reliability and real-time requirements.



• Many services offered in the network are bound to specific physical resources and often even to locations. I.e. a service giving a temperature value must ensure that it gets its values from sensors located at a certain location.

There are additional significant issues in an IoT network. Resources of a device may be limited compared to PCs or servers found in an Enterprise network. We however deal mainly with devices that are attached to an IP network and thus will have significant more computational power and memory compared to e.g. a smart card or a RFID. For example the discussions in this deliverable may consider a RFID reader but not the RFID itself. In particular the consequence is that - also we mainly talk about embedded devices - we assume to have enough computing power to perform the relevant security and management functions and we will have enough on-board memory to eventually store relevant data, i.e. secure IDs. Also note, the term "embedded" typically also includes ruggedized server platforms, visualization terminals and other components, which basically are powerful platforms with standard operating systems (i.e. Windows or Linux based), multi-core processors and gigabytes of disk space.

Thus we have the means to adopt Enterprise best practise but we have to optimize and adapt the processes.

2.2.4 Middleware and Subsystem internal Helper Services

Middleware is a set of software processes and libraries whose goal is to support the interaction among a collection of devices and applications, thus hiding away the fact that the system is distributed. Some middleware services build upon the services provided by operating systems (OS). An OS itself abstracts away the specific functionality of a computing system’s hardware and presents abstractions having to do with the processor, the short and long-term (file system) memory, local events (interrupts and networking messages), the protection of processes (if a multi-tasking OS) both with respect to their memory and with their access to the processor (scheduling), and of the users (if a multi-user OS). Similarly, other middleware services offer abstractions that allow one to program a distributed system as if it was a centralised one. So they offer the ability to use remote services (remote procedure calls – RPC – or remote method invocations – RMI), receive and produce remote events, and so forth. These abstractions are aided by a number of common middleware services, such as naming/trading, which allow an application to discover remote services – a role that in a classic OS may be undertaken by a (dynamic) linker. Some of these services may not be available in all middleware. For example, there are no remote services offered in a completely event-based middleware, so there is no need for a naming/trading service either, just like an application that only reads and creates interrupts but uses no libraries needs no real support from a dynamic linker.

The functionalities analysed below are the ones we currently consider pertaining to the IoT@Work middleware, and relevant for interaction among devices and modules in the IoT@Work scenario. Additionally, we distinguish between middleware related to the devices as a whole (identified as device level middleware in the following) and middleware related to specific application needs (identified as application level middleware in the following). Under the first category we include middleware functionalities that enable a device to become operative within its environment; while the second category considers functionalities that are specific to single applications. As evident, the list of functionalities can evolve over time as, for example, new applications and services are required and defined.



With reference to the device level middleware the currently identified functionalities within the bootstrapping scenario are:

• Device network configuration (including handling of device identities): under this umbrella we collect all functionalities required to check if a device can be allowed to operate on the network infrastructure, properly configure its network interfaces and admit it on the network. This class of functionalities (which includes features like support for NAC discovery and NAC negotiation, DHCP services discovery and IP related parameters request and acquisition, etc.) has therefore the objective of making a device properly ready from a network point of view;

• Device non-network-related configuration: under this umbrella we collect all functionalities required to complete the configuration of a device within its operative context. Therefore the functionalities in this class rely on the successful completion of the device network configuration to communicate with surrounding services or devices, in order to complete the configuration of the bootstrapping device. Among others, this class includes functionalities to acquire information about services and to acquire data required to access them. For example, the IoT@Work scenario can envisage the usage of a network-wide event notification service to collect log and audit events from all devices and services. These events can then be made available to a number of applications, such as monitoring applications. To this end, specific data like event notification service endpoints, device authentication, logging namespaces, etc. are required. This class of functionalities has therefore the objective of making a device ready from an operative point of view;

• Device system configuration checking: under this umbrella we collect all functionalities required to check if the software configuration of a device is in line with the overall policies3 and, in the case it is not, start a patch or update process. This class, therefore, collects features required to complete the bootstrap process4 of a device and switch its state into a final operative state.

These classes of functionalities normally involve both specific software modules within a device, as well as a set of deployed services (see Figure 2.2) with which each device has to interact to complete its bootstrap.

3 This class of functionalities does not include checks related to the compliance of the device software to

the NAC policies. Indeed, NAC checks are performed during the network configuration phase.

4 As an example, if a software update has been planned for a subset of PLCs and a PLC within this

subset is switched on, it will become aware of this software update and start the update procedure.



Figure 2.2: Bootstrapping Supporting services

Some network bootstrapping features have already been depicted and will be better defined in the following. Here a description of how a device interacts with the supporting environment so as to become active within an overall logging and audit service is provided as an example.

It is assumed that a device has completed its network identification and configuration phase, so it is a fully-operating device from a network point of view. Additionally it is assumed that each device generates logging and audit events according to the following taxonomy:

• Error: this event class identifies events related to significant problems in the device (e.g. fatal program errors). This class therefore is used to classify situations that compromise the expected behaviour of a device or of an application;

• Warning: this class identifies events that are not necessarily significant, but may indicate a possible problem. Events in this class indicate situations that could impact the expected behaviour of a device or of an application;

• Information: this class identifies events that provide information on the completion of phases, tasks, etc. (e.g. the correct initialization of a software module). Events in this class therefore confirm the status of the device or application and are targeted at providing useful information and data on the completion of some activity;

• Configuration: this class identifies events related in some way to the configuration of a device or application (e.g. successful loading of a configuration). Events within this class are useful to monitor configuration changes and reconfiguration completion;

• Success Audit: this class identifies events related to successful “security related” operations (e.g. successful completion of authentication processes, successful changes of credentials, etc.) both when a device/application is



operating as a “client” toward other services, as well as when the device/application is acting as a service accessed by “clients”;

• Failure Audit: this class is used to classify unsuccessful “security related” operations;

• TraceDebug: this class is used to collect much finer information, not covered by the previous classes. Under this class are events normally collected for debugging purposes. Normally the possibility for devices/applications to generate events of this type is disabled in production environment or subject to specific configuration/authorization procedures.

Furthermore each device publishes its events via an Event Notification Service so that all interested services can receive the kind of events of their interest. Each monitoring service specifies to the Event Notification Service the kind of events of its interest doing a kind of “subscribe”.

To support a flexible management of event dispatching, the log and audit events are organised in a hierarchical namespace, as depicted in Figure 2.3 and Figure 2.4 5.

Figure 2.3: Top level log and audit namespace structure

5 The figures provide an example of the organisation of the log and audit namespace. The actual

structuring has to take into account, for example, the production organisation (productions lines, categories of devices, etc.), as well as the monitoring objectives.



Log Category

Network

Switches

Sw1 ...

Sw2

Routers

Rtr1 ...

Rtr2

...

... ...

...

Production

Robots

Rb1 ...

Rb2

PLCs

PLC1 ...

PLC2

...

... ...

...

OSApp1

...

ServiceXOS

Figure 2.4: Bottom level log and audit namespace structure

As reported in Figure 2.3 and Figure 2.4, the log and audit namespace is structured into the taxonomy categories6 described directly under the root (LogRoot in Figure

2.3). Each category has a sub-tree with a common structure that is related to the specific production organisation, categories of devices and applications, etc.

In Figure 2.4 this sub-tree has been split among network and production simply to exemplify how to keep together events that are related to each other (or can be relevant for specific monitoring services).

At the bottom level there are single devices (switches like Sw1, Sw2, robots, etc.), or single software modules (e.g. PLC1 operating system, PLC1 Application App1, etc.).

Within this scenario a booting device needs to acquire the following information:

• Log–Audit Namespace structure: this information informs the booting device about where to publish its logging events. For example robot Rb1 will receive the information that its events have to be published under the topic7 LogRoot.<Log Category>.Production.Robots.Rb1, while the application App1 on PLC1 will publish its data under the namespace LogRoot.<Log category>.Production.PLCs.PLC1.App1;

• Device Logging Level: this value8 specifies for which log categories the device has to publish events. The value assigned to this parameter can impact other device/service configuration parameters9;

6 The TraceDebug category is greyed to highlight that is a category normally disabled or that require

specific action to be enabled.

7 Values within “< >” represent variable fields. Therefore “<Log Category>” can be Error, Warning, etc.

8 Usually the Logging Level is organised hierarchically (see “Java Logging Overview” on

http://download.oracle.com/javase/1.4.2/docs/guide/util/logging/overview.html) with the following range of values: SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, where SEVERE



• Event Notification Service Endpoints: this information provides the device or application the data required to actually setup network connections with the event management service, for example specifying the ENS server’s names or IP addresses and TCP port;

• Event Notification Service access credentials: this information has to be used by the device/application in order to gain access to the ENS as an event provider (i.e. event publisher).

The configuration information reported above can be acquired by a booting device dynamically (for example during the network configuration phase) or via the use of ad-hoc configuration files. After acquiring the above data the booting device has to perform the following operations:

• Connect to the ENS using the endpoints data;

• Present its credentials10 that, on one hand, qualify it as an event provider (i.e. event publisher), and on the other hand are used to assess if the device is actually allowed to report events under the declared Log Category. To this end it is envisaged to make use of Capability based Authorization credentials11. The booting device will therefore submit a duly digitally signed request to the ENS, requesting publish access to the log and audit namespace; this request must include the device Capability based Authorization credentials. The ENS will check the request and the credentials and, if they are acceptable, grant access to the requested namespace;

• In response to the submitted request, the booting device will receive the data required to actually become a fully operational event publisher, thus completing its log and audit configuration setup.

2.2.5 Applications

In the following paragraphs we provide some details on the application level middleware as defined in the previous sections. As evident, this middleware has to support interoperability among applications/services on different systems and comes into play after the device on which the application/service runs has completed its booting and is completely operative.

is the highest value. The Logging Level specifies the lowest level of events to be reported; a system therefore will report all events from its assigned Logging Level up to the SEVERE level. With reference to the log and audit namespace structure, a device with a Logging Event equal to INFO will only report SEVERE events under the Error namespace branch, WARNING and INFO events under the corresponding namespace branches, and drop all other kind of events. Anyway, best practises in distributed systems suggest to always report events pertaining to the classes SuccessAudit and FailureAudit due to their relevance. In the exemplified log and audit namespace all FINE, FINER, FINEST events will be reported under the TraceDebug branch.

9 For example a Logging LEVEL equal to FINE/FINER/FINEST could require some specific device

configuration or Event Notification Service access credentials in order to avoid DoS attacks flooding the monitoring system with a lot of events .

10 These credentials, as stated, can depend on the device Logging Level.

11 A Capability based Authorization credential is a kind of security token that grants to a specific subject (the

booting device in this case) rights (in our case the right to publish) on a specific resource (in this case one or more Log Category).



This middleware cannot be completely defined in this phase of the project, since its functionalities are heavily dependent on the specific needs of the applications and services. Therefore, in the following we focus on issues that are quite universal and independent from each specific application. As for the previous kind of middleware, the list of functionalities can evolve over time as, for example, new applications and services are required and defined.

With reference to the application level middleware the currently identified functionalities within the bootstrapping scenario are:

• Application configuration: under this umbrella we collect all functionalities required to provide configuration data to applications or services. For example, a production Energy Monitoring system can be based on the collection of energy consumption measures distributed via an Event Notification Service. In this context the software modules controlling the energy measuring sensors need to have configuration data for properly connecting to the event notification service and properly report their measures under the envisaged namespace;

• Application configuration checking: under this umbrella we collect all functionalities required to check if the application or service software is at a version/patch level in line with the policies and, in the case it is not, start a patch or update process. This class, therefore, collects features required to complete the application or service startup process, assuring that the application conforms to the policies;

• Interoperability functionalities: under this umbrella we collect a set of features whose objective is to support the interoperability and integration of applications. These features span from supporting services to identify and locate services (for example a UDDI service even with semantically enriched service descriptions), to asynchronous communication services. The Event Notification Service, from this point of view, can be considered as providing features within this functionalities category. Indeed, applications can use the ENS to exchange information12 and add flexibility to the integration patterns.

2.3 View Points of the Architecture

2.3.1 View Points

The overall brief architecture of the IoT@Work system plays a major role for a detailed analysis and specification of the bootstrapping process. We distinguish and briefly describe four different views on the architecture:

• The layer view shows how the IoT@Work architecture relates to the automation pyramid and the OSI 7-layer model. It shows how protocols and SW parts rely on each other to provide higher abstractions. Important for the bootstrapping process is the fact that the boot sequence starts in the lower layers and then goes up the layers step by step. Each layer needs prerequisites from the layer below and provides services or information for the layers above. For example a messaging middleware relies on a working network stack (layers 1 - 3).

12

The Energy Monitoring Service is an example of the use of the Event Notification Service as an integration/interoperability tool. Indeed, the devices measuring energy consumption within the production environment do not have to take care of managing, or even knowing, the applications that use their measures and, in addition, new energy related applications can be added to the environment simply authorizing them to gain access to the energy measures they need.



• The functional view identifies functional components (=services) and shows how they are related and which interfaces they have. Functions may be distributed over the network. They may depend on each other and thus enforce a certain boot sequence. For example a PLC functionality will need - amongst other prerequisites - a working system clock function before it can start.

• Resources are modelling physical properties of the system. Thus a resource view shows a coarse granular map of important resources and the management functions for them. A bootstrapping process must ensure that the relevant resources are available when needed.

• The slice view is an IoT@Work specific means to visualize various applications or application domains using the same resources and functions. As such it plays the role of a cross-layer virtualisation, implemented with very different means. Slices are associated with resources and security policies and are an important abstraction to model a multi-tenant capable system. For the bootstrapping, this implies that eventually the same functions must be configured for different tenants. For example a messaging middleware must start and configure its infrastructure and then eventually construct private name spaces per tenant. A network may construct i.e. VLANs (or other means) to support the slices and eventually configure filter rules on dedicated elements to enforce application or tenant specific policies.

These views will be used to structure requirements and to analyse their dependencies.

2.3.2 Layer View

Layers - as defined in the OSI seven-layer model 13 - are one way to structure the communication functionalities of a device. A layer always depends on the layers below, so they define a local boot sequence starting from the physical layer to higher layers. Figure 2.7 shows the layers from a network perspective, but the SW (or firmware) on a device can be described in similar terms and also has to boot in this sequence. For example a device may start from BIOS, then the hard disk boots, finally the operating system and the applications. IoT@Work concentrates on network related functionality, but it must be stated that the boot of local functions and network related functions is coupled and has dependencies. For example the IP layer boot will be triggered and performed by the operating system - it is in this respect part of the OS boot and controlled by the operating system until finally a specific application is started14. The OS is also responsible for parallel booting of sub-systems and for handling errors and timeouts.

At least for the layers 1 - 3 it is true that they will typically communicate with other entities in the network to set up their respective layer. If the adjacent counterpart is not responding or not reachable several consequences are possible:

• The boot process could fail or a communication layer could be not operational.

• The process can be significantly slowed down due to time outs.

• The OS may apply default values.

13

http://en.wikipedia.org/wiki/OSI_model

14 http://www.bootchart.org/for examples of the Linux OS boot sequences.



• The OS may try a different strategy to complete the boot, e.g., if the DHCP does not respond, the OS might fall back to IP auto-configuration.

Some of the layers may boot - or re-boot - at arbitrary times in an already running system. This may cause reconfigurations at different parts of the system.

Designing a system in a way that it allows to be re-configured, or partially re-booted, at any time is an important property of robust and self-configuring systems. It allows circumventing dead-locks and repairing or changing a system configuration.

2.3.3 Functional View

A Functional View is a way to structure the IoT@Work architecture into units which provide certain functionalities, no matter how these are implemented. But functions are provided by services running on one or more platforms, so for the bootstrapping process dependencies must be considered. Functions are typically composed from other, lower-level functions (non-exclusive list):

• Network Function: basic communication enabling functionalities provided by network devices like: switches, routers, NATs, firewalls, etc.

• Resource Management: relies on agents on the devices that are responsible for the respective resource, e.g. the traffic control functions for a network interface or the quota function for a file space.

• Security Management.

• Device: in its general form, a device has an Operating System to control other SW components and uses a hierarchy of specific functions to manage and access the HW (i.e. drivers, file system). Also contains at least one network interface as a special resource.

• Middleware Functions (at various layers) include:

o Gateways translating between different protocol realms or between administrative domains.

o Proxies which decouple entities and add filtering, caching and resource management (e.g. shaping, load balancing, policy enforcement) functionality.

o Messaging forwarding systems.

o Various management related functions, e.g. event logging.

o Network Infrastructure functions enriching and complementing the network function. Examples include address mapping, filtering, policy control and enforcement, network resource management, multicast, discovery of addresses (which can be mapped to discovery of services), etc.

• Storage Functions: may use a hierarchy of storage spaces and related management functions.

• Databases: will use storage functions as a backend.

• Many application-specific functions.

Looking at the bootstrapping process from the functional perspective makes it clear that the low level functions of a system must be available first, to allow a distributed and composite function to become operable. There is also again the danger of having dead-locks in the bootstrapping phase. That is why bootstrapping must be



done in multiple steps: eventually start a function with limited capabilities and default values, and then re-adjust to the final configuration.

2.3.4 IoT Resource View

Figure 2.5: Architecture – IoT Resource View

The resource view explains the impact of an IoT approach on the architecture developed in this project. We assume that things go beyond passive objects such as material or physical components, parts, or products, which are equipped with some digital components like an RFID. In this project, and when considering the factory floor, things are any physical entities that are packaged with some software component (either embedded in that entity or remotely linked to it), which offers a unified view on this bundle of components - something we call an IoT resource. Such an IoT resource can be accessed through some communication network that makes the resource accessible and can be manipulated through a service interface. The logic behind this manipulation can be in the form of some pre-defined protocol or application logic that can make use of that resource.

This assumption allows us to generalise the concepts of things to all types of components, devices, and physical parts, which both include some intelligence of some sort and are linked to physical entities. This definition would apply to both a network switch and a robot.

Embedded devices are often the type of things we target in this project. They are things, i.e. physical entities with often an embedded IoT resource (i.e. digital intelligence and communication interface). This concept of things is a fractal concept, meaning that a thing could also be a combination of smaller things, with their own resources, presented and made manageable as a single resource.

In IoT@Work a communication network offers a resource called a communication service, which basically offers connectivity with some properties like, e.g. Quality-of-service, message routing, and more. This communication network consists of things called network nodes that each offer a networking resource of their own and their combination or interaction forms a bigger and more complex network resource.

Similarly, some automation devices, like a robot, are a composition of different entities and their associated intelligence or IoT resources, like the robot arm, positioning sensors, and actuators (different motors), offer in a combined manner the



robot’s production resource. These compound devices may still provide one unique IoT resource view hiding the complexity within.

A manufacturing system is set-up to run different applications in a concurrent manner and accessing in some cases the same IoT resources. A distinction is made between sub-systems composing the whole manufacturing system. The production, monitoring, and safety sub-systems are examples of a whole chain of applications that have distinguishable properties. In the agile manufacturing requirements and approach described in D1.1, these sub-systems can be assumed to be whole vertical slices of the same factory (also explained in section 2.3.5).

We refer to the production sub-system as the combination of underlying production and networking resources that are accessed by control applications to automate the production. Although, it is understood that production applications are also defined in a hierarchical approach following the production pyramid model. We only consider the applications that interact directly with devices in the factory floor, and those defining the control logic of a production sub-system (i.e. as of state of the art running in programmable logic controllers (PLCs) in the factory floor).

The monitoring sub-system combines those applications that run mostly in a combination of ERP and MES back-end services, and gather data or events at the factory floor. This type of applications does not control the production resources (like robots) directly, but might access similar resources like sensor data (provided by a robot), or the same networking resource as that used by the production sub-system.

The third sub-system identified so far, is a hybrid subsystem that allows applications similar to monitoring applications to interact with the production for emergency stops or for safety reasons (e.g. the robot arm should stop immediately if a human operator is “too close” to the moving arm).

Within the IoT@Work bootstrapping approach, we concentrate on the IoT impact in simplifying the design and the setting-up of the different sub-systems. This same approach can then be generalised to modify each sub-system on its own (especially for agile manufacturing purposes) and automatically solve resource access with the help of well defined policies and conflict resolution algorithms.

The bootstrap process first looks at the common ways to add a single resource to an existing manufacturing system from the moment the IoT entity is powered-on and connected through its communication interface, until the resource description (i.e. the embedded intelligence and its description) is made available to a resource pool (in the form of a directory service of online resources that keeps descriptions of how the resources can be accessed and used). Due to the implicit hierarchy within the adopted IoT approach, the bootstrapping process, can be repeated for each level in similar steps, until it provides the final resources that can be accessed, and manipulated by the different applications.



Figure 2.6: Architecture – Bootstrapping

More concretely, we consider the bootstrap of single networking and automation end-devices from the moment they are powered on, until they are connected to the system and made ready for further application setup. Bootstrapping the application is not the focus of this project, although some assumptions are made and the final goal of IoT@Work goes in the direction of providing a bottom-up description of resources in a way to facilitate pairing between application logic (which might be pre-planned or engineered offline) and the underlying resources booted in the field, and providing a tracking of their changing context.

2.3.5 Slice View

Slices (see Figure 2.7) are the IoT@Work term for a cross layer virtualisation. This allows isolating users, applications, or complete application domains from each other, with respect to both security and resource management. A slice can be implemented by a combination of many means, in the simplest case as name spaces on different layers. Operating System virtualisation or VLANs may play a role as well. As an architectural principle, this will be detailed in later deliverables.

MAC (L2)

IP (L3-4)

various supporting

Services

Application (Layer6-7)

Physical Layer (L1)

Safe

ty

Auto

mation

Surv

eill

ance

oth

er

MAC address, virtual interfaces, VLAN

IP address (incl. various tunneling techniques)

specific namespaces

unique ID per instantiation ("user ID"), URL

potential techniques to support multi-tenancy

and virtualisation

Layers(loosely modelled after OSI layers)

applicationresp.

application domain

space diversity, channels, codes ...

Figure 2.7: Slice and Layer View

From the bootstrapping perspective, slices need additional support to set up their respective identifiers, name spaces (e.g. multicast group or address groups in a messaging system), associated policies and eventually other virtualisation means (e.g. VLANs on the Ethernet or Operating System Virtualisation). As such, the slice boot can be seen more as a configuration profile of the system. Only in rare cases will it need specific bootstrapping steps (e.g. if a virtualised OS must be started).



3 Revised System Requirements – Network View

The initial list of requirements published in D1.1 are scenario-driven requirements that more or less describe the characteristics of the targeted architecture and the behavior of the targeted automation systems, which are IoT-enabled. This chapter reviews the architecture requirements in Section 3.1 focusing on the network and field levels of factory systems, since these levels are the main focus of this work. The definition of advanced communication services is provided in Section 3.2 in terms of application requirements for real-time communication and reliability. This is done by analyzing state of the art and current practices in the automation field. In IoT@Work, we do not target specifically to change the automation applications, which are also evolving in the way they are designed and programmed. However, we try to describe the constraints of the existing applications in order to understand the way automation systems and networks are architected today. In Section 3.3 we revise the naming and addressing requirements provided in D2.1 and extend them to include needs for having secure IDs and a support for a cryptographic identification mechanism that is an essential part of IoT@Work Bootstrapping concepts.

3.1 Network View on Requirement Analysis

As stated in the grant agreement, amongst the first activities in the project is, besides the state-of-the-art research, the identification of requirements. The requirements listed in this section refer to the features and attributes that IoT@Work should fulfil from a network point of view with respect to bootstrapping. Therefore, they should provide a significant input to the overall bootstrapping architecture. Since the requirements collection and definition is a time consuming process, it is expected that they will be subsequently updated and refined further during later stages of the project.

Requirement

Automatic/assisted network configuration

Description

During setting up the network, supporting means for the user when configuring the devices shall be provided. Approaches may range from a central management entity to a distributed self-organization of devices. The goal is to minimize time-consuming manual overrides and the detection of configuration errors. Three phases can be distinguished:

(1) Network planning and design which belongs to the offline phase (cf. section 2.2.1), happens purely offline in engineering tools, and includes a first layout of networks (e.g. ring, trees), connecting nodes, cells, network management, control mechanisms, etc.

(2) Network commissioning and basic connectivity belongs mostly to the offline phase as well. First, configuration and optimization of the network, offline or online, testing and optimizing configurations in the real network or in a tool, e.g. a simulator, are part of this phase. Afterwards, a basic connectivity is established as described in section 2.2.3. This also includes the start-up of core network services.

(3) Network start-up, after having a basic connectivity established in phase 2, including that all core network services are running, the configuration can be downloaded to the network devices, followed by the start-up of the network.

Phase 1 can be tool-supported; phase 2 should be widely automatic; phase 3 should be fully automated.

Quantification

a) Basic connectivity between all network nodes is available on powering-on the network.

b) Automatic configuration of network paths without configuring protocol internal parameters in each node.

ID Type Target Area Priority



R-N1 IoT@Work Network Normal

Date Author Dependencies Feature Status Reference / Scenario

11-2010 Bootstrapping New

History

Requirement

Rapid and deterministic network initialization

Description

Phase 1 and 2 as explained in R-N1 are not time critical, as done in an early project phase. Phase 3 is more time-critical, as done during plant start-up. Bootstrapping the network shall be fast to guarantee a rapid system start-up. However, this start-up phase could be distinguished for 2 views.

(1) Bootstrapping the whole system, for the first time (here the start-up time is not that critical and can take up to few minutes). The resulting network configuration shall be deterministic to produce a consistent scenario fulfilling all application communication needs.

(2) Start-up of each single device connected to the remaining system needs to be fast including device addressing, authentication, resource allocation per device, service discovery, semantic identity/role of device, place in the application, and communication needs.

Quantification

a) The whole network could take up to a few minutes to finish all configuration phases




11-2010 R-N1 Bootstrapping New

History

Requirement

Seamless network re-configuration supporting system extensibility

Description

It is necessary to allow system extensions at run-time without interrupting the currently running applications. This may refer to new hardware as well as software updates. Existing bandwidth reservations must not be deleted by updates or extensions. System extensions differ if they affect:

(1) Adding a single component/device related to automation/safety/or a monitoring application, like a new generation sensing device, a new energy monitoring device, etc.

(2) Adding a device that offers a shared resource, e.g. CPU for more processing power, or a network device to offer a new redundant link.

(3) Sub-system clones, e.g. replacing robot arm or module by another for agile manufacturing (same robot, different products) or adding a whole cell similar to existing cell by cloning it.

In case (1), we want to consider those cases where extensions do not require physical interruption of the production (no need to stop the machines), in this case extensions will result in no network interruptions. In case (2), we have a similar requirement to case (1), but in case of network extensions, we might consider reconfiguring the network but with no interruptions to the system. In case (3), we want to protect the control plane from extensions with the same way as in cases (1) and (2), i.e. near-zero interruption for the remaining system.



Quantification

It has to be distinguished whether it is:

a) A network node (switching/routing).

b) An end-device.

c) A bridged end-station.





History

Requirement

Smooth device replacement and restart

Description

Fast re-initialization of all network related functionalities after device replacements is necessary to immediately get back into a productive state. This can be also referred to as the Plug-in procedure. Existing resource reservations for the services used and offered by the old device must be recovered and, where necessary, transferred to the new device. If the system is turned to a stand-by mode, system restart has to occur fast.

Quantification

a) Replacement node should be equivalently (possibly identically) configured as compared to the original failed node.

b) The remaining connectivity should stay unchanged.

This depends also on whether this is an end-device or a network device.





History

Requirement

Authentication, security, data integrity

Description

Allow a controlled initialization when extending or changing the system (e.g. limited network access, limited access rights until authentication of device/software element). Network elements are all authenticated before messages (hello, updates, alarms,) are accepted and used at a protocol level. The messages should be authenticated, or signed.

Quantification



Authentication needs to be solved for all types of devices that are authenticated for certain characteristics. Authentication is followed by authorization to access resources (or manipulate and provide new resources). Enforcement and policing can control the access etc.





History

Requirement

Support of arbitrary network topologies

Description

Arbitrary network structures must be supported. Typical network structures are combinations of meshes, rings, lines, stars, combs, etc. Different topologies are typical at different levels of the automation pyramid, given the different QoS requirements each layer has, while all layers need to be interconnected redundantly.

Quantification

a) Which topology is preferred by which protocol? As starting point we make a review of existing

topologies at different network levels (see Figure 3.2).

b) Scalability of routing/switching mechanism? 2000 nodes?





History

Requirement

Clear identification and addressing of devices

Description

All devices, their functionality and their location must be clearly mapped to non ambiguous identifiers. Devices have to be easily addressable in order to be reachable from anywhere in the factory, office, or even world wide. There might be differences between identifying an automation device and a networking device (e.g. switch, routers, gateways).

Quantification

Amongst others network topology and localisation information is necessary for meeting this requirement. This information can be obtained by suitable protocols (e.g. routing protocols) and made available to other components.







History

Requirement

Offline and online routing along arbitrary paths

Description

The automation network may employ a combination of frame forwarding (ISO/OSI data link layer) and routing mechanisms (ISO/OSI network layer). The goal is to support a flexible routing of traffic flows along individual paths. That is, not to restrict the forwarding of traffic to a certain active tree topology as is done in Ethernet and not to impose a uniform routing of all traffic flows along a common path as is done in classical IP link state routing.

Quantification

a) Existing of signalling for establishing a network path.

b) Support of arbitrary paths (e.g. possibility to traffic engineer a path along an arbitrary sequence of selected network nodes, or two disjoint paths).





History

Requirement

Network and service reliability

Description

In order to enable the network to compensate failures, such as device or link outages, it is important to implement redundancy mechanisms. Besides built-in hardware redundancy at the network devices, this topic mainly covers the securing of normal traffic routing/forwarding by an alternative backup path, which remains intact in case of a failure along the primary path. This involves the necessity to route traffic flows along disjoint paths that do not share any node or link except for the end nodes (where supplementary hardware redundancy may be necessary). Various flavours of resilience mechanisms are conceivable such as alternative end-to-end paths or backup paths for every individual link of the primary path. If ultra short recovery times are required, it may be necessary to simultaneously transmit data on both the primary and secondary paths. Another important aspect of network reliability is the discovery of and fast reaction to failures. There shall be no impact of a failure for non-affected traffic flows.

Quantification

a) Redundant communication (e.g. one stream over two disjoint paths, or load-balanced communication over two paths).

b) Link/node failure detection.

c) Switch-over time (Grace-time of application?).

d) Computation of alternative paths after failure.

e) Multi-homed hosts.







History

Requirement

Quality of Service (QoS)

Description

The automation network shall facilitate distributed closed-loop applications with real-time requirements. As a result, the communication services must fulfil certain prerequisites in terms of bandwidth, latency, jitter, and packet loss. These requirements are different for each industrial application and each level in the automation pyramid.

The classification of applications requirements is based on the pyramid model of automation systems (cf. section 3.2).

Quantification

QoS-aware routing





History

Requirement

Network interconnectivity and reachability of devices

Description

The considered automation network is interconnected to other network segments of the automation pyramid. Furthermore, it may be connected to corporate office networks and the Internet for remote access.

Quantification

a) Multi-point communication should be possible (i.e. anycast and multicast).

b) Separating network resources for different logical networks running on top of the same physical network (VLANs).





History



3.2 Advanced Communication Services and Industrial Automation Needs - QoS Requirements in the Factory

Industrial systems can always be categorized according to their deployment considering the structure of the automation pyramid shown in Figure 3.1 abstracts the whole manufacturing site into three levels, each level offering different functionalities. The amount of data to be transferred is at its maximum at the highest level and decreases towards the bottom, whereas the real-time demands are very limited at the top level and increase at the lowest level.

Figure 3.1: The automation pyramid and corresponding requirements

The plant level is dominated by enterprise resource planning (ERP) systems, which are in charge of controlling the overall process and the supply chain management. At this level, process data is only required for accounting, production planning, etc. These data are provided by systems of the underlying (control) level. Usually, standard IT infrastructure is used for communication networks, since no data with real-time constraints have to be transferred.

At the control level, manufacturing execution systems (MES) are dealing with the manufacturing process itself and its control. That means collecting data for the manufacturing process, including data for monitoring and maintenance. The MES can directly influence the field level, resulting in higher real-time requirements as compared to the plant level. However, less data need to be transferred than at the plant level.

The lowest level of the automation pyramid is called field level, and is situated very close to the actual physical process. It has the highest temporal requirements for its communication network, but the least amount of data to transfer. This is because, devices at the field level usually exchange cyclic traffic with small payload sizes. However, other types of data exchange might also be present and are becoming increasingly popular due to their increased efficiency, e.g. change of state (COS) or polling. Furthermore, the reliability of data transmissions plays an important role, since an error might cause severe damage to the whole process. In the domain of factory automation, several application scenarios exist at the aforementioned levels:

Plant level



• ERP systems

Cell level

• Control of the production process

• HMI's for monitoring and operation of manufacturing

• Diagnosis of manufacturing systems

Field level

• Exchange of process-related data from sensors and actuators

• Distributed control systems

• Networked control systems, e.g. motion control applications

In general, the severity of common requirements for distributed industrial automation systems depends primarily on their deployment with respect to the automation pyramid. Amongst others, the temporal requirements are most important for the real-time behavior of a certain system. Real-time behavior means that functions of a

specific system are executed in a time interval [R, D], which is determined by the

application environment. In this context, the release R denotes the earliest

permissible starting time and the deadline D the latest permissible completion time. A

real-time condition is considered to be hard when the utility of a data transmission

abruptly drops to 0 as soon as the deadline D is missed. Missing the deadline D may

result in damage to product parts or equipment. When having a soft real-time condition, however, missing the deadline D can be sometimes tolerated, either because partial results are still useful or because it only matters not to miss more than N out of K deadlines.

Table 3.1 – QoS requirements in control and field level communication

Real-time requirements Real-time Application

Level

Latency Jitter Class

PLC <-> PLC Control/Field 10 – 100ms - 1

PLC <-> Distributed I/O Control/Field 1 - 10ms > 1ms 2

Synchronous motion Field < 1ms < 1µs 3

These temporal requirements, i.e. latency, and jitter are quantified according to different real-time classes in Table 3.1 and depend mainly on the corresponding application of interest. QoS class 1 describes communication between PLC’s or visualization systems, such as human machine interfaces (HMI), in order to transmit status messages or nominal values. Only moderate latencies are required and it is not necessary to consider the jitter. The real-time requirements for the network regarding communication between PLC’s and decentralized periphery, e.g. I/O devices, can be found in QoS class 2. Finally, applications for motion control have the highest requirements and are designated as QoS class 3. For such applications, latencies in the sub-microsecond range and a jitter of less than 1 µs are required. An extensive network requirements summary in accordance to the previously introduced automation pyramid architecture is shown in Figure 3.2.



Figure 3.2: Network Requirements of Factory Automation Applications



3.3 Naming and Addressing Needs of Automation

3.3.1 Directory entries and data-base management

The directories listing device IDs at remote maintenance provider should match those of the factory owner.

The IERC Strategic research roadmap suggests developing solutions for managing, at the same time, different "names" for the same object. These names can, for example, identify the object according to a "naming strategy" of the provider (e.g. "Comau Robot NG3-SN xxyyzz") or of the owner (e.g. "FIAT Mirafiori Robot NG3 Line 1-Cell 3").

Objects can also be identified as members of a specific class (e.g. "Robot NG3"), especially if we take into account the IoT@Work objective to dynamically plug new devices at the network level. We must be able to manage different naming approaches and to manage "links" between different names of the same object. One approach is for the remote provider to have its object's name (in its Name System Management service) pointing to the corresponding object's name within the factory owner Name System Management Service. If the owner updates its internal ID, this will not affect any external links, provided that the "internal system" maintains links able to redirect the name resolution from the "old name" to the "new one". In this approach, "names" cannot be removed from the system unless the referred object has been removed from the system too.

3.3.2 Dealing with Address Changes

If a locator (usually IP address) changes at the factory owner, this change has to be kept transparent for the maintenance provider.

This requirement should be an improvement of the current state of the art. Often, maintenance providers have to maintain a common list of network details of their supplied equipment, even if the network is owned by the factory owner. For the maintenance provider, having a "non routable" IP address outside the private domain (i.e. the factory plant) doesn't provide any useful information. The intervention of a network administrator to set-up a connection for the external maintenance provider is often the only way to connect to the right device. Identifying the correct device is done through comparing data bank entries of both administrators with the pre-configured IP address.

3.3.3 Remote Access

Connections/Updates/Access requests should be routed automatically to the right device/service locator.

As common state of the art practice, remote connections to a device are achieved through two different approaches:

- Connecting to the device through a GPRS gateway over the public Internet using a VPN. These connections do not access the factory’s network nor require approval by the factory owner. The factory owner might specify the type of read/write access rights and ownership of the data being accessed. Those radio gateways are a major headache concerning security and network management as they essentially create a potential backdoor not controlled by a firewall or other means. IoT@Work attempts to avoid these constructs by two means: (i) by a clean approach to do remote management over the access gateways (and/or firewalls) into the factory network, and (ii) by controlling the access rights of a device and its services. So even if a backdoor would occur, the potential damage can be limited.



- Connecting to a device through the factory owner’s network, with limited and controlled access. These connections are short lived and are set-up through firewalls of the factory’s networks. These connections are routed to reach the end-device through the locator of the device or service being remotely accessed.

The above stated requirement applies mainly to the second scenario, where remote connections are enabled through the factory owner’s network. For example, when setting up a remote connection to a specific robot within Cell xx in Line yy at the Mirafiori FIAT plant, the maintenance provider should use only the known ID they manage in their directories for that specific robot. The IoT@Work “naming system” has to resolve the addresses at both the maintenance provider side and the Mirafiori plant side, and identify the gateways to be used to provide a secure connection over the public Internet. The “naming system” has also to resolve the IDs of the specific robot into an internal one. Using IPv6 addresses in addition gives the opportunity to further limit the validity or range of addresses to a scope for the factory floor, enterprise or a global scope. This can be achieved by means of IPv6 prefixes for these areas.

Updates of the IDs at either side should not affect the other side. There is the need to have some intermediate system (indirection infrastructure) within the factory’s infrastructure (or plant) so that the plant owner policy can be applied (e.g. checking integrity, virus scan, specific test suites to eventually check for unaltered, approved devices, etc.).

3.3.4 Cross-Checking Device Identity

Semantic information of a device can be used to check the identity of the device remotely.

The use of directories alone, containing locators for each device and service can be in some cases insufficient. An example of mixed identities happens when wiring several instances of the same type of device in the same sub-system (e.g., several I/O devices might be connected to the same switch). A common mistake is when a wire is plugged into the wrong I/O device leading to an error that is discovered during testing of the newly commissioned system. To avoid this, semantic information related to the location: geographic, topological, and hierarchical (line ID, Plant, Robot arm ID, etc.), should be stored alongside the locator of the device, to allow an automatic identity check. When setting up a remote connection to a device in the factory infrastructure, this extended directory information can be cross-checked automatically before setting up the remote connection.

3.4 Types of Device IDs

The following is the list of requirements on IDs for identifying hardware devices, virtual devices, or services:

• Global unique secured ID per hardware device:

o ID for the individual hardware entity.

o Assigned by the HW manufacturer and brought onto the device during the manufacturing process.

o ID could also provide information like manufacturer, type of device, integrated services, …

o Secured in such a manner that only the manufacturer can generate a genuine ID (e. g. by signing the ID with a private manufacturer key).



o Verifiable by using a public root certificate provided by the manufacturer.

o Bound to the HW so that it can be checked that the ID belongs to the certain HW, e. g. by producing a “fingerprint” over the components of the device:

� Binding may be supported by TPM (trusted platform module).

� For easy reading of the ID it may be stored in addition in RFIDs or bar code labels attached to the HW.

• Network ID (network address):

o Selectable but unique in the local network the device is installed in.

o Could be provided by the Ethernet MAC, IP address, automation protocol address or combinations of different addresses.

• Symbolic device ID:

o Symbolic name of the device that is assigned during engineering and used in the engineering process.

o The symbolic device may be a virtual device that implements a certain functionality or service.

o Symbolic devices must be mapped to a physical device during installation.

o There may be many Symbolic Devices mapped onto one physical device.

• Firmware ID:

o ID should indicate the Firmware version and revision.

o May be used for indication of licensed functionality.

o Must be adaptable after changes of firmware.

It is possible that only a subset of these IDs is directly accessible, while the remaining ones are only indirectly accessible through some database.



4 System Bootstrapping Procedures and Assumptions

This chapter describes the process of bootstrapping. This is viewed from the application point of view, then from the component point of view. In this deliverable, we concentrate on a device-centric bootstrap to simplify the process, while we make assumptions on what is already running, and the prerequisites for bootstrapping an end-device, such as:

1. The application program is defined and ready to be deployed on top of “ready-devices”.

2. The network resources are up and running, to allow a step-based bootstrap.

3. Naming and addressing schemes follow the application view, the network view has been planned, and the relevant steps to eventually configure the address assignment have been done.

4.1 Plug&Work Scenario Boards

We here take a step back and describe different scenarios covering various possible aspects of Plug&Work. The objective of the scenario descriptions is to present story boards that may be addressed by Plug&Work mechanisms.

4.1.1 Scenario 1: Software Update with minimal Impact on Production

In this scenario we assume that the devices are already connected and running as part of a distributed system. We now want to “plug” a software update on the devices to get to “work” an updated distributed system.

An example story board goes as follows:

An Enterprise Admin wants to seamlessly install an OS software update to both non-production (i.e. office, server or monitoring terminals) and production machines (PLC, I/O, Sensor/Actuators) without disturbing production. The targeted production machines also include PLCs and HMI devices running same (or embedded version of) OS as office machines.

The production machines should only receive software updates when they are “not in production” mode. The Enterprise Admin must ensure that this constraint is respected. Production mode is not determined by static time windows, but by a target machine/device property that must be evaluated on the fly. For each device this property is set on the fly in relation to the overall production system.

We assume that there is a specific management operation supporting locking/unlocking devices for maintenance actions. This operation affects the ‘state’ each device and the overall system can be in. When a device is “not in production”, it can be locked and put in a “in maintenance” state; it can be unlocked/released again, after which the production system can take the device “in production” again.

The Enterprise Admin uses an enterprise system management system (e.g. Microsoft System Center Configuration Manager) to distribute the OS software update. All the office machines and all the relevant production machines are configured to be able to receive updates from this central management system.

The Enterprise Admin formulates a management script that explicitly includes the constraint of only doing software update when not in production, and that explicitly includes the required lock/unlock operations (which are not supported by the enterprise system management system, but by a separate system). The script will be executed by an orchestration service which will trigger all required operations from



the enterprise system management system and from the separate state-based production system.

Through the central system management system, the Enterprise Admin can at any point ask for a compliance report to check how many of the machines received the update so far. The Enterprise Admin is not actively involved in installing software updates to any individual machines when appropriate (i.e. when not in production mode); this is automatically done by the orchestration service, using the management plan with the given constraints.

4.1.2 Scenario 2: Bootstrap Energy Usage Assessment Devices and Application

In this scenario we assume that new devices need to be “plugged” as part of a new or extended distributed application that one wants to deploy and get to “work”.


A Plant Operator wants to perform an in-depth energy usage assessment of a specific section of the plant. This section consists of 3 factory cells (each operated by a PLC); within each factory cell there is on average dozens of devices. To this end, 30 energy monitoring devices are to be plugged in at various locations and attached to various devices, 10 monitoring devices in each of the cells. There is also a software update that needs to be performed on each PLC in order to be able to correlate energy usage of single components to the actual process.

The Plant Operator now seamlessly wants to have the energy monitoring devices deployed, the PLCs updated, and the energy assessment to be started. We assume that the back-end energy monitoring software application can start to produce useful results for a cell, once within a cell 90% of monitoring devices are operational, and once the cell’s PLC is updated. That is, it makes sense to start looking at the numbers from that point on, and results will get more accurate as remaining 10% of monitoring devices gets plugged in or fixed.

We assume that the starting point of the whole roll-out is a design plan which tells at which locations an energy monitoring device should be added. All monitoring devices are of the same type, and the plan does not specify which specific monitoring device should be attached where. Monitoring devices are taken off the shelf, and are identified/validated when a worker attaches them to a planned location. We assume that monitoring devices in the plan are associated with abstract id’s; mapping to physical id’s is done e.g. via a directory service.

There are some dependencies and constraints that need to be addressed. We assume that adding energy monitoring devices does not interfere with the production system and can be done at any time. We furthermore assume the following actions for connecting a monitoring device:

• A worker will physically plug-in a device to a location that is included in the plan. The worker will “check in” the device out-of-band; e.g. the worker reads in physical id (e.g. barcode) and registers that with the directory service as the mapping to the abstract id as included in the plan.

• The device will automatically establish connectivity to the network, and as part of that the authenticity of the device will be validated in-band. If valid, the directory service will get updated with the device MAC.

• A successfully connected and validated device needs to be given access to the energy assessment VLAN , and be issued an IP address etc.



• Once on the VLAN, the device needs to be given the appropriate credential to be able to publish energy usage on behalf of that location. When given the application credential the device can start to publish.

Updating the PLC can only be done when not in production (see Scenario 1); we assume that there must also be already one monitoring device successfully connected in the cell, before the PLC can be updated; and we can update the PLC as soon as such one monitoring device is successfully connected. Also, before updating a PLC, the associated field devices must stop all automation steps to ensure a "save" state of the automation machinery but remain on to be reachable from the PLC which then can eventually update configurations and continue operation.

The Plant Operator wants to receive confirmation when he/she can start looking at the energy assessment, and does not want to be involved in any of the individual steps regarding the deployment of the energy monitoring devices. The deployment should be automatically done using a deployment scenario, that is written by the Plant Operator, and that captures the dependencies and constraints above. The Plant Operator can enact the deployment scenario through an orchestration service, which will schedule and conduct all management operations as appropriate, following and resolving the specified dependencies and constraints.

4.1.3 Scenario 3: Plugging in/out factory modules

In this scenario we assume that devices can be “plugged” in and out, and the system can automatically “work” in an adjusted and appropriate fashion.


A Plant Designer designs a new manufacturing line which has one or more modules that can be dynamically plugged in and out of the line. Modules on the line may behave differently when this ‘mobile’ module is plugged in or out.

Rather than hard-coding/configuring all possible behaviours in all devices, the Plant Designer makes use of a Plug&Work approach that allows deploying the right configuration at the right situation on the fly. That is, when a specific module is plugged in, other modules are immediately as possible reconfigured into the desired behaviour when that specific module is plugged in; when a specific module is plugged out, other modules are immediately as possible reconfigured into the desired behaviour when that specific module is plugged out. Other required actions beyond reconfiguration, such as notifications, starting/stopping subsystems or modules, etc., are carried out too when plugging in and out modules.

The Plant Designer captures all the required reconfigurations and other relevant actions, as part of a reconfiguration scenario, that can be orchestrated whenever modules/devices are plugged in or out the system.

4.2 Assumed Scenario Board Description

In the following paragraphs, to exemplify how a distributed service can bootstrap, we describe the bootstrapping of the Energy Monitoring Service.

The description is structures in a set of sections starting from a description of the current scenario, its drawbacks, the IoT@Work enhanced Energy Monitoring Service and, finally, how this distributed service manages its bootstrap.



4.2.1 Initial Bootstrapping Scenario

Figure 4.1 depicts a typical installation of the Energy Monitoring system. Production devices (e.g. robots) have their power supply managed via remote controllable electrical switches (like for example the Siemens SENTRON 3VL).

Plant Ethernet

PLC

PLC

Switch

Switch

Cell 1

WorkCell

Panel

PLC Switch

WorkCell

Panel

Cell n

Energy

Monitoring

��

Figure 4.1: Current usual Energy Monitoring configuration

At the cell level there are other remote controllable electrical switches that provide power to the switches within the cells. There can be further energy distribution layers within a plant (in Figure 4.1 there is one additional plant level switch that control power supply to the cells’ switches).

The power supply switches can have monitoring devices (like the Siemens PAC3100/3200) that collect power supply related data from the switches and are able both to display them locally (via their control panel), as well as to forward them to PLC and MES applications.

The following figure provides a typical energy monitoring configuration.



Figure 4.2: Siemens SENTRON configuration (source Siemens SENTRON Powermanager brochure)

It is up to the PLCs and MES applications to collect the energy measures, perform (if envisaged) some computation and provide the measures or computation outcomes to other applications or monitoring systems (see the Energy Monitoring room in Figure 4.1).

The energy monitoring applications can remotely control the power supply switches in case they discover unwanted situations for example commanding to a remote switch to switch off the power to a robot.

4.2.2 Current Drawbacks

Adding a new switch or monitoring device, as well as changing a configuration of one of them, implies revising PLC/MES configuration, often revise the kind of computation these applications have to perform.

Adding a new application/service that requires energy consumption related data has impact on the PLC/MES applications, in particular if the new application/service requires only specific subsets of the collected data (e.g. all power data related to robots of type XYZ).

It is complex and expensive to provide access to actual power requirements of specific systems to an external suppliers or maintainers, especially if those data are requested only for limited period of time.

It is complex and expensive to restrict access to power consumption data to specific subsystems, or specific information (e.g. peak power requirements) to external suppliers or maintainers.

4.2.3 IoT@Work Future Scenario

The enhanced energy monitoring system has to be flexible enough to solve the above issues still maintaining a hierarchical power distribution architecture.

Additionally, from a Plug&Work point of view, is has to be able to support new measuring devices or new monitoring applications with minimal changes and reconfiguration issues.

The following paragraphs report first how the IoT@Work ENS can be used to address the reported issues.



4.2.4 Event Collection and Notification Service

The Event Collection and Notification Service (ENS) can solve most of the above issues acting (as depicted in Figure 4.3) as a common collector and distributors of all events related to energy consumptions as measured by the monitoring devices (like the Siemens PAC3100/3200).

Event Stream

B1

Even

t St

ream

B2

Event Stream

A3

Administrative Domain Border

Ev

en

t S

tre

am

A1

Ev

en

t Stre

am

A2

Figure 4.3: ENS based Energy Monitoring configuration

As depicted in the figure, the ENS will receive the measured values from the measuring devices (see the Event Data arrows in the figure), arrange them into specific Event Streams that collects specific subsets of the collected events according to rules defined by the owner of the data.

The ENS is able to manage hierarchical namespaces. For the Energy Monitoring activity, therefore, the collected energy consumptions measures could be arranged as reported in Figure 4.4. In principle the namespace is arbitrary, but we propose to use t names that are clear for the people dealing with a namespace. But we are not suggesting to have semantics within the names. In the following discussion, we use a namespace which is purely an example for clarification.

Each monitoring device will publish its measured events under a specific leaf of the hierarchical namespace (e.g. the PAC3100 monitoring the energy requirements of robot P1C1Rob1 will publish its measures under EnMonRoot.Production Line P1.Cell P1C1.Robots.Model NG-X1. P1C1Rob1). Those events will be broadcasted by the ENS to all Event Streams having, directly or indirectly, subscribed to that topic.



Figure 4.4: Energy Monitoring Hierarchical namespace

The ENS will support not only applications/services subscription to specific levels within the hierarchy (e.g.: EnMonRoot.Production Line P1.Cell P1C1.Robots.Model NG-X1. P1C1Rob1, or EnMonRoot.Production Line Pn.Cell PnC1.Conveyors or EnMonRoot.Production Line P1), but also to filtered subsets of the collected events.

For example, suppose as external supplier is interested, and authorized, to monitor all energy related events of the new robot model Model NG-X1.

In this context the external supplier can be authorized to subscribe to an Event Stream collecting all events in the energy monitoring hierarchical space related to all Model NG-X1 robots. Its Event Stream meets therefore the following condition:

EnMonRoot.*.Model NG-X1.*

In order to support monitoring applications in obtaining not only measured values, but also information on how to interpret those measures (i.e. meta-information on the measures, the measuring device, the production process to which refer the measures, etc.) the published events can not only provide the actual measures but also other data like:

• The ID of the device to which the events are related (e.g. P1c1Rob1);

• The ID of the measuring device;

• A URI that points to a semantic description of the provided data (e.g. the kind of measured parameters, the kind of measuring device, units of measures, tolerance, …), so that a subscribing application can acquire all information to properly interpret and analyze the received events;

• A timestamp of the event.

As evident adding a new device to a plant simple requires, on the monitoring device side, to provide to it the information related to the topic under which it has to publish the event and how to connect to the ENS, while, on the subscribing applications side the changes can even not be required at all if the subscribing applications/services have an Event Stream that already embraces to the new devices.

For example an application that monitors the power requirement of the whole production line P1 (i.e. having an Event Stream with all events within Production Line P1.Mirafiori Plant Energy Monitoring) will receive those additional events and will add them to the other events without any need of revising the application.



4.2.5 Energy Management Service Bootstrapping

The Energy Management Service is a distributed service based on the following functional components:

• measuring devices: these are the feeders (events producers) of the distributed system. The bootstrapping of these devices can be described distinguishing between the device booting phase (OS bootstrap, network configuration, etc.), whose objective is to have the device connected to the other devices (network, measuring sensors, etc.) and the booting of the energy monitoring feeder (e.g. the software component in charge of the structuring of the data to be published via the ENS). The first booting phase is not relevant for the Energy Monitoring Service bootstrap (and is similar to the bootstrap of other devices). The second booting phase envisages obtaining the data to connect to the ENS (a “configuration activity” which, apart from the different set of data, is similar to the configuration activity a device has to perform for connecting to the log and audit service,, connecting to the ENS and start publishing the measures;

• the ENS middleware: these functional components are better described in the following and must pre-exist, as already highlighted;

• energy monitoring applications15: these components are the consumers of the energy measures and can be running within the same production plant or in remote facilities. They need to perform the phases described in Section 2.2 for the application level middleware. Most of these applications will be configured in such a way to be automatically started when the OS on the system on which they reside has completed its start-up; others can be started on request when needed.

As evident from the above description the proposed approach has the advantage of decupling the deployment of single components (measuring devices and monitoring applications) from the status and deployment of the other components. For example a new monitoring application can be deployed simply configuring it and granting it access to the measuring data it requires in the Energy Monitoring namespace on the ENS and providing to the new application its ENS access credentials.

4.2.6 Energy Management Service Middleware Bootstrapping

The Energy Management Service (ENS) middleware (MW) needs to have been bootstrapped and be operational before the ENS measurement devices and the ENS application are bootstrapped. The MW needs to be able to support both the routing of events from event producers to event consumers, as well as the remote invocation of services of devices and applications. Thus, it needs to support both an Object-Oriented (OO) paradigm of interaction, whereby entities make service requests to other entities and an Event-Based (EB) paradigm of interaction, whereby entities publish and subscribe to events of interest. In order to enable the interaction among different components, the interaction vocabulary needs to be fixed and be known. In the OO paradigm, this vocabulary is the set of known interfaces, which are offered by server objects and used by client objects. In the EB paradigm, the interaction vocabulary is the set of known event types that producers emit and subscribers receive. These vocabularies can be stored at an Interface Repository as in CORBA, 15

We also include under this category scripts running within Complex Events Processing Engines. The use of CEP frameworks can easily simplify the deployment of monitoring applications based on patterns recognition in data streams. From the bootstrapping point of view starting a CEP script is similar to starting an OS based monitoring application or service



from where components can read them and discover dynamically how to interact with other components, or they can be communicated to components through configuration files during bootstrapping, or they can be discovered by having each component advertise its interfaces at certain time points (e.g., when it is first connected to the network, when it identifies a new component, periodically, etc.). Along with the interaction vocabulary, the MW must keep track of the interaction points, i.e., the component ports that can accept service requests of a particular type or that can produce events of a particular type. Just like the interaction vocabulary, the points can be stored at a Naming/Trading service as in CORBA, described statically in configuration files, or discovered dynamically through advertising. Finally, the non-functional capabilities of interaction points need to be established, to identify the security policies, the real-time characteristics of services/events, the reliability characteristics of these, etc.

The MW needs to be able to support both the OO and the EB paradigm because of their different dis/advantages and the different requirements of factory automation subsystems and applications. So while EB allows for a minimum coupling of components, easy replacement of components, easy evolution of event types, and N-to-M interaction, it cannot support very hard R-T requirements. The OO interaction paradigm can support harder R-T requirements exactly because of the greater coupling of components and its 1-to-1 interaction pattern. The lower complexity of OO MW makes it easier to support hard R-T requirements with low latency and most importantly low jitter values, while at the same time making it more difficult to support reliable interaction in the presence of faults.

4.3 Engineered application process & application programming

Currently, the main steps of the application programming are realized offline, i.e., before the real automation system is actually initialized and started. Therefore, the whole procedure is distinguished in an offline phase and an online phase can be described in several sequential steps:

Offline

1. The offline device information is represented as an electronic device description (EDD) file which can have different formats, for instance GSDML. The EDD files of all involved automation devices have to be imported into the engineering system.

2. The application is programmed with respect to the IEC 61131-3 standard [32] using one of five existing languages.

3. The engineering system creates a project with configuration and parameterization.

4. The created project is downloaded to the controller (PLC).

Online

5. The controller is started up and configures all devices in the start-up phase of the system.

6. The cyclic data exchange between the PLC and field devices is started and the whole system is running.

The application programming is based on the IEC 61131-3 standard and usually realized by means of engineering software. Hence, most of the existing engineering systems and PLCs are based on this standard.



IEC 61131-3 harmonizes the way people look into industrial control by standardizing the programming interface. This includes the definition of five languages: sequential function charts (SFC), instruction list (IL), ladder diagram (LD), function block diagram (FBD) and structured text (ST). Each program is additionally structured by means of modularization and declaration of variables. This leads to an increased re-usability, a reduction of errors and a higher efficiency. In addition, IEC 61131-3 structures the configuration of control systems. A more detailed description of the engineering process and related issues can be found in section 6 of D2.1.

4.4 Network Resource Configuration and Planning

Network planning refers to the offline step of defining the topological and physical layout of networks in the factory hall. This essential step is already specified in the IEC 61918-2010 standard [33]. Often this step is carried out based on the following decision criteria:

1. Geometrical and architectural layout of cells, planned machinery and their placement in the factory.

2. The network topology and the expected network load

3. The planned end devices and their capabilities (network interfaces)

4. Application details and their needs (real-time requirements, robustness, etc.)

A detailed description of the overall design and planning process of industrial automation systems can be found in [34]. This document provides specific information about the needed design and planning steps for the industrial Ethernet standard Profinet and includes a detailed step-by-step design of a whole plant.

The topology that is designed to satisfy all above criteria is often done according to experience and practice of the network engineers. The network has to first connect all those devices that communicate directly with each other, then the dimensioning of the technology used is performed according to the expected load as this has been illustrated in Section 3.2. The design of the network architecture inside a single plant is similar to designing a local area network or a wide area network when connecting several production plants to the back-office. The network dimensioning is done according to deal with simply formulated requirements related to bandwidth and latency needed (i.e. when selecting which Ethernet or bus technology). The number of devices that might be connected to a certain cell defines the number of switches deployed in a given cell. And ring network topologies are often used to provide physical redundancy. Depending on the distances to be covered, it is also quite common to have line topologies, e.g., in applications like conveyor belts for extensive automation plants.

The design steps of a network topology is followed by an offline one detailing of both address assignment and planning;, simulative and maybe modelling of traffic load when all devices should be running. This step results into configuration details for bandwidth planning and optimisation of certain routes (upgrading the network to include higher performance switches or add another switch to fragment the communication load).

Following this connectivity and bandwidth oriented network setup, a second phase occurs to really define sub-networks and network management services like DNS, DHCP, and SNMP services. This step is based on an offline and often manual planning effort using tools that define which address space can be associated with which devices, and whether these use a manual setup (configure each device IP address manually) or using DHCP servers.



Application programming and testing (including commissioning which includes configuring the application parameters and signals to network connections and communication relationships) takes place in the last step.

The IoT@Work approach to bootstrapping will change the level of details and manual steps required at the commissioning part. Some network planning based on the above-mentioned criteria will still exist. But the configuration of both device addresses and names then transferring application details to the network will change drastically making this commissioning automated and self-organising in nature. The manual steps will also be reduced to those related to physically connecting the devices together and not on configuring protocol details at each device individually.

This setup will be based on the bootstrapping process defined in this deliverable.

4.5 Pre-requisites of Running Network Services

4.5.1 Overview

Network Services in our terminology are functionalities enabling the forwarding respective routing of data packets, managing addressing and resources or enforcing security (authentication and policy enforcement).

Today, we can distinguish a configuration and commissioning phase and the operation phase. In the first one, no automation application is running. The devices in the network will power up and form a network which has connectivity and traffic engineering or security rules can be configured on the single nodes using management or vendor specific configuration tools. The second phase then starts the applications. IoT@Work aims at moving some of the explicit configuration work of the first phase into the operational phase by means of auto-configuration mechanisms or intelligent on-demand configuration. Doing so reduces efforts for "manually" configuring a network (=tool support, but initiated and controlled by an human operator) and enables more flexibility if application demands change or faster reactions on failures.

Before the bootstrapping of the devices can access network functionality, it must be assured that the network is up and running. This means:

• Ethernet is running and provides connectivity (apart from the device in question)

• IP is running in the network to allow access to the device in question)

• The mandatory network services are running, these are address mapping services (i.e. DNS or alike), network configuration services (i.e. DHCP or Zero Conf), eventually routing and multicast services.

• eventually device configuration services must be present, i.e. a Profinet device may need an external station (i.e. PLC, management station) which sets its IP address using the Discovery and Configuration Protocol (DCP).

• if needed, other middleware services must be running (i.e. presence service)

• security services must be running and it must already be possible to find (discover) them

• all information needed to authenticate the device and to eventually configure the network or middleware service (policy control, firewall etc.) must already be there (RADIUS/LDAP...)



4.5.2 Basic Connectivity Boot Sequences

Within the Basic Connectivity boot phase, several tasks have to be accomplished:

Sequence Task pre-requisite steps result

1 Ethernet physical layer boot

adjacent station up and running

PHY auto negotiation

PHY works, bandwidth and duplex mode have been auto negotiated with neighbour(s)

2 IEEE 802 network configuration

MAC address available

secure unique ID available

- LLDP neighbour discovery

- STP or alike (switches only)

- 802.1x NAC

VLANs and forwarding rules are operational (i.e. from spanning tree protocols).

Authentication (802.1x) is completed.

3 IP address configuration

Layer 2 up and running

- preconfigured or auto-configuration, includes subnet mask for IPv4

- neighbour detection on IP layer (optional)

IP address is valid

4 Local routing configuration

Set up local routing tables, either static (end device) or via routing protocol

IP stack (L3) is operational

5 set local clock NTP/PTP(IEEE 1588) clock available

sync local time to system time

clock in sync with other systems

6 (optional) Set up local filter rules

L2/3 up and running

apply security rules and QoS policies

7 (optional) create IP - symbolic name associations

L2/3 up and running

perform dynamic DNS registration (may be pre configured)

symbolic name available and resolvable by other stations

Table 4.1 Basic Connectivity Boot Steps

All of the steps above require information from neighbour stations or some configuration service (i.e DHCP) if not statically preconfigured. Typically the results of layer 2/3 configuration can be checked in the SNMP MIB II.

We propose to use a two-stage approach for the network (Layer2/3) boot. The first stage will perform a IEEE 802.1x authentication using eventually a temporary IP address, then after authentication is completed steps 2 and 3 are repeated and the network will grant access with the device/service specific rights and resources. Details of this so-called Network Access Control procedure can be found in Section 5.2.3.

Note, step 3, the IP address configuration, may also include the configuration of higher layer information such as IP addresses for the domain name service (DNS) or other services (i.e. Windows domain controller). The reason is that those pieces of information are typically managed and made available from one place via DHCP (or PPP for dial-up and DSL links). Getting an IP address via DHCP may also be used



as a first registration of a device, i.e. the DHCP data base may be used as an IP layer presence service.

Prerequisites for this stage are a pre-configured MAC address and eventually a physical layer address (i.e. SSID of a WLAN). In the following text we will not consider media depended PHY address or configuration.

Typically the MAC address has been configured beforehand by the vendor of the respective device and hardcoded into the ROM of the interface. But it is possible to overwrite the MAC address with other values. Care must be taken here to create valid and unique MAC addresses, but it is possible to i.e. add semantics such as the topological position within the network in the address (see [40]). As basic connectivity is not possible without an Ethernet MAC address.

IoT@Work compatible devices must have an secure, unique identifier which is also pre-configured (see D2.1). Unlike current systems which often use a MAC address as initial ID, this secure ID is the root of all other IDs which will be configured. It identifies a device. Because multiple interfaces per device are possible, a per-interface configuration must use both the device ID (or something derived from it) and the MAC address.

For all other parameters (see table) off-line pre-configuration, local auto configuration or loading from central repository are possible. Section 3.3 details this topic.

4.5.3 Ethernet Start Up

Ethernet (IEEE 802.3) is discussed here as a prominent and important example of a Local Area network protocols, but many of the discussions here are also valid for Wireless LANs or any other protocol from the IEEE 802 world. Focus in this paragraph is on network functionality, not on device local issues. Note, end devices which cannot forward frames may use only a subset of the functionality described below.

First part of the start-up phase is the start of the physical layer which includes i.e. the capability detection of the PHY, e.g. the auto-negotiation of link speeds, modulations and more. In wireless media this may have a far higher complexity which we do not detail here. A PHY of a device can only start properly if at least one adjacent node is already working16, otherwise the PHY will remain in an offline mode. That is, an end device can only start its network interface if the adjacent switch (or access point) is already up. Typically, a network interface (be it on a switch or an end device) will start as soon as the PHY detects connectivity. The detection of this physical connectivity depends on the media, for example a signal wire may become active or a wireless channel is available due to activities of other devices.

After PHY has completed its bootstrapping, the Media Access Layer will start. An important aspect here is the assignments of a MAC address, which - in start-of-the art networks - is typically pre-configured. In IoT@Work this could be assigned dynamically in this phase based on semantics of the network. This however may create a chicken-egg problem: gathering semantics (i.e. the topology or authorisations) requires at least a partially working network. One way to resolve this issue is to use a pre-configured MAC address in a first step, then after using this address a new MAC can be configured.

After the MAC layer of a device works, adjacent devices on a link will start to exchange information using the Link Layer Discovery Protocol (LLDP). This is an 16

if two adjacent devices power up exactly at the same time, Ethernet will ensure a deadlock free start up.



important step as devices can learn about their neighbours and a management tool later on can access and collect the information to deduce a topology and device data base off the network. LLDP uses a multicast MAC address, so it is independent from the interface MAC address of the respective devices and thus should still work even if the interfaces do not have correct MAC addresses at this stage.

Next step after the LLDP is to set up forwarding engines for devices which use Ethernet forwarding protocols such as e.g. the Spanning Tree Protocol or much faster variants used in industrial networks. That is, the devices (typically switches) must apply the respective protocols to form a system wide cloud.

IoT@Work will logically segment a network into "slices" or commonly called Virtual LANs (VLAN) - that is a mapping of an incoming identifier to an outgoing port - forming logical paths through the network. VLAN configuration may include rules for access and resource control, i.e. bandwidth limitations for a certain MAC address. Note, there are many means to implement VLANs, most notably IEEE 802.1Q for Ethernet. At this point, we do not define a specific protocol or method to implement VLANs. But, from the bootstrapping perspective, it is important to have an initial VLAN configuration working in this early stage of the network boot.

Access control, as defined in IEEE 802.1x is the next step (see also Section 5.1.3) for end devices (not for switches within the network). Note, depending on the results of an 802.1x step, the VLAN configuration of an edge switch may be changed to reflect new rules.

4.5.4 IP Layer Start-Up

After Ethernet is operational, the IP layer will start. Even though the basic project approach follows state-of-the-art procedures, we will have to use the Standards in a clever and innovative way to ensure IoT@Work requirements are met. This mainly affects the way IP addresses are managed, created and assigned and it affects the IP routing as one important component of the resource management. IP routing can be seen as a network service which must be running before any end devices can reach basic connectivity if the network is structured into IP sub-networks. Here we take this as a given prerequisite and do not go in details as this will be a major issue in some of the next deliverables.

The IP address configuration is a major issue which must be solved before any other higher layer service can start. It can be locally pre-configured, auto-configured (calculated by some algorithm) or managed from a database (i.e. DHCP).

• pre-configured: no further network service needed, the assignment is done from a planning/commissioning tool instead. This is mainly what is done today.

• auto-configuration: the device creates its IP address itself, applying one of the algorithms specified in the relevant standards. Then it may need to perform a check for duplicates (i.e. APIPA17). Encoding of semantics into the address may need information from lower layer (see above) or from network service. Note, this only works for so-called link local-addresses which are valid only in one Ethernet domain.

• managed: pre-planned addresses are assigned by means of DHCP.

The preferred way for IoT@Work is to use DHCP to assign addresses. Prerequisite is a running DHCP service and an Ethernet capable of forwarding packets to this service. Then the start up procedure for IPv6 is:

17

Automatic Private Internet Protocol Addressing (APIPA) is an algorithm specified by Microsoft Inc. for IPv4 auto configuration.



• A client sends a Solicit message to locate servers.

• A server sends an Advertise message to indicate that it is available for DHCP service, in response to a Solicit message received from a client.

• A client sends a Request message to request configuration parameters, including IP addresses, from a specific server.

• the DHCP recognizes the device by means of the DHCP Unique Identifier (DUID) in the neighbor solicitation message. We intend to derive the DUID from the secure device ID, so planning/commissioning tools may eventually use this to properly administer the DHCP service. The server then will return the requested information by means of a "Reply" message.

• A client sends a Confirm message.

A DHCP reply can have many more configuration parameters, i.e. addresses for the Domain Name Service or the default gateway.

Note that those steps will be performed per network interface. An interface in general will have multiple IP addresses, for example a auto-configured link-local address, site-local and global addresses assigned by DHCP. We propose to use managed site-local addresses for all purposes which do not need to be globally accessible.

4.5.5 Network Supported Remote Boot

After the IP address is operational, a device may access a server (e.g. a remote repository) to load configuration or SW (such as firmware images). This can be supported by using the Dynamic Host Configuration Protocol (DHCP, RFC 2131 or RFC 3315 for DHCPv6) or the BOOTP protocol (which is a subset of DHCP). DHCP will return the addresses of those repositories. Context information coming from the steps before (i.e. point-of-attachment, device specific parameters from the secure credentials) may be used to choose proper configuration automatically if the DHCP server has been configured to reflect the planning and allows to dynamically create the repository addresses depending on context info.

Note, some HW platforms may have support for remote booting build in the BIOS, i.e. specified after the Preboot Execution Environment (PXE) from Intel Corp.18. PXE (respective the corresponding functionality in the newer Unified Extensible Firmware Interface (UEFI) is not IPv6 capable, so if it is used, an IPv4 infrastructure must be still available.

4.5.6 Network Basic Services

Several essential services are needed for the operation of the network. The most important ones include:

• Time Service(s): in order to have synchronised clocks in all networked devices, a master clock must exist in the automation network and a mechanism must be available to distribute the reference time taking delays into account. We will need both the IP-layer Network Time Protocol (NTP, RFC5905/RFC1305) and the IEEE 1588 for the Ethernet layer. Both should use a common master clock to be in sync. A time service must be available in early stages of the bootstrapping process.

• Neighbourhood and topology discovery: for network management but also to create location information for the IoT@Work plug&work and for rerouting mechanisms, it is important to detect adjacent device for a network port or to

18

http://www.pix.net/software/pxeboot/archive/pxespec.pdf



derive information of the network topology. This can be achieved by the LLDP during the Ethernet boot and periodically in the operation phase. Another mean to research network topology and state are the Ethernet “routing” protocols (STP or alike) which also start operation in very early stages during Ethernet boot. To access this info, the IP layer of the devices offering this information must be operational as the access typically will use SNMP which needs an addressable agent on each device. Further work will evaluate whether IPv6 neighbourhood discovery can complement (or at least partially) replace the LLDP based mechanisms.

• DHCP (DHCPv6) server respective DHCP relays must be available.

• A name mapping service (DNS) must be up and running before symbolic names can be resolved.

• For IPv6/IPv4 migration, eventually protocol gateways must be available. Tunnelling technologies or protocol translation between IPv4/IPv6 are not in the scope of IoT@Work.

• The pre-requisites for the ENS need to be satisfied (see below).

The ENS system, as depicted in 4.2.4, has a set of components that need to be pre-deployed. Specifically:

• The ENS PDP (Policy Decision Point) is the service in charge of managing ENS access request validation and decision. As already highlighted access control is based on the usage of capability based authorization tokens (called capabilities). In a capability based security environment this service is in charge of validating the capability in the service request, as well as the congruence of the request both against the provided access capability, as well as against additional access policies. As for an XACML PDP the outcome of the access request evaluation is an Allow or Deny;

• The ENS Revocation Service is in charge of managing capability revocations. Its work, therefore, envisages both the validation of the received capability revocations, as well as updating PDP rules and access policies accordingly;

• The Event Notification Service that is in charge of the actual collection and dispatching of the events.

Optional monitoring and/or logging services may complement the set of services supporting OSI layers 2 and 3 in their operation. It is beneficial to allow logging of the boot process per device or service already during the bootstrapping. Because logging or monitoring facilities in the network can only be reached after the IP layer is operational, the devices must have a device local buffer for the logging messages which then can be send to the remote logging facility or which can be read by the monitoring service during operation. For both logging of device/service specific events or for monitoring of certain parameters, a clock is important to allow correlation and analysis of distributed events. Thus a device must have an internal clock which is available from very early stages in the bootstrap process and which will be adjusted later on by means of the system time (NTP, IEEE 1588).



5 Device Bootstrapping – Process Description

In the term of this section, a device is assumed to be a physical thing or entity which is equipped with the following additionally components:

• a memory,

• an operating logic, and

• some kind of communication interface.

These restrictions ease the integration and interaction of a device with the IoT and its components. The simplest (and most stupid) kind of device that satisfies these three conditions is a pure thing/entity with an RFID tag attached to it. Such a device returns a constant data unit – mostly its ID – upon an external request. An RFID extension helps identifying and also linking the latter ID to the location of the device. However, devices are often more complex regarding the memory capacity, the functionality, and the communication or regarding a composition of particular devices into a compound structure.

The bootstrap of a device is both an internal and external process. The focus in this deliverable is on the external process that allows the device to be identified, authenticated, and finally integrated into the IoT system. Once the bootstrap is finished, the device is a fully accepted and operating component or subsystem of the IoT which provides its functions and/or services to its users.

When talking about bootstrapping, two different situations may occur. Firstly, the device performs its very first bootstrap or performs a bootstrap in a new context. Assume there is a “drive-motor” which needs to be attached to an assembly line. It may be brand-new and come directly from its vendor and has never worked before or it may come from a material store and has been used before in some other context. In any case, this drive – new or used – needs to integrate into its new environment and will have to go through a bootstrapping procedure. The second situation refers to a device which has been at its place and in its function before but has been offline or in a suspend mode for some time. Then it needs to re-join into its context. In the following, we refer to the first situation since it is the more general one which includes the second one as a subset.

5.1 Bootstrapping Functional Components

In Section 2.2 (pag. 17) we provided a classification of the bootstrapping functional components distinguishing between classes dealing with devices bootstrapping (there reported under the device level middleware) and applications bootstrapping (collected as application level middleware). Each of the functional components provides features partially described in that chapter. In the following we provide more details on the most relevant and crucial components involved in the bootstrapping process.



Figure 5.1 The First Functional Components for a Plug&Work Architecture

5.1.1 Device Directories

5.1.3.1 Scope of the Device Directory

The Device Directory (DD) is a data management system used by the IoT for discovering and managing descriptions and presence (static and dynamic context), of devices things or entities. The DD is used during the whole lifetime of the IoT but gains special importance during the devices' bootstrap phase. Here, the term 'bootstrap' is used equivalently to the term 'plug& work'.

The DD is a software component for collecting and storing device data descriptions that is created during the bootstrap phase for each device connected to the IoT. Such a device descriptor is an entry of the DD that reflects the discovered by the network. This make the DD a mandatory component of the IoT infrastructure since it may be used to retrieve device descriptors, to select one, and to address the related device.

The DD provides two main function clusters:

• Support of device presence management

• Support of device utilization

The device management cluster comprises functions for

• Device registration

• Device de-registration

• Device data modification, which in fact refers to an update of device descriptor in the DD caused to a change of the devices state.

• Description of authentication methods and related policies.

Registration is the initial entry of a device descriptor in the DD. Modification is a change of the device descriptor caused by a change of the device descriptive state



(context information). Deregistration means either removing the device entry from the DD or at least designating the device entry data as removed. Authentication support is a part of the registration or bootstrap process. There may be further functions for managing devices which are not in the focus of this deliverable.

The device utilization support comprises the following functions:

• Device discovery

• Device look-up

• Device subscription

According to discussions taking place within the IOT-A project regarding the Resolution Infrastructure, the device discovery is a service for users who want to explore devices in a certain context based on a rough specification of the desired result. The difference in IoT@Work is that the devices might be already known to the users, to allow checking the inventory (resulting from the bootstrap) against the planned and as a online data base for management or other purposes mentioned above. An example for a device discovery is the call like: "find all devices of type T within radius R of place X which are currently in suspend mode". This should not be confused with discovery protocols, which might be used to discover the presence of the device and automatically register the device in the DD.

In contrast, the device look-up refers to addressing a device based on a unique identifier or key. The look-up result enables the user to access the device or apply its related service. So the result may be an IP address, a MAC address or a URI. The term address resolution is sometime used instead of look-up since a name or key is resolved to an address which may be a network address or some sort of locator.

According to the subdivision of the DD functions into two sets for device management support and device utilization support, the DD has two kinds of users:

• The device or device owner applies the device management support functions primarily.

• Any user or service that needs to address a device or its service will mainly work with the device utilization support functions.

The device owners are responsible for their devices. There could be a user interface for device registration, modification, etc. but having in mind the large number of devices, an individual management by the owners appears impossible. Instead, the devices act on behalf of their owners. Consequently, the devices will have to take care of their management by themselves as already addressed in the self-* approaches such as that in [18]. This is particularly true for the processes of the bootstrap and authentication phase where user involvement and interaction is required to be minimal.

A similar idea holds for the device utilization support functions. They are usually applied by services, control components, or other devices – in the following referred to as users – rather than directly by humans.

With this in mind, players in the bootstrap process are the device, the DD, and the bootstrapping basic services, which include device authentication, naming and addressing, and context collection services, see Figure 5.2 below. The user (in this case an application or process) will benefit from the DD and the registration of devices as the may use the retrieval functions which helps them keeping the IoT services and systems running.



User

Security

Functional

Components

Device Device Directory

Device

Addressing

Service

Device

Context

Service

Figure 5.2: The Device Directory Actors

5.1.3.2 Requirements Aspects for the DD

From the description of the Scope of the DD, the basic requirements for bootstrapping devices are:

• Minimum interaction of the device owner

• Minimum pre-configuration of the device

These two rather contradicting requirements need to be satisfied by an acceptable compromise. Here, the solution is to have no interaction by the device owner and instead a simple pre-installed standardized start-up behavior on the device. This behavior must enable the device, when plugged to any place in the IoT network infrastructure and switched on, to find a service to apply for registration, given that the device can be authenticated, provided a unique ID, and allowed to interact with other management services.

In principle the DD can provide a contact point for new devices that indicates the sequence of procedures triggered during the bootstrap. This can be inspired from discovery protocols and registries like DHCP and DNS, where both are seen as possible implementations of the two functionalities. The DD components have to fulfil certain requirements which are listed below:

• always on: whenever a device starts bootstrapping, the DD is ready to provide the necessary functions

• highly reliable: as a consequence of the always-on requirement, the robustness of the DD is the main preconditions for flexible growth and change of the IoT

• ubiquitously accessible: wherever the device starts bootstrapping, the DD is present to accept the device's registration request

• highly scalable: thousands of devices might need to be managed by the DD



According to the requirements and tasks of the DD, it will consist of three sub-components.

• DD Guide: It helps devices the find the DD registration interface and includes pointers to other bootstrapping components (e.g. address of the authentication server, local DHCP server, local EDD entry point). The guide can also allow the owner of the system to define device classes and bootstrap functions for each device class.

• DD Cache: It stores the device descriptors during the registration phase and keeps track of device bootstrapping phases (e.g. authentication step)

• DD Data Store: It stores the descriptors of authenticated devices and enables users to retrieve them.

This decomposition is depicted in Figure 5.3 below.

UserDevice

Device Directory

DD Guide

DD Cache

DD Data Store

Security

Functional

Components

Device

Addressing

Service

Device

Context

Service

Figure 5.3: The Device Directory Reference Model

It is worthwhile to mention that the basic bootstrapping services are not really a part of the DD. It is used just as one services of the IoT Infrastructure and may be applied by any other component, too.

5.1.2 Naming and Addressing Components

The innovation in IoT@Work is not about creating a new coding scheme for both naming and addressing functionalities, but rather supporting plug and work scenarios by using auto-configuration for both assigning identifiers (names) and addresses (locators). Naming and addressing relies on the following functional components:

1. Device naming and unique identification

2. Addressing link and network layers

3. Device location and context collection



The coding technology for a device identity might depend on the type of device or object that is involved. In the bootstrapping methods specific for the energy measuring device, we are considering an IoT entity with embedded intelligence and therefore the device operates an IP stack and naming mechanism similar to DNS.

The addressing of lower layers can be defined according to each class of devices or entities differently. The options that exist for IP-enabled devices have been discussed in Section 4.5. Whereas in this project we do not attempt to pursue inventing new protocols, the use of existing technology is interlinked to the controlled bootstrapping process defined in this deliverables.

While the main functionalities are protocol independent, the use of IPv6 has some specialities. IPv6 supports network prefixes having link-local, site-local or global scope. Link-local refers to an Ethernet domain, packets with those destination addresses will not be routed. Thus link-local addresses can be used in cases where a device shall only communicate with devices or services in this local scope. If devices/services could potentially be accessed from anywhere in the factory or enterprise network, site-local addresses can be used. Global routable addresses should only be used if it is really needed and security means (i.e. firewalls) are configured to control access.

The 64 bit host portion of a IPv6 address is called Interface Identifier and can be pre-allocated or auto-configured. There are two means of auto-configuration: Stateless and statefull. Stateless means the device (actually the interface on a device) forms an address on its own. It is for example possible to map the Ethernet MAC address to the Interface Identifier, but it is also possible to use a random number or a unique device ID (i.e. a Device UID conforming to RFC 3315). For IoT@Work devices having a secure device ID, it might be possible to derive an Interface Identifier from this secure ID. The second possibility for auto-configuration is to use DHCPv6 in a similar manner as with.IPv4. Note, DHCPv6 may also be needed for managed pre-configured interface to inform about the addresses of name or time servers.

For neighbourhood discovery and router discovery (standard gateway in IPv4), IPv6 uses ICMP messages rather than ARP (ARP is no longer used in IPv6). Those discovery messages use IPv6 multicast addresses; the use of IPv6 multicasts can also be used for other discovery purposes avoiding unwanted broadcasts. Note, because of the high importance of ICMP in IPv6, ICMP shall not be blocked or disabled in the network.

The discussions above show that topological information is an important context. Topological information is a graph describing the network, the most important information for the addressing issues discussed here describes to which adjacent nodes a device has connectivity. Adjacent means, without intermediate nodes in between. And topology always reflects the logical or physical structure on one specific layer. That is, the IP layer topology may differ from the layer-2 (Ethernet) or from the structure of a higher layer message bus. For our purpose, topology should reflect the physical structure as close as possible in order to identify and distinguish devices attached to different points in the network. This means that we need a physical layer neighbourhood discovery which can be achieved by e.g. LLDP. To get the complete graph of a network, typically a management station collects those pieces of information from all devices at different layers and then combines them into a graph [31]. Getting the whole graph is a time consuming process, but getting the neighbourhood information is simple and can be done in the bootstrapping process.

5.1.3 Device Security and Context Functional Components

The following description gives an overview of functional components, from the security view, which are necessary for bootstrapping or in the context of



bootstrapping. There will be more security components, e. g. for secure communication that will not be considered here.

5.1.3.1 Authentication Function

Authentication of devices, servers and services is an important step in the bootstrapping process. Depending on the phase of bootstrapping different entities have to be authenticated:

o Device Authentication Device Authentication is the first security action after connecting a device to a network. The device has to prove its identity and eventually more properties to the already running network. Successful authentication is a prerequisite to proceed in device bootstrapping. Prerequisite for authentication is a secure device ID.

o Service Authentication In a later step in the bootstrapping process, services have to authenticate towards the authentication infrastructure. The authentication credentials are not bound to the secure device ID but rather are bound to the specific service. Service Authentication is not limited to the bootstrapping phase only and is also required in the operational phases to guarantee authenticated service access.

5.1.3.2 Location and Context Function

Device semantic discovery service: should include a process to discover the different descriptions of a device and providing either pointers from the DD server to that information or making a standard and short description in an XML-like dialect to describe each single device according to its type, possible role assigned by the manufacturer and its functions, configuration interface, or services offered. This type of information can be coded through an EDD or an EDF description for automation device offering the possibility to provide a device description. This information constitutes the static part of the context data stored (or pointed at) for each device entry. Network devices like switches and routers, can provide this information through a SNMP MIB. Whereas end-devices like sensors, actuators, and controllers, are described according to the standard used for this. In this project the assumption is that we can access this information of legacy standards, and translate it to a standard XML-like description.

Additional information that may be necessary in automation environments is location of the devices. The access rights and the granting of access may be dependent on the location of access. Therefore a location service will be necessary. Location may be specified using different parameters, e. g.

o Switch and switch port the device is attached to

o Neighbours of the device

o GPS coordinates if available

o WLAN “coordinates” if available

If the location is identified the information stored in the device or elsewhere must be protected against intentional or unintentional manipulation.

5.1.3.3 Integrity and Authentication of data

Depending on the used protocols crypto mechanisms to provide data integrity for the bootstrapped data are necessary



o Message/Datagram Integrity, if connectionless protocols are used (e. g. UDP)

o Connection (Session) Integrity if session based protocols are used, (e. g. TCP).

5.1.3.4 Data Confidentiality

Depending on the used protocols crypto mechanisms to provide data confidentiality for the bootstrapped data or parts of the data (e. g. keys for further security mechanisms) are necessary.

o Message/Datagram confidentiality

o Connection confidentiality (secure channel)

5.1.3.5 Secure (Device) Inventory

Secure generation and provisioning of system inventories is the base for services like system integrity and anti counterfeiting of devices. The Secure Inventory of the current system has to be updated as part of the NAC process of devices. After successful NAC the inventory is updated with the assessed device. The inventory must be protected against manipulation and is prerequisite for additional security services like:

o Secure management of network and system

o Support for anti counterfeiting

o Support for maintenance

The device inventory is a functional block that could be part of the device directory. It is defined here as part of the security-centred functional blocks, whereas a final implementation of all components of the DD will be defined at a later stage of the project.

5.1.3.6 Pre connect Network Access Control (NAC)

The assessment of devices’ compliance with defined policies.is a prerequisite to start bootstrapping. In a first approach NAC may only check authenticity. Improved approaches may also test for OS versions, firmware/software versions, certain capabilities required by the devices, resource consumption and availability, etc. For realisation of NAC additional requirements will arise like:

o Existence of a NAC manager to control NAC and actions depending on NAC result

o 802.1X capable industrial devices and switches (for authentication of the device prior to access to the network)

o SNMP and 802.1q (VLAN) supporting industrial devices and switches to manage devices and network depending on the NAC result.

5.2 Plug&Work Device Bootstrapping Process Description

This section describes the chronological process of bootstrapping explaining the dependencies between the above functional components. This is done by describing several steps, which from the point of view of IoT@Work will result in a full knowledge about the connected device and its context. We also mention the state reached (or result) of each step clearly. It is worthwhile keeping in mind that the scope of this deliverable has been to define the functional components needed for bootstrapping and not choosing the technology or defining an implementation of



these functional blocks. If technologies or protocols are mentioned, they represent a possible implementation, which needs to be compared with the final requirements in the next phase of the project.

5.2.1 PHASE I: Establishing Basic Connectivity

Figure 5.4: Sequence Chart of Bootstrapping a single Device

1) Register in Device Directory (Device Static and Dynamic Context collection)

The device sends its initial message as a zero-hop broadcast in order not to flood the whole network. Thus, the implantation of the DD Guide is located in each network segment on order to catch all initial messages of devices. DD Guide hosts may be switches, routers, or any other computing equipment. DD Guide: catches the message, extracts its content and analyses it regarding the capabilities of the device.

Result of step (1): the bootstrap process can start guided by the DD.

2) Reading/Using Hard-coded ID: a pre-requisite for secure bootstrapping is the availability of a secure HW ID placed in the device itself through an incorporated memory (e.g. TPM) by the vendor. This ID is cryptographic secured and can be read or transmitted as soon as the device is turned on and the stack is ready.

Result of step (2): world-wide unique and secure HW ID is available for authentication purpose

3) Device authentication (at physical & link layers):

Based on the HW ID the device is authenticated on the link layer by 802.1x. After successful authentication the device has access to the operational network



domain. In case the authentication was not successful either the network access is not granted or the device gets access to a specific limited network subnet only.

After the device is authenticated it may be checked additionally if the device is compliant to the system’s security policies by a NAC like mechanism. If the device is not compliant to the policies either the network access is not granted or the device gets access to a specific limited network subnet only.

Result of step (3): permit to use the Ethernet

4) Neighbour detection

using, LLDP, IS-IS, or IEEE 802.1q, which are protocols running at link layer that allow locating the network leaf at which the device is connected network topology..

Result of step (4): topology context is discovered

5) Device Addressing

There are different ways to configure device addresses: Case (1) can we use a random address for each device? Case (2) or do we need a structured address for identifying the network location (e.g. topology sequence, neighbourhood, sub-net)

• If case (1), the random address needs to be mapped to a symbolic domain name (locating name) that is stored in a DNS-like device directory (or using URLs). This address allows the device to have basic connectivity and allow it to be contacted by any other element. It requires the address to be reported in the device directory as (Device Name, Address) tupple

• If case (2), the address space is structured to automatically to reflect topology information, neighbourhood relationship or subnet relationship. The address can encode context of the device and could e.g. allow devices of the same type, or those belonging to the same subnet to share the same prefix. The address is allocated through an algorithm that calculates a logical address only valid for the position within the system of a given device. As soon as the device is connected to another port or another subnet, the address needs to be revoked.

Result of step (5): IP Stack operational, higher layer naming (at least) prepared

Involvement of the DD in coordinating Steps 1 till 5:

• Option 1: the device cannot download and run software dynamically � it need some representative/agent/proxy that does the job instead.

1. DD Guide: forwards device data to DD Cache

2. DD Cache: executes bootstrap protocol as a representative of the device with:

a. Authentication Server

b. Device Addressing Service

c. Device Context Service

d. …



3. When successful: DD Cache: sends result data (IP address, context, etc.) to device

4. Device: assumes data, and sends an acknowledgement to DD Cache

5. DD Cache copies device descriptor to DD Data Store, when successful: sends acknowledgement to device

• Option 2: the device is JVM-/ OSGi-enabled or can download and run software using any other mechanism

1. DD Guide: sends bootstrap protocol client and further data to device

2. Device: executes bootstrap client protocol with:

a. Authentication Server

b. Device Addressing Service

c. Device Context Service

d. …

3. When successful: device assumes new data (IP address, context, etc.)

4. Device: registers at DD according to the bootstrap client protocol

5. DD: copies device data into DD Data Store, when successful: sends an acknowledgement to device

Both options should result in the state: device is registered and ready for application pairing through its context

5.3 Bootstrapping Higher Layers

5.3.1 Phase II: Pairing "Engineered ID" vs. "Self-Configured ID”"

In current engineering tools a symbolic name (ID) is pre-planned in a CAD (Computer Aided Design) environment with each pre-commissioned device. The step of commissioning the real-device is targeted by IoT@Work bootstrapping architecture. The knowledge of device neighbours, associated context, role, etc, should be linked with the virtual pre-planned ID. Similarly, the optimal result of the bootstrapping phase, is a similar knowledge about the device generated and stored in the DD. The correlation between the pre-planned device and the commissioned one should generate a list of manual checks to guarantee the result of the comparison. This then leads to further configurations of the device based on its virtual and planned ID.

5.3.2 Phase III: Instantiating Pre-Engineered Application Details

In order to explain the step above, we use the example of the energy measuring device, which we introduced in Section 4.2. We only refer to the end of the instantiation of the energy measuring application that configures the types of events that a measuring device has to provide. On the source side, i.e. the power measuring devices, they need to have "credentials" to connect to the Events Notification Service, as well as to be able to provide "tags" that identify the published measures. In Section 4.2.6 we mention that each event has the following info: the ID of the device to which the events are related (e.g. P1c1Rob1), the ID of the measuring device, a URI that points to a semantic description of the provided data (and, perhaps an additional URI that points to a semantic description of the monitored device), a timestamp of the event. Perhaps some of these data can be recovered



from other sources; for example if the measuring device ID is univocally associated to a device (e.g. robot P1c1Rob1) and to a URI that provides the semantic description, than we can avoid publishing those data with each event and having to provide these data to the measuring device in its bootstrap phase. In any case the data needed for the ENS can be passed via configuration files or via the network; it depends on how the device, or better its app, is arranged and on the kind of network configuration service we plan to use! On the access authorization to the ENS (to publish events) a device possibly needs to have an authorization capability (it's an XML file, we can even arrange the things providing a reference to the XML file

instead of the real file) and the possibility to digitally sign the access request.



6 Configuration Engineering and Bootstrapping

This chapter considers the final stages of bootstrapping, where the network and middleware layers have already been setup and the system applications need to be initialised in their turn. It starts by briefly describing the main bootstrapping requirements of system applications, then considers different approaches for achieving this bootstrapping, and finishes by identifying the future steps and experiments for the IoT@Work project.

6.1 Factory Automation Applications

Despite the complexity involved in it, bootstrapping the network and middleware layers is only the first part of initialising/re-configuring a factory automation system. In fact, in many aspects it is the easiest part of the overall bootstrapping task. This is because it is mainly a generic task – it can be performed in a similar fashion in different systems, as it only needs to establish a more or less standard, basic functionality. On the other hand, the bootstrapping of applications requires to consider the specific goals of the overall system, identify the resource requirements that are implied by these goals, and then orchestrate the available system resources in order to meet these requirements. Given the complexity of the task, abstraction plays a major role, both in identifying the resources and in expressing the requirements. This introduces the problem of how to map abstract notions into more concrete ones, e.g., how to translate application process deadlines into thread deadlines at different computer systems and the interconnecting network infrastructure. To better understand the problems, we will sketch here a simple model of an application subsystem (since an application system is itself a subsystem of the factory automation system).

An application subsystem deals with a set of input and output plant signals, which have specific types and value ranges. These plant signals represent physical quantities, as read by sensors and updated by actuators, e.g., distances, velocities, angles, etc. Application engineers use them to specify how the application needs to respond to changes of its input signals by changing its output ones. Along with their type and value range, these signals are also characterized by their non-functional properties, e.g., how fast inputs can change, how quickly outputs need to be updated, whether there are any limits at changes such as imposing a maximum acceleration, what the level of signal noise can be, etc. While some of the plant signals relate to the factory inputs and outputs, others relate to internal factory devices, e.g., robots at some cell, etc. Thus part of the information characterizing the plant signals is the factory architecture, i.e., the set of existing devices and how these are interconnected.

An application subsystem is developed in order to achieve specific goals at the production, safety, or monitoring domains. As such, the concepts it deals with are (or should be) mostly close to the domain and not to implementation ones such as IP addresses, etc. Thus, the first problem that arises is how to map the higher-level, domain concepts (plant signals) to lower-level, infrastructure ones. This problem becomes even more difficult since a lot of these mappings need to be distributed ones. For example, a communication channel between two physical entities in a factory will usually have to be mapped into a number of different network links, some of which are parallel to each other for extra robustness. Thus the domain-level QoS requirements will need to be distributed to the different infrastructure components and these components will need to be orchestrated appropriately in order to meet the domain requirements. Given the difficulties involved with distributed control, due to the existence of faults, and the fact that no component can always have full knowledge of the global system state, system design is a particularly difficult task. It



becomes more difficult by the fact that the currently available mechanisms at infrastructure components for specifying and guaranteeing QoS requirements is far from being standardised. One of the reasons for this is that most infrastructure components have been developed for general-purpose computing systems, where the main QoS guarantee is that of “best-effort”. In other words, in many cases there is no way to control these components effectively, as is the case with OPC, Web-Services, or classic operating systems. Specialized components like R-T CORBA, R-T DDS, R-T OS, etc. are usually employed in an ad-hoc manner. Indeed, in many cases their interfaces are not fully standardised, e.g., the number of process priorities available, whether they support multiple processes, multiple users, etc., partially so as to allow the use of a multitude of different off-the-self components, e.g., different OSs. At the same time, these R-T infrastructure systems have been developed from a computing viewpoint, which views systems as being completely discreet. On the other hand, factory automation systems have to deal with and control physical processes, which are of a hybrid nature mainly, i.e., combine discreet states with continuous functions. Many such hybrid systems are un-decidable and even those that are decidable sometimes prove to be far more complicated than what our current formal specification methods19 can realistically analyse. In order to render complicated/un-decidable systems to ones that can be formally analysed, one needs to apply strong abstractions on them, e.g., approximating non-linear functions with linear ones, discretizing continuous functions, etc. This essentially produces systems that are over-provisioned and where the control logic is over-pessimistic. As a consequence of this, it becomes more difficult to quickly adapt systems, either due to changed requirements or due to faults, since it is difficult to identify quickly the new set of constraints that will still keep the overall system in a set of safe states and at the same time achieve the production goals themselves20. In many cases, the inability to easily characterize certain properties of the system arises from advancements in computing. For example, modern processors employ a number of speculative mechanisms in order to reduce the average processing time required for computations, which are difficult to characterize. In some cases, these mechanisms have as a side effect the increase of the worst-case behaviour of the system. For example, it is not unusual for embedded designers to disable the cache memory of their computing systems, since it can increase the worst-case time needed – a dirty cache needs to be flushed to RAM before allowing new data to be loaded in, thus making a data read as costly as a data write and a data read. The decrease in manufacturing scales is another complicating factor – it leads to an increase in computation faults (some of which are transient), especially given the harsh conditions that are present in factories. Thus there is an increase in the types of byzantine fault behaviours a system designer must guard against, which further complicates the control logic.

6.2 How to Engineer and Trigger Bootstrapping

From the discussion above it becomes obvious that the overall architectural design of a factory needs to be performed at design time. At that point plant signals need to be identified and mapped to specific infrastructure resources and associated constraints. For example, the water_pressure signal may need to be mapped to the signals of a set of sensors and the constraint that at least N of them are operating correctly, so as to be able to identify faulty ones. In order for the lower layers to be able to bootstrap themselves, they need to have access to the architecture, the signal mappings and their constraints. At the same time, the application control logic has identified the 19

That is, logic-based ones.

20 The safest system is one that does nothing – unfortunately that is not a very useful system.



non-functional characteristics of these signals, e.g., their period, jitter bounds, etc. These need to be provided to the lower layers as well, so that appropriate resource reservations can be made, e.g., divide network bandwidth into separate virtual links, with different QoS characteristics for each. Currently most of this information is defined statically at design time and systems can only support limited changes that have already been considered at design-time. This is usually done by characterizing these changed system settings as different operational modes, and providing different control logic and QoS requirements for each of them. When the system changes to a setup that has not been identified at design-time, then the system is considered to be faulty and maintenance actions must be performed to bring it back to one of its modes. To avoid the high cost associated with maintenance, architectural events relating to the availability and unavailability of system components, e.g., the loss of the N+3 water pressure sensor, need to be communicated to the system maintainers so that they can take preventive action early on, to keep the system running.

One trend is for this high-level information (architecture, signals, QoS characterization, etc.) to be described in a more structured and standardized manner, following the Model-Driven Architecture (MDA) paradigm. This is part of the overall approach towards the virtual planning of factories, which has been a major trend in the automation domain over the last years. By using simulated factories in early development phases, system integrators and factory operators want to (i) minimize design and construction times, (ii) reduce the number of errors in the factory and (iii) optimize factory designs and configurations. This could enable more complex factories, faster processes, and more integrated production chains, i.e. chains comprising more steps of the production and value addition process. So far, research in virtual factories has mainly focused on the plant and machine factory construction aspect. But more complex factories also lead to more complex automation systems. In the following subsections a concept, a first prototype for the modelling approach and a corresponding tool chain for an automation-centred virtual design and testing process will be described.

6.2.1 Main Concepts of Model-based Development

In the field of embedded software engineering, several projects and standards have addressed model-based software development on the system-level. Examples come (i) from the field of software process quality assessment (e.g.CMMI), (ii) from the field of safety critical system (e.g. IEC 61508), from the field of automotive software engineering, or (iv ) more and more form the field of automation, e.g. in the IEC 61499 standard [35], in the IEC 61131 standard [32], or from several research projects such as Socrades [36], etc. All these approaches are based on the same principles. In the following, these principles are sketched and differences to the approach presented here are outlined.

Explicit System and Software Models: For some years now, the introduction of explicit system and software architecture models has been one of the main trends in the field of embedded software development; this also holds true for the automation domain, e.g. in [36]. The presented approach is also based on such models.

Separation of Concerns: An overall system model comprises different concerns such as the logical software model, the hardware topology, the network description, and the plant (or factory) model. Separation of concerns means that these aspects are modelled separately; only in the final modelling step the models are mapped onto each other—creating the overall system model. Separation of concerns leads to a higher level of model reuse and it eases the modelling since users regard (i.e. model) one aspect after the other. Our modelling approach is inspired by Object Management Group’s (OMG’s) Model-Driven Architecture (MDA) paradigm (see [37]



for details): In a first step, a logical architecture comprising only software modules and plant models is created—MDA’s Platform Independent Model (PIM). Later on, this logical architecture is mapped onto a specific hardware topology, creating the system architecture—MDA’s Platform Specific Model (PSM).

Abstraction: Abstraction is the reduction of systems onto specific features which are usually accessed via standardized interfaces. Projects such as Socrades [36] and standards such as OPC-UA (see [38] for details) have defined abstraction layers for PLCs. These projects use a software-centric methodology: Different software components agree on common interfaces or signals. Here, software and plant models are abstracted via plant signals, i.e. signals whose semantics are defined by references to the plant. With such a plant-centric methodology, changes to the automation system become easier.

Closed-Loop Offline Simulation: System and especially software systems are simulated and tested in a closed-loop simulation by simulating both software and plant models. Usually a standard PC is used for an offline simulation, i.e. the simulation is done offline and without real PLCs. Here, all models are executable, i.e. can be simulated. Using this method, open-loop simulations of single software components, closed-loop simulations of software architectures, and simulations of whole systems are supported.

6.2.1.1 Platform Independent Model

As can also be seen in Figure 6.1, Platform Independent Models (PIMs) comprise software components (i.e. the software architecture) and plant models. Software components are control algorithms, diagnosis software algorithms, or monitoring software modules. These software components do not communicate directly with each other or with I/O drivers but via so-called plant signals. Plant signals are variables whose semantics are defined by the plant model, e.g. in Figure 6.1 software component 1 sets the robot’s ”angle1” signal or start the conveyer belt by a ”start” signal.

Figure 6.1: Platform independent model for an automation system

Please note that plant signals usually correspond to (physical) states of the real plant, i.e. unlike software interfaces, they exist in reality and do therefore not change when the software architecture is modified. This increases the level of software reuse, since software modules do not have to refer anymore to software-specific interfaces. Since we only consider automation systems, in most cases it is sufficient to restrict to the plant signals.

Plant models are connected via the same set of plant signals. So plant signals are used to (i) connect software components to each other, (ii) connect plant models to each other, and (iii) connect software components and plant models. I.e. plant



signals form an abstraction layer between software and plant models, the plant signal abstraction layer. E.g. in Figure 6.1, the robot plant model is connected to software component 1 via the ”Robot.Angle1” signal.

To model PIMs, a Domain Specific Language (DSL, see also [39]) is used; this allows for a textual creation of system (PIM) models. This DSL mainly comprises the following top-level language elements:

Plant Signals: A hierarchical model of all important plant signals, i.e. sensor and actuator values. Signals have a certain type and value ranges. Furthermore, constraints between plant signals are modelled.

Plant Model: A hierarchical model of the plant or process, where plants are modelled using established modelling tools such as Simulink. Plant models offer and require signals.

Software Architecture Model: A hierarchical model of the (logical) software architecture. Software components offer and require signals. Please note that, unlike in Standards such as SysML or AUTOSAR, software components are not connected directly–this is done via the plant signals.

Connections: Connections are used to connect (i) signals offered or required by software components to plant signals and (ii) signals offered or required by plant models to plant signals.

6.2.1.2 Platform Specific Model

Platform Independent Models (PIMs) have the advantage that they can be reused for a large variety of hardware topologies (i.e. platforms). For this, the PIMs are mapped in a second step onto a specific hardware topology, creating a Platform Specific Model. Many features of the PSM can be derived automatically, e.g. the schedule for real time communication networks such as I/O driver configurations or Profinet configuration. This automatism allows developers to focus on the most difficult part of the development process: The development of the applications.

Such a PSM for the PIM from above can be seen in Figure 6.2 Software component 1 has been mapped onto PLC 1, component 2 onto PLC 2. Now, unlike in Figure 6.1, the signal to start the conveyor belt must be transmitted via the bus while the robot control signal can be transmitted directly to the robot. Please note that the same PIM can be mapped onto a different hardware topology, i.e. this approach leads to higher level of model reuse.

Figure 6.2: Platform specific model for an automation system

To model PSMs, again a Domain Specific Language (DSL) is used; PSMs are defined by mapping PIMs onto hardware topologies. This DSL mainly comprises the following top-level language elements:



(i) Platform Independent Models (PIMs): A reference to a PIM from above.

(ii) Hardware Topology: A model of the hardware topology including PLCs, networks, I/O devices.

(iii) Mapping: The mapping comprises a mapping of software components to PLCs, a mapping of plant signals to I/O device ports, and a mapping of plant signals to network signals.

6.3 Future steps towards a plug and work bootstrapping in a modular manufacturing system

Currently the bootstrapping on the application layer requires a PLC application which is rather static. It allows adaptations of the process, but every possible change in the system has to be planned at design time. The Lemgoer Model Fabric (LMF) is chosen to serve as a realistic example application for an agile manufacturing process in IoT@Work and will be used for validation and testing purposes of project results. Therefore, the current problems of a flexible bootstrapping on the application layer are highlighted and a possible future bootstrapping procedure is discussed in this chapter based on the LMF example.

6.3.1 A realistic case study: the Lemgoer Model Fabric

Within IoT@Work the Lemgoer Model Fabric (LMF) will be used as one of the pilot scenarios for showing project results. The setup of the LMF is shown in Figure 6.3, more details about the architecture of the LMF are provided in appendix B of D1.1. The LMF is designed in a very modular structure, i.e., it consists of several modules with different tasks. Hence, it can be considered as a classical example for an agile manufacturing process which has the ability to be frequently changed due to different production needs or other reasons by just adding or removing modules.

Figure 6.3: Real setup of the Lemgoer Model Fabric (LMF)

In the context of IoT@Work the emphasis will be mainly put on two modules which represent the last two production cells within the LMF. One of them is moveable and can be added and removed during runtime. This flexbile module is responsible for popcorn production and finally filling the product. This means, two different scenarios can be demonstrated, either the flexible module is connected or it is disconnected. In the first case, the previous cell automatically empties the bottles that contain the corn. The raw material is hand over to the flexible production module and further processed. In case of a disconnected module, the full bottles which arrive will be sent to a storage area until they are processed.



Figure 6.4: Agile manufacturing scenario

A further generalization of the above described agile manufacturing scenario is shown in Figure 6.4. It shows an agile manufacturing scenario with an arbitrary number of manufacturing steps 1 … n. In its normal state, the product is stored in a certain condition after manufacturing step n. By being able to add different additional manufacturing steps A, B, … , X, the final product can be altered in a very flexible way by just adding a corresponding additional module. After it has been processed by the additional module the product is stored after manufacturing step n+1.

6.3.2 Future implementations and extensions

Since currently available solutions do not support such agile processes, it has been implemented as follows. States for both cases, an included or excluded last manufacturing cell, are planned and programmed in the LMF application. The main PLC that controls the LMF is able to distinguish between both states by means of a signal that is set if the flexible modules is connected. In this case the control software recognizes the changed plant structure and adjusts the program flow to a new flow. Based on this concept, the basic approach of an agile manufacturing process can be shown and demonstrated. Even though it still has to be configured and programmed in advance.

In order to achieve a real agile manufacturing process, the integration of a new module has to be realized as automatic as possible, i.e., in a plug and work fashion. Whenever a new module is available or another module is removed, new input and output data is provided to the PLC and its running application software. Hence, a new automation application is required to also support the additional module. Typically, this software is implemented using tools and programming languages according to IEC 61131-3 [32], for example Structured Text (ST) or Function Block (FB). A function block comprises a certain number of inputs and outputs which are associated to variable names. These are connected statically to a certain memory area within the PLC. Therefore, these variable names do only have a meaning inside the engineering tool. During compilation, the variable names are replaced by memory addresses of the driver.

The needed process on the application layer for including a new module is therefore limited to the necessity of including a modified PLC application in an already established environment. The upcoming model based development approaches, as described in section 6.2.1, might be a suitable approach to solve this problem. In such a model, the production facility is described in detail. Usually, the model includes a description of both mechanical components and software components. On



an abstract level, the software components are reading and writing data to and from the hardware components to control them. Therefore, each component in an automation system can be represented by a number of input and output ports. The interconnection of these components is realized by links with a certain name and each link is used to transport information between two components. Therefore, a link always represents a connection between any type of component. By defining the datatype of the link, the model can be used as basis for deriving the necessary variable mapping for data exchange by interpreting each link as a variable of a certain type. In order to accomplish such a model based reconfiguration process, the following steps are necessary:

1. Model of the adapted manufacturing process

2. IEC 61131 code generation

3. PLC application update

First of all, the model has to be developed with a sufficient level of detail, i.e. all important and relevant aspects for the PLC application are considered. It should be possible to import this model into a PLC programming tool which will be then used to compile the necessary IEC 61131 code. Finally, the generated code will be uploaded into the PLC and can be used as soon as the changes are applied to the real system.



7 Conclusions and Next Steps

At this stage of the project, we may state that the first building blocks of the architecture have now been defined. The approach followed in this deliverable has been to avoid a clean slate solution when talking about a transparent and self-configuring IoT purposed systems. This applies to automation systems, where configuration and commissioning takes a heavy toll in the whole cost of engineering the system.

This work is major advancement towards understanding the boundaries of what is possible with today’s technologies and limitations. We now have managed to model our automation system into a real IoT whose things are intelligent or smart embedded devices interacting with each other in an automated manner. The steps towards an internet of machines are becoming clearer.

First the definition of the interactions between the machines in factory automation will still be a task of programming several applications defining the automated interaction between the machines, its sensors, and its actuators. There will also be other types of applications running besides the production which are data-driven like tracking or monitoring the production process, optimizing the production or its environmental footprint, safety. These are our future IoT-enabled applications. The IoT should enable this multitude of applications to run and interact with the same underlying IoT infrastructure.

The ultimate goal of the bootstrap process is to allow a simplified configuration of each of the offline programmed applications, without interrupting the other existing applications. This process should be as simple as compiling a code in a modern computer. The resources defined in the applications are described according to their type, role, and placement in the system. The pairing process requires the IoT to check or collect the same knowledge from the underlying networked things and devices, while comparing that with the planned one.

This rather abstract definition of bootstrapping is explained in this deliverable by focusing on the knowledge base that the IoT can collect. We call this knowledge context, which is gradually constructed from the moment a device instance is added, until it is allowed to become a fully productive part of the system. The device is identified through its neighbourhood, encrypted device descriptions, and other checks which occur automatically. The pieces of information are automatically collected by the functional blocks that make out the IoT@Work bootstrap architecture. This knowledge is also made available for the task of pairing the device instance with its offline ready program or configuration. In that sense the task of bootstrapping in IoT@Work will result in an automatic configuration of device instances as they are planned by the application programmer (or automation expert).

This automatic configuration also includes well defined concepts for trust and security that make the system robust against misconfigurations, malicious behaviour, or wrong information.

It is also important to keep in mind that some of the technologies described throughout the document but do not yet consist a final choice of implementation. An example would be to state that the Device Directory is an element that could reuse implementations of neighbour discovery in IPv6 for the discovery process, DNS and DHCP protocols for collecting device context and properties or even implementing part of the directory itself. However, other subcomponent like the directory guide, which could be used to define the bootstrap stages and or indicate the formats used for semantic and ontology description of device roles, might be out of scope for the above protocols and require further thoughts when it comes to implementation.



We therefore propose to include the functional elements defined so far in a larger IoT@Work architecture model that will define further dependencies between middleware services and their interactions. And at a later stage will return to selecting the technologies according to their properties, fulfilment of the requirements, and ease of integration in a real system.

Furthermore, the agile implementation process also will permit to define those elements, which we can easily implement and those that we will specify but and only emulate.

The approach described above fits within the expected and planned milestones of this project, which are shown in the table below. The focus is made on those milestones that will benefit directly from the outcome of this deliverable..

Sub-task Title Objectives Deliverable

T2.3 Solutions for auto-configuring names and addresses within a bootstrapping architecture

• Auto-configuration prototyping for addressing and naming

• Requirement assessment for a bootstrapping architecture

• First architecture elements of IoT@Work middleware

D2.2

(M12)

WP1/T1.1 Architecture requirements & assumptions

• Revision of Plug&Work requirements, and needs

• First architecture specification centered on Plug&Work IoT

• Implications on technical requirements WP2/3

D1.2

(M14)

WP3/T3.4 Solution for Security Bootstrapping

• Secure Plug&Work framework definition and mechanisms, considering usage of naming and addressing from a security perspective

D3.2 (M22)

Table 7.1 – Follow-up Activities Based on This Deliverable



8 References

[1]. Obiltsching, Günter, Automatic Configuration and Service Discovery for Networked Smart Devices, Electronica Embedded Conference Munich 2006, Applied Informatics Software GmbH, Austria, 2006

[2]. Barthel, Alexander; Analysis, Implementation and Enhancement of Vendor dependent and independent Layer-2 Network Topology Discovery; School of Computer Science, Professorship of Computer Networks and Distributed Systems, Chemnitz University of Technology, 2005

[3]. Reed, Archie. Implementing directory services. McGraw-Hill Companies, 2000

[4]. Howes, Tim; Smith, Mark C.; Good, Gordon S.; Howes, Timothy A.; Understanding and Deploying LDAP directory services. Macmillan Technical Pub, 1998

[5]. Obiltsching, Günter. Automatic Configuration and Service Discovery for Networked Smart Devices. Electronica Embedded Conference Munich 2006, Applied Informatics Software GmbH, Austria, 2006

[6]. IoT-A: Internet of Things – Architecture – EU funded project URL: http://www.iot-a.eu/ - Last checked: 2010-10-28

[7]. Electronic Device Description Language. URL: http://www.eddl.org/Pages/default.aspx - Last checked: 2010-11-09

[8]. FDT/DTM, EDD und TCI: ein Wegweiser für ein erfolgreiches Geräte-Management. URL: http://www.process.vogel.de/fdt/articles/111489/ - Last checked: 2010-11-12

[9]. Grossmann, D.; Bender, K.; Danzer, B., "OPC UA based Field Device Integration," SICE Annual Conference, 2008 , pp.933-938, Aug. 2008

[10]. PLCOpen Organization. URL: http://plcopen.org/ - Last checked: 2010-11-12.

[11]. Drath, R.; Luder, A; Peschke, J.; Hundt, L. “AutomationML - the glue for seamless automation engineering” in Emerging Technologies and Factory Automation, 2008.

[12]. Gershenson, Carlos; Heylighen, Francis. When Can we Call a System Self-organizing? Centrum Leo Apostel, Vrije Universiteit Brussel, 2003.URL: http://arxiv.org/ftp/nlin/papers/0303/0303020.pdf - Last checked:2010-10-06

[13]. M. G. Mehrab B I, A. G. Ulsoy and Y. Koren. Reconfigurable manufacturing systems: Key to future manufacturing. Department of Mechanical Engineering and Applied Mechanics, The University of Michigan (Journal of Intelligent Manufacturing (2000) 11, 403-419), 2000

[14]. Bukva, Senad; Enste, Udo; Uecker, Felix. Selbstkonfiguration und automatisiertes Änderungsmanagement von MES-Systemen. Engineering, LeiKon GmbH, 2009. URL: www.atp-online.de.Checked: 16.09.2010

[15]. B. Favre-Bulle, A. Zoitl C. Dutzler.Zukunftstrends der Automatisierungstechnik und die Bedeutung von adaptiven Produktionssystemen. Institut für Automatisierungs-und Regelungstechnik (ACIN), Technische Universität Wien, 2006. URL: http://www.springerlink.com/content/r7xw34814l3850ug/ - Checked: 14.05.2010

[16]. Claudia-Melania Chituc, Francisco José Restivo. Challenges and Trends in Distributed Manufacturing Systems: Are wise engineering systems the ultimate answer?. Department of Informatics Engineering of the Faculty of Engineering of the University of Porto and LIACC – Artificial Intelligence and Computer Science Laboratory, Rua Roberto Frias Porto 4200-465 Portugal, 2009 - URL: http://esd.mit.edu/symp09/submitted-papers/chituc-paper.pdf - Checked: 14.05.2010

[17]. Moore, Michael; Suda, Tatsuya. A Decentralized and Self-Organizing Discovery Mechanism. Department of Information and Computer Science, University of California, Irvine, 2002. URL: http://lion.cs.uiuc.edu/muri_afosr/Pub_Suda/27_discovery_ains.pdf - Checked: 14.09.2010



[18]. Kramer, Jeff; Magee, Jeff. Self-Managed Systems: an Architectural Challenge. Faculty of Engineering at Imperial College London; Department of Computing at Imperial College London, 2007. URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.739&rep=rep1&type=pdf - Checked: 13.10.2010

[19]. Tommila, Teemu; Ventä, Olli; Koskinen, Kri. Next Generation Industrial Automation - Needs and Opportunities. VTT Automation; Helsinki University of Technology, AUTOMATION TECHNOLOGY REVIEW, 2001, URL: http://mv-sirius.fh-offenburg.de/ecmIC/SensorActuatorObj-Worksheet/atr_2001.pdf, Checked: October, 8

th

2010

[20]. Kleegrewe, Christian, Mersch, Henning and Epple, Ulrich. Service-orientation on behalf of self-configuration for the automation environment. Proceedings of the IECON 2010 Conference, Phoenix, AZ, http://iecon2010.njit.edu/, IEEE, 2010

[21]. S. Cheshire. Zero Configuration Networking. URL: http://www.zeroconf.org/

[22]. Obiltsching, Günter. Automatic Configuration and Service Discovery for Networked Smart Devices. Electronica Embedded Conference Munich 2006, Applied Informatics Software GmbH, Austria, 2006

[23]. Narten, Thomas; Nordark, Erik; Simpson, William; Soliman Heshman. Neighbor

Discovery for IP version 6 (IPv6). IETF, RFC 4861, September 2007. http://www.ietf.org/rfc/rfc4861.txt

[24]. Hinden, Robert; Deering, Stephen. IP Version 6 Addressing Architecture. IP Version 6 Addressing Architecture. IP Version 6 Addressing Architecture. IP Version 6 Addressing Architecture. IETF, RFC 4291, February 2006. http://www.ietf.org/rfc/rfc4291.txt.

[25]. Stuart Cheshire and Marc Krochmal. “DNS-Based Service Discovery”. IETF, Internet

Draft, published 8th March 2010, last visited December 2010. URL: http://files.dns-

sd.org/draft-cheshire-dnsext-dns-sd.txt

[26]. McCloghrie, K.; Rose, M. Management Information Base for Network Management of TCP/IP-based Internets: MIB-II. IETF, RFC 1213, Hughes LAN Systems and Performance Systems International, March 1991.

[27]. Case Jeffrey; Fedor, Mark; Schoffstall, Martin Lee., Davin, James. A Simple Network Management Protocol (SNMP). IETF, RFC 1157, May 1990.

[28]. Berners-Lee, T.; Fielding, R.; Msinter, L. Uniform Resource Identifier (URI): Generic Syntax. IETC, RFC 3986. W3C/MIT, Day Software and Adobe Systems. January 2005.

[29]. Moats, R. URN Syntax. AT&T. IETF, RFC 2141. May 1997.

[30]. M. Soebell, Ubuntu's Upstart event-based init daemon, Feb. 2008,

http://www.linux.com/archive/feature/125977

[31]. Iwan Schafer, Max Felser, Topology Discovery in PROFINET, Berne University of Applied Sciences, 2th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA 2007), http://www.felser.ch/download/ETFA-01-2007.pdf

[32]. International Electrotechnical Commission, IEC 61131-3 Standard - Programmable controllers - Part 3: Programming languages, December 2003.

[33]. International Electrotechnical Commission, IEC 61918-2010 Standard - Industrial communication networks – Installation of communication networks in industrial premises, July 2010.

[34]. Profibus User Organization, Profinet Design Guideline, Version 1.04, November 2010.

[35]. A.P. Kalogeras, C. Diedrich, P. Neumann, and G. Papadopoulos. Function block definition based on the iec 1499 language. In Industrial Electronics Society, 1998.



IECON ’98. Proceedings of the 24th Annual Conference of the IEEE, volume 1, pages 169–172 vol.1, Aug-4 Sep 1998.

[36]. F. Ubis, T. Kirkham, B. Matthews, J. L. Martinez Lastra, R. Harrison, V. Villase Herrera, and A. Chowdrey. The challenges along the road to the realisation of a factory automation lifecycle. Annual International Computer Software and Applications Conference, 0:559–562, 2008. Socrades Project.

[37]. S. Mellor, K. Scott, A. Uhl, and D. Weise. MDA Distilled: Principles of Model-Driven Architecture. Addison Wesley, 2004.

[38]. OPC Foundation. OPC-UA. http://www.opcfoundation.org/, 2004.

[39]. Richard F. Paige, Dimitrios S. Kolovos, Louis M. Rose, Nicholas Drivalos, and Fiona A.C. Polack. The design of a conceptual framework and technical infrastructure for model management language engineering. Engineering of Complex Computer Systems, IEEE International Conference on, 0:162–171, 2009.

[40]. Jasperneite, Jürgen; Imtiaz, Jahanzaib; Schumacher, Markus; Weber, Karl: A Proposal for a Generic Real-time Ethernet System. In: IEEE Transactions on Industrial Informatics(5) S.: 75-85, May 2009.

d2.2 iot work final-v1.0 1 - pdfs.semanticscholar.org · omg dds the omg data-distribution service...

Documents