mantis · mantis d2.9 reference architecture and design specification final version work package...

Cyber Physical System based Proactive Collaborative

Maintenance

MANTIS

D2.9 Reference architecture and design

specification final version

Work Package WP2 – Service platform architecture development

Version 1.02

Contractual Date of Delivery 28/02/2018

Actual Date of Delivery 04/07/2018

Dissemination Level Public

Responsible Fraunhofer

Contributors Paul Verhallen (PHC), Martin Becker (FHG), Sö ren Schneickert (FHG), Sven Camenzind (FHG), Karmele Inxausti (IKERLAN) , Jose Luis Flores(IKERLAN), Lorenzo Manero (IKERLAN),Felix Larrinaga (MGEP), Luis Lino Ferreira (ISEP)

Reviewers Karmele Inxausti (IKERLAN), Lorenzo Manero (IKERLAN), Felix Larrinaga (MGEP), Sö ren Schneickert (FHG), Paul Verhallen (PHC), Martin Becker (FHG), Urko Zurutuza (MGEP)

D2.9 Reference architecture and design specification final version 662189 – MANTIS – ECSEL-2014-1

ii http://www.mantis-project.eu MANTIS

The MANTIS consortium consists of:

Num. Short Name Legal Name Role Country

1 MGEP Mondragon Goi Eskola Politeknikoa J.M.A. S.Coop. CO ES

2 MONDRAGON Mondragon Corporacion Cooperativa S.Coop. BEN ES

3 IKERLAN Ikerlan S.Coop. BEN ES

4 TEKNIKER Fundacion Tekniker BEN ES

5 FARR Fagor Arrasate S.Coop. BEN ES

5.1 KONIKER Koniker S.Coop. TP ES

6 GOIZPER Goizper S.Coop. BEN ES

7 ACCIONA Acciona Construcción S.A. BEN ES

8 MSI Mondragon Sistemas De Informacion S.Coop. BEN ES

9 VTT Teknologian Tutkimuskeskus VTT Oy BEN FI

10 LUAS Lapin Ammattikorkeakoulu Oy BEN FI

11 NOME Nome Oy BEN FI

12 FORTUM Fortum Power And Heat Oy BEN FI

14 WAPICE Wapice Oy BEN FI

15 AAU Aalborg Universitet BEN DK

16 DANFOSS Danfoss A/S BEN DK

19 VESTAS Vestas Wind Systems A/S BEN DK

20 SIRRIS Sirris Het Collectief Centrum Van De Technologische Industrie BEN BE

21 ILIAS Ilias Solutions Nv BEN BE

22 ATLAS Atlas Copco Airpower Nv BEN BE

23 3E 3e Nv BEN BE

24 PCL Philips Consumer Lifestyle B.V. BEN NL

25 PHC Philips Medical Systems Nederland B.V. BEN NL

26 PHILIPS Philips Electronics Nederland B.V. BEN NL

27 S&T Science and Technology B.V. BEN NL

28 TU/E Technische Universiteit Eindhoven BEN NL

29 RUG Rijksuniversiteit Groningen BEN NL

30 UNINOVA UNINOVA - Instituto de Desenvolvimento de Novas Tecnologias BEN PT

31 ISEP Instituto Superior de Engenharia do Porto BEN PT

32 INESC Instituto de Engenharia de Sistemas e Computadores do Porto BEN PT

33 ADIRA ADIRA - Metal Forming Solutions S.A. BEN PT

34 ASTS Ansaldo STS S.p.A. BEN IT

35 CINI Consorzio Interuniversitario Nazionale per l’Informatica BEN IT

36 AIT Austrial Institute of Technology GmbH BEN AT

37 HBM Hottinger Baldwni Messtechnik GmbH BEN AT

38 INNOTEC Innovative Technology and Science Limited BEN UK

39 AITIA AITIA International Inc. BEN HU

40 BME Budaperst University of Technology and Economics BEN HU

41 JSI Josef Stefan Institute BEN SI

42 XLAB XLAB d.o.o. BEN SI

43 FHG Fraunhofer Institute for Experimental Software Engineering IESE BEN DE

44 M2X M2Xpert GmbH & Co KG BEN DE

45 STILL STILL GMBH BEN DE

46 BOSCH Robert Bosch GMbH BEN DE

47 LIEBHERR Liebherr-Hydraulikbagger GmbH BEN DE

48 SATA Sataservice Oy BEN FI

49 XTEL Xtel Aps BEN DK

50 NEOGRID Neogrid Wireless Aps BEN DK

662189 – MANTIS – ECSEL-2014-1 D2.9 Reference architecture and design specification final version

http://www.mantis-project.eu iii MANTIS

Document Revisions & Quality Assurance

Revisions:

Version Date By Overview

0.1 01/12/2017 P. Verhallen Initial document

0.2 01/03/2018 P. Verhallen, M. Becker, S. Camenzind, S Schneickert

Added various sections and comments

0.3 20/03/2018 J. Flores, K. Intxausti Reworked Security chapter

0.4 22/03/2018 L. Manero, B. Tijsma Added Edge computing section

0.5 30/03/2018 F. Larrinaga Reworked Goizper Reference Implementation

0.6 18/04/2018 S. Camenzind Added various sections and comments

0.7 01/05/2018 P. Verhallen First version for review

0.8 20/05/2018 P. Verhallen Processed review comments from MGEP & IKERLAN

0.9 01/06/2018 M. Becker Processing of remaining review comments

1.0 03/06/2018 P. Verhallen Final edits

1.01 03/07/2018 S, Schneickert Quality Review

1.02 04/07/2018 U. Zurutuza Final Edition and Formatting


iv http://www.mantis-project.eu MANTIS

Abstract

This deliverable is the entry point to all architecture documentation in the MANTIS project, which describes all aspects of the MANTIS Reference Architecture, Interfaces, and Information Model. Where

applicable the document at hand links to more detailing architecture documentation in the context of MANTIS; for its own elaborations, D2.9 traces its content to the MANTIS requirements elicited in WP1 of the project.

The intent of this document is to provide guidance to partners, stakeholders, and any other interested

organisatios on how to establish a Predictive Maintenance (PdM) solution, which fits their business characteristics. It does this by identifying key architecture drivers and based on these drivers suggests architecture styles, patterns and topics to consider (such as security concerns). Through the help of reference applications & use-cases, it offers a more concrete view on how one can implement a PdM solution.

The document is structured as follows: chapter 1 introduces the MANTIS project and its goals followed by a summary of the Service Platform Architecture development work package and its vision and mission.

Also in chapter 1, the reader will find a disambiguation of the term ‘Reference Architecture’, which is the

starting point for the explanation of the architecting approach, used in MANTIS. Skip chapter 1, when

you are not interested in the project context and the general approach taken.

This specification document derives in chapter 2 the MANTIS related business, key functional and quality

goals, and describes the system context and scope as well as the maintenance domain in chapter 3. Skip chapter 2 when you do not want to check on the developed architecture drivers, and skip chapter 3 when

you are not interested in the delineation of system and context, and when you are familiar with the maintenance domain.

The document then transfers the ground setting considerations of chapters 2 & 3 into the MANTIS Reference Architecture, by three steps. Chapter 4 considers the aspect of what the reference architecture for PdM should support (functional perspective), chapter 5 depicts how the reference architecture for

PdM solution should be built (logical perspective), and chapter 6 gives examples concerning which technologies and solutions can be used for deploying the architecture.


http://www.mantis-project.eu 1 MANTIS

Table of Contents

1 Introduction ...................................................................................................................... 4

1.1 MANTIS Overview ................................................................................................................ 5

1.1.1 MANTIS Goals ............................................................................................................................ 6

1.2 WP2 Overview ..................................................................................................................... 7

1.2.1 MANTIS Platform Architecture Vision and Mission........................................................................ 7

1.3 Purpose of the Architecture Documentation ............................................................................... 9

1.4 MANTIS Reference Architecture Overview ............................................................................... 11

1.4.1 What is a reference model? ........................................................................................................ 11

1.4.2 What is a reference architecture? ................................................................................................ 12

1.4.3 Elements of the MANTIS ARM .................................................................................................. 12

1.5 Outcome of the MANTIS Reference Architecture ....................................................................... 13

2 Architecture Drivers ......................................................................................................... 14

2.1 Business Drivers................................................................................................................... 15

2.2 Stakeholders ....................................................................................................................... 17

2.3 MANTIS Platform Requirements Overview ............................................................................... 19

2.4 Key Functional Goals ............................................................................................................ 25

2.5 Quality Goals ...................................................................................................................... 27

2.6 Driver Prioritization .............................................................................................................. 31

2.7 Maintenance Related Standards .............................................................................................. 31

3 System Scope, Context and Maintenance Domain ................................................................. 32

3.1 System Scope and Context .................................................................................................... 33

3.2 Maintenance Domain Model .................................................................................................. 35

3.3 MANTIS Feature Model ........................................................................................................ 37

3.4 Architecture Principles and Constraints .................................................................................... 40

3.5 Design Approach ................................................................................................................. 40

4 Reference Model (Functional Perspective)............................................................................ 41

4.1 Foundations ........................................................................................................................ 41

4.1.1 Condition monitoring and diagnostics of machines ....................................................................... 41

4.1.2 Internet of Things – Architecture................................................................................................ 42

4.1.3 Industrial Internet of Things Reference Architecture ..................................................................... 42

4.2 Information View [Information Model]...................................................................................... 44

4.2.1 Definition of the MANTIS Information Model .............................................................................. 44

4.2.2 Other information-related models in MANTIS .............................................................................. 48

4.3 Functional View ................................................................................................................... 49

4.3.1 Functional Concept.................................................................................................................... 49

4.3.2 Function Tree ........................................................................................................................... 50

5 Reference Architecture (Logical Perspective) ........................................................................ 56


2 http://www.mantis-project.eu MANTIS

5.1 Overview ............................................................................................................................ 56

5.2 Data Processing and Management .......................................................................................... 59

5.3 Interoperability .................................................................................................................... 60

5.4 Lifecycle Management .......................................................................................................... 61

5.4.1 Application ............................................................................................................................... 61

5.4.2 Interface ................................................................................................................................... 62

5.4.3 System ..................................................................................................................................... 62

5.4.4 Continuous Automation ............................................................................................................. 62

5.5 Security.............................................................................................................................. 63

5.5.1 Introduction .............................................................................................................................. 63

5.5.2 Supporting the vision of MANTIS ............................................................................................... 63

5.5.3 Security Information Model ........................................................................................................ 64

5.5.4 MANTIS Security Implementation .............................................................................................. 64

5.6 Reference Applications .......................................................................................................... 76

6 Reference Technologies and Solutions ................................................................................. 79

6.1 Reference Technologies ......................................................................................................... 79

6.2 MANTIS Reference Solutions ................................................................................................. 80

6.2.1 Use Case: Conventional Power Production................................................................................... 80

6.2.2 Use Case: STILL & Ansaldo ....................................................................................................... 83

6.2.3 Use Case: Adira ......................................................................................................................... 84

6.2.4 Use Case: Goizper ...................................................................................................................... 84

6.2.5 Use Case: Philips Healthcare ...................................................................................................... 88

7 Conclusions ..................................................................................................................... 90

8 List of Tables .................................................................................................................. 91

9 List of Figures ................................................................................................................. 92

10 Literature and References .............................................................................................. 94

11 Appendix .................................................................................................................... 98

11.1 Key Architectural Concepts.................................................................................................... 98

11.1.1 Architectural Styles & Patterns .............................................................................................. 98

11.1.2 Edge Computing .................................................................................................................. 107

11.2 Design Decision Rationales .................................................................................................. 117

11.2.1 Main solution structure ........................................................................................................ 117

11.2.2 Edge-to-Platform Communication ......................................................................................... 118

11.2.3 Separate gateway from asset ................................................................................................ 118

11.2.4 Platform as a cloud ............................................................................................................. 119

11.2.5 Maintenance function deployment ......................................................................................... 120

11.2.6 Common functionality groups ............................................................................................... 121

11.2.7 Data processing patterns ...................................................................................................... 121

11.2.8 Data types and storage ........................................................................................................ 122

11.2.9 Data storage organization .................................................................................................... 122

11.2.10 Data access via data management facade .............................................................................. 123



11.2.11 Collaborative proactive maintenance ..................................................................................... 123

11.2.12 Lifecycle management on gateway ........................................................................................ 124

11.2.13 Lifecycle management on platform ........................................................................................ 124

11.3 Architecture Driver Solutions ............................................................................................... 125

11.3.1 Main solution ...................................................................................................................... 125

11.3.2 Data Processing and Management ........................................................................................ 126

11.3.3 Lifecycle Management .......................................................................................................... 127



1 Introduction

Context and challenge

MANTIS incorporates different industries in various stages at the pathway towards a fully deployed system

for proactive maintenance. The project aims at speeding up the development of such systems and their adaptation for the participating industries. This requires a common view of the overall architecture and design so that experiences can be efficiently reused and new findings can be generalized to support a wider span of use cases and industries. The specific challenge addressed by this task is hence reusability on a system level.

Objective

The objective was to develop a uniform service platform architecture and overall design that can be used to describe the particular construction and configuration of solutions for the different use cases of MANTIS. A further objective was that this service platform shall support the reuse of experiences and

new findings across different industries, manufacturing and producing, less or more mature regarding their use of proactive maintenance.

Description of work done

Task 2.1 inventoried the service platform architecture and design of the MANTIS use cases and documented their common parts into a reference architecture and design of the project. Based on this

baseline, the WP developed the reference description to promote reuse of experiences and new findings between different use cases and industries. In addition, the documentation presents strategies for how individual industries can evolve in the area of proactive maintenance.



1.1 MANTIS Overview

The MANTIS proactive service maintenance platform and its associated architecture draw inspiration from the Cyber Physical System approach. Physical systems (e.g. industrial machines, vehicles, renewable

energy assets) operate in an environment, where everything is continuously monitored by a broad and diverse range of intelligent sensors.

This results in massive amounts of data. Systems are characterised for example by their usage history,

operational condition, location, movement and other physical properties. These systems form larger networks of heterogeneous and collaborative systems-of-systems (e.g. vehicle fleets or photovoltaic and windmill parks), and are connected via robust communication mechanisms able to operate in challenging

environments.

Sophisticated distributed sensing and decision making functions are performed at different levels in a collaborative way ranging from:

(i) the local nodes (that pre-process raw sensor data and extract relevant information before transmitting it, thereby reducing bandwidth requirements of communication),

(ii) over intermediate nodes (that offer asset-specific analytics to locally optimise performance and

maintenance), (iii) into cloud-based platforms (that integrate information from ERP, CRM and CMMS systems

and execute distributed processing and analytics algorithms for supporting global decision making processes).

For the optimum maintenance of assets, different systems and stakeholders will have to share information, resources and responsibilities. In other words, collaboration is required. Such a Collaborative Maintenance Ecosystem will have to be able to:

Reduce the adverse impact of maintenance on productivity and costs

Increase the availability of assets

Reduce the time required for maintenance tasks

Improve the quality of the maintenance service and products

Improve labour working conditions and maintenance performance

Increase sustainability by preventing material loss (due to out-of-tolerance production)

Optimize spare part management

Focus is on a proactive maintenance service platform architecture that can enable service-based business models and improve asset availability at lower costs through continuous process and equipment monitoring and data analysis. MANTIS also aims to identify and integrate critical information from further sources such as production, maintenance, equipment manufacturers and service providers.

Such service platform architecture will also have to take the needs of the industries into account in the

forefront of service-based business and operations as well as less mature ones. In this way, improvements in maintenance can be achieved gradually and consistently.

The overall concept of MANTIS is to provide a proactive maintenance service platform architecture that allows the precise estimation of future performance, the prediction and prevention of imminent failures

and should also be able to schedule proactive maintenance tasks.

This proactive maintenance service platform will consist of distributed processing chains that can efficiently transform raw data into knowledge while minimising the need for data transfer bandwidth.

To achieve this overall objective, we will need a smart integrated domain knowledge system with advanced data monitoring, communication and analytics capabilities, which is overall dependable and secure.



Smart sensors, actuators and cyber physical systems capable of local pre-processing and local data storing/buffering.

Robust communication systems for harsh environments.

Distributed machine learning for data validation and decision-making.

Cloud-based processing, analytics and data availability.

HMI to provide the right information to the right people at the right time in the right format.

Figure 1-1 MANTIS overview

1.1.1 MANTIS Goals

MANTIS aims to:

(1) Define the overall service platform architecture of the MANTIS distributed system for proactive maintenance.

(2) Develop the next generation framework for highly distributed sensing, including pre-processing, data

acquisition and adaptive information processing maintenance, to be known as the MANTIS Framework.

(3) Conceive a distributed collaborative maintenance decision making system.

(4) Provide user-friendly, ergonomic and intuitive context-aware human-machine interaction based on the MANTIS Maintenance Framework.

(5) Identify, define and implement new business opportunities through services provided by the MANTIS Framework.

(6) Establish higher maturity in the partner organisations regarding maintenance support. (7) Generate society, business and technology awareness supporting the rapid exploitation of solutions

demonstrated by MANTIS.



1.2 WP2 Overview

WP2 will devise the overall architecture of the MANTIS distributed system for proactive maintenance. The MANTIS service platform will address issues that impact on several steps in the chain of turning raw

data into information usable for distributed decision-making. In addition, the architecture has to address aspects such as interoperability, consistency, availability, reliability, robustness, safety and security of the system as a whole.

1.2.1 MANTIS Platform Architecture Vision and Mission

1.2.1.1 Vision

Based on input from WP1 on requirements and use case descriptions, this WP will develop a reference service platform architecture and overall design. This service platform shall allow for industries participating in MANTIS to take advantage of progress on proactive maintenance in related but different

industries. Also, it allows for less mature industries in the project to reuse experiences from industries in the forefront of proactive maintenance. They can thereby ensure that improvements in maintenance can

be achieved gradually and consistently with future plans and best practice. Important aspects that the architecture should address are:

Interface, protocol and functional interoperability ensuring that several cooperating vendors can effectively assemble the complete MANTIS service platform. Includes the need to identify or develop standards for data semantic representation and exploitation.

Data validation ensuring that data analyses are made on data that give clean, correct and useful data information about the system.

Distributed data, and information processing and decision-making ensuring consistent behaviour and avoid contradicting actions, e.g. between local and distributed data analysis and decision making.

Information validation ensuring that created information still is relevant for the system analysed.

Safety and fault tolerance ensuring that critical information remains available and following decisions can be taken or proposed although partial system failure.

System and service level security ensuring that the system incorporates means to hinder misconfiguration and can be protected from wire-tapping and various attacks.

System engineering and reusability of defined and existing services.

System verification and validation of the service platform architecture and overall design, covering both functional and non-functional properties.

The necessary steps of turning raw data into maintenance information will be characterized based on the

requirements coming from WP1. These characteristics will be used to model the service architecture. The starting point will be mentioned requirements and results from other projects i.e. Arrowhead project and

the Arrowhead Framework, which specifically addresses interoperability, consistency, safety and security. Further results from IMC-AESOP will be used to address reliability, robustness and engineering.

1.2.1.2 Mission

The mission of the platform architecting activities in MANTIS includes:



Devise the overall architecture of the MANTIS distributed system for proactive maintenance

Address issues that have an impact on several steps in the chain of turning raw data into information usable for distributed decision-making

Key aspects: interoperability, consistency, availability, reliability, robustness, safety and security of the system as a whole.

1.2.1.3 Structure

The platform architecting activities in MANTIS are structured into the following tasks within WP2 and

produce results as illustrated in Figure 1-2.

Figure 1-2 WP2 structure

The deliverables of Task2.1 and Task2.2 have based on their content a strong interrelation. The reference

architecture provides the basis and the interoperability specification adds further details to support the

necessary interoperability. This deliverable also lays the foundation for the information source integration with respect to the information model.



1.3 Purpose of the Architecture Documentation

The recognition, codification, and use (reuse) of generalized artefacts is an approach used to great advantage in traditional engineering disciplines, where they are typically called conventional designs. The

approach is being used in system & enterprise architecture as well, expressed in a variety of concepts and terms that include reference model, reference architecture, domain architecture, product line architecture, architecture pattern, and architecture style. What these have in common is that they are deliberately

defined at a general level that applies to multiple enterprises or architectures. They facilitate the development of a specific system architecture by providing common entities, functions, relations, and concepts or terms that can be tailored and specialized to the context and purpose of a particular architecture. Architecture communities are still in the process of reaching consensus on definitions of these

artefacts and the distinctions between them.

A reference architecture is a generalized architecture that can be specialized to a particular architecture such as an enterprise architecture, a system architecture, or a software architecture. A reference architecture can be based on one or more reference models. Reference architectures can also be defined for specific domains, and are sometimes called domain architectures.

The development of a specific reference architecture can make use of any of these generalized artefacts.

All of them help in various ways to avoid having to create a whole reference architecture from scratch, and help leverage the knowledge and experience that went into the formation and definition of the generalized models, architectures, and patterns. [1] [2]

A reference architecture provides a template, often based on the generalization of a set of solutions. These solutions may have been generalized and structured for the depiction of one or more architecture structures based on the harvesting of a set of patterns that have been observed in a number of successful

implementations. Further it shows how to compose these parts together into a solution. Reference architectures will be instantiated for a particular domain or for specific projects.

Adopting a reference architecture within an organization accelerates delivery through the re-use of an effective solution and provides a basis for governance to ensure the consistency and applicability of technology use within an organization. In the field of software architecture, many empirical studies have shown the following common benefits from adopting a software reference architecture within

organizations: [3] [4]

(1) improvement of the interoperability of the software systems by establishing a standard solution and common mechanisms for information exchange;

(2) reduction of the development costs of software projects through the reuse of common assets; (3) improvement of the communication inside the organization because stakeholders share the same

architectural mind-set; and, (4) influencing the learning curve of developers due to the need of learning its features.

The role of the reference architecture in MANTIS is to provide guidance on how to instantiate an architecture for a particular MANTIS domain or specific MANTIS task, and to ensure consistency and

applicability of technologies, interoperability mechanisms, data formats and models, data analysis tools, etc. to be used in the different MANTIS use cases.



To achieve the MANTIS goals (stated in section 1.1.1) the following tasks must be performed:

Missing data analysis solutions have to be developed and provided in a goal-oriented and value-adding way.

To this end, data owners have to collaborate with data analysis experts. Together they have to understand the problem and identify possible solutions.

In order to achieve this, they have to speak the same language and have to collaborate in a way that assures the necessary quality attributes.

Communication between involved stakeholders and the integration of system elements and functionality has to be eased.

Existing proprietary and thus heterogeneous solutions have to be integrated into a more homogeneous maintenance ecosystem.

Figure 1-3 The purpose of a common MANTIS architecture

This deliverable is intended to be a self-contained document and therefore uses input (e.g. overviews and summaries) from other (MANTIS) documents to provide an integrated reading experience. Several aspects

of the architecture (such as interoperability) are described in more detail in the referenced documents. (As can be seen in Figure 1-2.)



1.4 MANTIS Reference Architecture Overview

The architecting activities of WP2 want to provide reference solutions to the other work packages, especially to use cases in WP7. Inspired by IoT-A [5], we subsume these reference solutions in an

Architecture Reference Model (ARM). The MANTIS ARM is one of main results of the entire WP2 activities.

To organisations, an important aspect is the compliance of their technologies with standards and best

practices, so that interoperability across organisations is ensured. This is especially of interest when an ecosystem is targeted, that comprises system elements from different partners. If such compliance is given,

an ecosystem may form, in which every stakeholder can create new businesses that “interoperate” with

already existing businesses. The MANTIS ARM will provide best practices to the organisations so that

they can create compliant MANTIS architectures in different application domains.

If the MANTIS project succeeds, many PdM system architectures in the future are based on / or inspired by the MANTIS ARM with some specific architectural design choices taken. They will consist of special

“flavours” of a PdM reference architecture where of course interoperability is not sacrificed but on the

contrary ensured.

Before we dive into the MANTIS ARM, we have to clarify some main terms, including reference model and reference architecture as they are used in an ambiguous way in practice, which bears the risk of misunderstanding and disappointment.

1.4.1 What is a reference model?

According to OASIS (Organization for the Advancement of Structured Information Standards) [6], a

reference model is an abstract framework for understanding significant relationships among the entities of

some environment, and for the development of consistent standards or specifications supporting that environment. A reference model is based on a small number of unifying concepts and may be used as a basis for education and explaining standards to a non-specialist. A reference model is not directly tied to any standards, technologies or other concrete implementation details, but it does seek to provide a

common semantics that can be used unambiguously across and between different implementations."[1]

There are a number of concepts rolled up into that of a 'reference model.' Each of these concepts is

important [7]:

Abstract: a reference model is abstract. It provides information about environments of a certain kind. A reference model describes the type or kind of entities that may occur in such an environment, not the particular entities that actually do occur in a specific environment

Entities and relationships: A reference model describes both types of entities (things that exist) and their relationships (how they connect, interact with one another and exhibit joint properties). A list of entity types, by itself, does not provide enough information to serve as a reference model.

Within an environment: A reference model does not attempt to describe “all things”. A reference model

is used to clarify “things within an environment” or a problem space. To be useful, a reference model should include a clear description of the problem that it solves, and the concerns of the stakeholders who need to see the problem solved.

Technology agnostic: A reference model's usefulness is limited if it makes assumptions about the technology or platforms in place in a particular computing environment. A reference model typically is intended to promote understanding a class of problems, not specific solutions for those problems. As such, it must aid the process of imagining and evaluating a variety of potential solutions in order to assist the practitioner. Note: That does not preclude the development of a reference model that

describes a set of software applications, because the problem space may be “how to manage a set of

software applications” .

https://en.wikipedia.org/wiki/Reference_model#cite_note-1



1.4.2 What is a reference architecture?

According to Wikipedia [3] (based on ISO/IEC/IEEE 42010 [2]) reference architectures provide a template solution for the architecture (aka. architectural blueprint) for a particular domain. It also provides a

common vocabulary with which to discuss implementations, often with the aim to stress commonality.

A reference architecture often consists of a list of functions and some indication of their interfaces (or APIs) and interactions with each other and with functions located outside of the scope of the reference

architecture.

Reference architectures provide a template, often based on the generalization of a set of solutions. These solutions may have been generalized and structured for the depiction of one or more architecture structures

based on the harvesting of a set of patterns that have been observed in a number of successful

implementations. Further, it shows how to compose these parts together into a solution. Reference architectures will be instantiated for a particular domain or for specific projects.

1.4.3 Elements of the MANTIS ARM

The MANTIS ARM consists of five elements:

Reference Model: Provides the highest abstraction level for the definition of the MANTIS Architectural Reference Model. It promotes a common understanding of the PdM domain. The description of the PdM Reference Model includes a general discourse on the PdM domain, a PdM Domain Model as a top-level description, a PdM Information Model explaining how PdM knowledge is going to be modelled, and a PdM Communication Model in order to understand specifics about communication between many heterogeneous PdM devices and the Internet as a whole.

Reference Architecture: Is the reference for building compliant PdM architectures. As such, it provides views and perspectives on different architectural aspects that are of concern to stakeholders of the PdM. The terms view and perspectives are used according to the general literature and standards. The creation of the PdM Reference Architecture focuses on abstract sets of mechanisms rather than concrete application architectures.

Feature Model: Introduces key concepts to characterize common and varying aspects in the architectures to be derived from the reference architecture.

Guidelines: While the PdM Reference Model and MANTIS Reference Architecture give the needed models, views and perspectives for deriving a concrete architecture out of it, it is of the utmost

importance to guide the architect during this derivation process. The “Guidance” element is therefore an extremely important part of this work achieved by the MANTIS project. It discusses how the provided models, views and perspectives can be concretely used. Guidance can be found throughout this document in the form of principles, reference technologies, architectural concepts and Styles & Patterns.

Reference Applications: Reference architectures are abstract architectures for solutions in a well-scoped domain. In order to ease reasoning about the reference architecture, especially during the design and evaluation phase, and the communication of the reference architecture, a small set of reference applications is used. They show the diversity of the included solution variants, and thus illustrate architecture signification features and related design decisions.



Figure 1-4 MANTIS Architectural Reference Model building blocks

1.5 Outcome of the MANTIS Reference Architecture

The intent of this deliverable is to provide guidance to partners on how to establish a PdM solution, which fits their business characteristics. It does this by identifying key architecture drivers and based on these

drivers suggests architecture styles, patterns and topics to consider (such as security concerns). Through the help of Reference Applications & Use-Cases, it offers a more concrete view on how one can implement a PdM solution.



2 Architecture Drivers

This chapter contains all the architecture significant requirements, drivers, and constraints considered to

build the MANTIS reference architecture. It does so by following the principle of architecting for concrete

stakeholder concerns, also called Architecture Drivers (cf. left side of Figure 2-1). These stakeholders concerns usually belong to one of the following categories: Business Drivers, Functional Requirements, Quality Requirements, and Constraints.

Figure 2-1 Relationship between architecture Drivers and Architecture Design.



2.1 Business Drivers

As described in D6.2 the proactive maintenance value chain can be classified into 3 categories; asset manufacturers, asset service providers and asset end users. Below you find an excerpt from chapter 7,

which lists the business drivers per category.

Table 2-1: Business Drivers of the Asset Manufacturer

ASSET MANUFACTURER MANTIS ENABLED VALUE DRIVERS BUSINESS OUTCOMES

Asset design & production Asset reliability analysis Advanced HMI Improve product lifecycle Tap into customer base Machine virtualisation Improve and optimization of product design

Innovation New product introduction R&D process optimization Customer connected products

Supply chain & logistics Stock management Delivery time optimization Work piece traceability

Customer experience Production/CRM integration Improved customization Downtimes reduction

Accounting & business models Maintenance & warranty limits As-a-service business models Tap into customer base New payment models which transform capex into

opex for asset end-users Financial services Perpetuation of revenue streams instead of one-off

asset sale for suppliers

Asset design & production Increase equipment lifespan Increase operational efficiency Improve quality Idle time reduction Start-up time reduction

Innovation Accelerate time to market Integrate real-time customer feedback Analyse remote product performance

Supply chain & logistics Increased customer satisfaction

Customer experience Improvement of accuracy of warranty modelling Production output customization

Accounting & business models Customer’ s financial challenges overcome Differentiation Retain customer Gain new customers



Table 2-2: Business Drivers of the Asset Service Provider

ASSET SERVICE PROVIDER MANTIS ENABLED VALUE DRIVERS BUSINESS OUTCOMES

Condition Monitoring & Health Assessment Asset failure demand prediction Alarm triggering Advanced reports Remote monitoring Assets usage analysis

Asset retrofitting Customised and systematic sales actions

Other data-based services Add-on services for primary products (e.g. consulting

on best usage of products) Advance training solutions

Supply chain & logistics Spare parts supply management Connected supply chain network Stock management Delivery time optimization Spare part traceability

Service operations Service (& labour) planning & scheduling Tap into customer base Business KPI definition and monitoring

Condition Monitoring & Health Assessment Downtime reduction Increase equipment lifespan Operational Efficiency Increase Quality Improvement Maintenance cost reduction (for end-user)

Asset retrofitting Customer inks consolidation Increase equipment lifespan

Other data-based services Additional revenue streams Potential sales argument for future offers

Supply chain & logistics Customer satisfaction degree increase

Service operations Maintenance operation cost reduction

Table 2-3: Business Drivers of the Asset End User

ASSET END USER MANTIS ENABLED VALUE DRIVERS BUSINESS OUTCOMES

Asset utilization Remote monitoring Connected factory Downtime management

Employee Productivity Training management Advanced HMI Assistance systems Staffing readiness

Sustainability Energy management Quality control

Supply Chain & Logistics Factory safety management Connected supply chain Product & process traceability Indirect monetization of insights from collected data

Asset utilization Equipment lifespan increase Maintenance cost reduction Time to process orders reduction Production costs reduction

Employee Productivity Efficient training through collaborative solutions Worker mobility increase

Sustainability Energy consumption reduction Quality improvement Operational efficiency increase Rework & scrap reduction

Supply Chain & Logistics Accident reduction Stock reduction

The business drivers listed above influence the architectural design and decisions in MANTIS as they have been taken into account as input to the requirements and therefore lead to certain feature capabilities and or design decisions.



2.2 Stakeholders

In order to facilitate the efficient usage of this document, the respective stakeholders of the MANTIS project in general and the MANTIS reference architecture model are identified and their concerns are

analysed in the following sections. Links to the related sections, where the concerns are addressed, help with the navigation in this document.

The stakeholders of the MANTIS reference architecture form the central reader group for this document.

According to ISO42010 [2] different stakeholders have different concerns in the system-of-interest and therefore are interested in different parts of this architecture documentation.

On the other hand, the stakeholders have different concerns and constraints onto the architecture that

they have to be asked for.

A stakeholder in software or system architectures is a person, group, or entity, with an interest in or concerns about the realization of the architecture [ISO42010].

Typical stakeholders in the architecture of software-intensive systems are owners, users, managers, requirements engineers, architects, developers, evaluators, regulatory bodies.

The following figure provides an overview on the considered MANTIS stakeholders:

Figure 2-2 Stakeholder overview

uc Stakeholders

MANTIS Project

Supplier

System owner

Data analysis provider

Data provider

External reviewers

HMI technology

providerOEM

Partner

Project management

Regulation

organization

Sensor technology

provider

Service platform

provider

Solution element

provider

Solution provider

Standardization

organizationFunding organization

Maintenenance service

provider

Service technician



Table 2-4 MANTIS Stakeholders

Stakeholder Description

Data analysis provider Partner, who develops and provides data analyses

Data provider Stakeholder, who provides data in the MANTIS ecosystem

External reviewer Reviewer of the MANTIS project

Funding organization Organization, who funds MANTIS

HMI technology provider Partner, who provides HMI technology

OEM Partner, who acts as an Original Equipment Manufacturer in the

MANTIS ecosystem

Partner Partner of the MANTIS project

Project management MANTIS Project management function

Regulation organization Organization, who is concerned with safety, security and data protection issues

Sensor technology provider Partner, who develops new sensor technology

Service platform provider Solution provider, who provides a technical implementation of the MANTIS service platform

Service provider Solution provider, who provides a service on the MANTIS service platform

Solution element provider Organization who provides a solution element in a MANTIS ecosystem, for instance an analysis service, sensor, HMI framework

Solution provider Organization who provides a MANTIS solution ecosystem

Standardization organization Organization who creates/governs/controls standards

Supplier Partner, who acts as an supplier in the MANTIS ecosystem

System owner Partner, who acts as an owner of a physical system in the MANTIS

ecosystem



2.3 MANTIS Platform Requirements Overview

In WP1 of MANTIS, the tasks 1.2 to 1.5 elicited and refined the requirements for the MANTIS platform. From the activities in these tasks, a list of categorized requirements (functional and non-functional)

resulted. WP1 also identified the requirements’ relevance for the MANTIS reference architecture. The

respective results can be found in D1.4-D1.6 (where D1.6 is the last deliverable in this series of

documents), and related documentation (like e.g. Excel files holding the single requirements).

Table 2-5 shows platform architecture relevant requirements elicited in WP1 and assigned to task 2.1. The table shows the IDs (for traceability reasons), the requirement category and the description.

When going through the requirements, task 2.1 investigated some restrictions (yellow), and discarded

three of them (red). An explanation follows below the table. (see also D1.6 and related documents)

Table 2-5: Requirements, directly related to the MANTIS platform

ID Category Description

Function - Handle Data

MPR_0001 Data - Measurement The platform must enable to measure all kind of distributed process data

MPR_0002 Data - Measurement The platform must enable to measure all kind of environmental data

MPR_0003 Data - Aggregation The platform must enable to aggregate distributed data

MPR_0004 Data - Calculation The platform must enable to calculate data from all kind of data input

MPR_0005 Data - Transformation The platform must enable to transform data - syntactically - semantically

MPR_0006 Data - Logging The platform must enable to retain and log all kind of data

MPR_0007 Data - Manipulation The platform must enable a legal data discovery

MPR_0008 Data - Manipulation The platform must enable to export data

MPR_0009 Data - Manipulation The platform must enable to audit data

MPR_0010 Data - Manipulation The platform must enable to destroy all kind of data

MPR_0011 Data - Transmission The platform must enable to transmit all kind of data

MPR_0012 Data - Security The platform must enable to encrypt and decrypt data

MPR_0013 Data - Security The platform must enable to certify data

MPR_0014 Data - Access The platform must enable to define the following data access rights: - read - write - delete

MPR_0015 Data - Access The platform must enable to access data portions concurrently

MPR_0016 Data - Monitoring The platform must enable to monitor all kind of data - locally - remotely




MPR_0017 Data - Analysis The platform must enable to analyse distributed data by an open list of analyses. Analyses currently to be supported are: - root cause of failure analysis - symptom analysis - automatic sensor data interpretation - remaining useful life estimation - calculation of optimized maintenance strategy

MPR_0018 Data - Analysis The platform must enable to analyse distributed data in real time

MPR_0019 Data - Analysis The platform must enable to analyse distributed data in non-real time

MPR_0020 Data - Analysis The platform must enable to access distributed data storages

MPR_0021 Data - Analysis The platform must be robust against missing data

MPR_0022 Data - Collection The platform must enable to collect data in different points of the system

MPR_0023 Data - Storage The platform must provide a GUID for data items to be stored

MPR_0024 Data - Storage The platform must provide a time stamp for data items to be stored

MPR_0025 Data - Restoration The platform must enable to restore data

MPR_0026 Data - Back-Up The platform must enable a back-up mechanism for all kind of data

Function - Handle Event

MPR_0034 Data - Event Handling The platform must provide an event handling mechanism that supports the following event types: - Simple event - Complex event

MPR_0035 Data - Event Handling The platform must provide an event handling mechanism that supports the configuration of an event trigger

MPR_0036 Data - Event Handling The platform must provide an event handling mechanism that supports the following action types: - Alert - Notification via E-Mail - Notification via SMS - Control of actuators

MPR_0037 Data - Event Handling The platform must provide an event handling mechanism that supports the configuration of action conditions

MPR_0038 Data - Event Handling The platform must provide an event handling mechanism that supports the configuration of action targets

MPR_0039 Data - Event Handling The platform must provide an event handling mechanism that supports the configuration of action time values

Function - System Operation

MPR_0041 Data - System Operation When operating the system, the operator must be able to calibrate sensors




MPR_0042 Data - System Operation The platform must allow to configure the maintenance data transmission for any component of interest along the following categories: - scope - frequency - target

MPR_0043 Data - System Operation When operating the system, the operator must be able to configure components

MPR_0044 Data - System Operation When operating the system, the operator must be able to parameterize components

Data - Form

MPR_0045 Data - Form The platform must support the handling of data in the following forms: - image - video - sound - text - database

Data - Type

MPR_0046 Data - Type The platform must support the handling of data of the following types: - metadata - aggregated sensor data - interpreted sensor data - process data - event data - master data - bills of material - parameter data - configuration data - history data - design data - reporting data - geo-position data - manual data - environmental data - business data

Data - Format

MPR_0047 Data - Format The platform must support the handling of data of the following formats: - structured - unstructured

MPR_0048 Data - Compliance The platform must support to adhere to given data compliance rules

Quality - System Reliability

MPR_0055 Reliability - Tolerance The platform must be fault tolerant

MPR_0056 Reliability - Buffering The platform must provide a mechanism for the buffering of data




Quality - System Availability

MPR_0057 Availability - Time Span The platform must be available - 24 hours a day - 7 days a week - 364 days a year

MPR_0058 Availability - DropOut The platform's availability (MTBF/(MTBF+MTTR)) must be >= 99.999%

MPR_0059 Availability - Repair The platform's MTTR must be <=1h

MPR_0060 Availability - Procedures The platform must provide the following standard procedures: - Monitor system for prediction of availability - Notification on known system downtime - System self-diagnosis - Handle system events

Quality - Maintainability

MPR_0071 Maintainability - Update The platform must provide an update mechanism

MPR_0072 Maintainability - Upgrade The platform must provide an upgrade mechanism

Quality - Safety

MPR_0073 Safety - Fail The platform operations must be fail-safe

MPR_0074 Safety Integrity Levels When monitored, components connected to the platform must not reduce their respective SIL

Quality - Physical Robustness

MPR_0075 Physical Robustness - Temperature The platform must be robust against the same temperature range as all connected/monitored components

MPR_0076 Physical Robustness - Hostile Environment The platform must be robust against the influence of hostile environments

Guaranty - Processing of All Data

MPR_0078 Guarantee - Data Procession The platform's architecture must guarantee that all registered data is processed

Guaranty - Execution of Defined Workflows

MPR_0079 Guarantee - Workflow Execution The platform's architecture must guarantee the execution of all defined workflows

Security - Data Security

MPR_0081 Data & System Security - Ownership The platform must provide a mechanism for handling data ownership

MPR_0082 Data & System Security - Role The platform's data handling must support a role concept

MPR_0083 Data & System Security - Authentication The platform's data handling must support a secure authentication of - data pieces - identities

MPR_0084 Data & System Security - Authorization The platform's data handling must support a secure authorization, i.e. a data access policy

MPR_0085 Data & System Security - Encryption The platform's data handling must support data encryption and decryption

MPR_0086 Data & System Security - Layers The platform's data transmission must support a transport layer security




Security - Security Levels

MPR_0087 Data & System Security - Levels The platform's architecture must define different security levels

Security - Security Zones

MPR_0088 Data & System Security - Zones The platform's architecture must define different security zones

HMI - Data Access

MPR_0107 HMI - Data Access (Location) The platform accessing HMI's application must customize data access along the following aspects: - location-awareness - location-based services - geo-fencing

MPR_0108 HMI - Data Access Restrictions The platform must allow to deliberately restrict data access along the following aspects: - time - time span - scope (of included assets)

MPR_0109 HMI - Data Access (Configuration) The platform must support to restrict data access subject to: - device configuration - application configuration - connection type

MPR_0110 HMI - Data Access The platform must allow an HMI connected to it to access data locally

MPR_0111 HMI - Data Access The platform must allow an HMI connected to it to access data remotely

HMI - Visualisation

MPR_0112 HMI - Visualisation The platform must be able to provide an HMI with information to visualize - Failure - Failure Case - Event - other data



Restricted requirements

MPR_0007: The fulfilment of this request is possible only via personnel involved. The system must

provide a data usage control mechanism.

MPR_0018: For this request, a latency must be given normally. Additionally, guaranteeing a real-time analysis not only depends on the platform.

MPR_0023: The request for a GUID comes with strong implications. This requirement currently does not hold for all systems and domains.

MPR_0078: An architecture cannot guaranty that all data is processed. It can eventually support

the processing of data by means of a watchdog.

MPR_0079: An architecture cannot guaranty to execute all defined workflows. It can eventually

support the execution by means of a watchdog.

Discarded requirements

MPR_0074: A reference architecture or MANTIS cannot solve this request at all.

MPR_0075: MANTIS does not develop any hardware with respect to its reference architecture

MPR_0076: MANTIS does not develop any hardware with respect to its reference architecture

The following subsections will derive functional and quality goals from the above listed requirements.



2.4 Key Functional Goals

Table 2-6 describes the key functional goals for MANTIS, derived from the MANTIS platform requirements listed in chapter 2.3.

Table 2-6: Key Functional Goals of MANTIS ARM

Key Function ID

Title Traces to

MARMKF_001 Extract, collect, and measure maintenance related data of system of interest, processes and environment

MPR_0001;MPR_0002;MPR_0004;MPR_0022;MPR_0045;MPR_0046;MPR_0047

MARMKF_002 Transform, manipulate and aggregate maintenance related data on each tier of the architecture

MPR_0003;MPR_0005;MPR_0010;MPR_0012;MPR_0013;MPR_0022;MPR_0024;MPR_0085

MARMKF_003 Transmit all maintenance related data between system parts, HMIs, data storages and analysis services

MPR_0011;MPR_0042;MPR_0112

MARMKF_004 Retain and log all maintenance related data MPR_0006;MPR_0023;MPR_0056

MARMKF_005 Store and back-up maintenance related data on each tier of the architecture

MPR_0023;MPR_0024;MPR_0026;MPR_0056

MARMKF_006 Restore maintenance related data MPR_0025;MPR_0056

MARMKF_007 Secure maintenance related data on each tier of the architecture

MPR_0012;MPR_0013;MPR_0085

MARMKF_008 Secure maintenance related data on all transmission paths MPR_0086

MARMKF_009 Monitor maintenance related data on each tier of the architecture locally and remotely

MPR_0016;MPR_0060

MARMKF_010 Analyse distributed maintenance related data for real time scenarios

MPR_0017;MPR_0018;MPR_0020

MARMKF_011 Analyse distributed maintenance related data for non-real time scenarios

MPR_0017;MPR_0019;MPR_0020

MARMKF_012 Control all kind of access and usage of maintenance related data on each tier of the architecture locally and remotely

MPR_0007;MPR_0008;MPR_0009;MPR_0014;MPR_0081;MPR_0082;MPR_0083;MPR_0084;MPR_0107;MPR_0108;MPR_0109;MPR_0110;MPR_0111

MARMKF_013 Handle simple and complex maintenance related events MPR_0034

MARMKF_014 Notify on maintenance related events via local and remote alert components and network based messaging systems

MPR_0036

MARMKF_015 Configure maintenance related event mechanisms MPR_0035;MPR_0037;MPR_0038;MPR_0039



Key Function ID

Title Traces to

MARMKF_016 Control, operate and configure maintenance related operating, actuating and sensing system components locally and remotely

MPR_0036;MPR_0041;MPR_0043;MPR_0044;MPR_0071;MPR_0072

MARMKF_017 Control, operate and configure maintenance related data transmissions

MPR_0042

MARMKF_018 Check maintenance related data compliance MPR_0048



2.5 Quality Goals

In the following tables the quality goals for MANTIS, derived from the MANTIS platform requirements (listed in chapter 2.3), are denoted:

Table 2-7: Quality Goals for MANTIS

Driver MARMQG_001.Adaptability :: Adaptability for different environments

Environment & Stimulus

The MANTIS system is up and running; A new MANTIS system part is going to be installed

Response The system is able to adapt to different system environments without influencing other system parts

Priority High

Traces to

Driver MARMQG_002.Adaptability :: Adaptability for new components


The MANTIS system is up and running; A new MANTIS system component is going to be installed

Response The system is able to adapt to different system components without influencing other system parts

Priority High

Traces to

Driver MARMQG_003.Adaptability :: Use of preferred technologies


The MANTIS system is up and running; Users of the MANTIS system bring their own technologies

Response The MANTIS system offers interfaces that technologies of all kind can connect to via an adapter

Priority High

Traces to

Driver MARMQG_004.Availability :: Planned downtimes of system parts


The MANTIS system is up and running; For maintenance reasons parts of the system are offline

Response The system parts offline still collect the maintenance related data they generate, buffers it and transmits it when online again

Priority Medium

Traces to MPR_0021;MPR_0055;MPR_0057;MPR_0058;MPR_0059;MPR_0073



Driver MARMQG_005.Availability :: Unplanned downtimes of system parts (software)


The MANTIS system is up and running; Software components of the system fail

Response The system does not crash The system notifies the system operator immediately All non-affected data generators buffer their data

Priority Medium


Driver MARMQG_006.Availability :: Unplanned downtimes of system parts (hardware)


The MANTIS system is up and running; Hardware components of the system fail

Response The system does not crash The system notifies the system operator immediately

The system’s software adjusts to the new situation until the hardware failure is resolved

Priority Medium


Driver MARMQG_007.Availability :: Concurrent access on data


The MANTIS system is up and running; Several users/components of the MANTIS system access data concurrently

Response The system provides the data to the requesting users/components concurrently

Priority High

Traces to MPR_0015

Driver MARMQG_008.Robustness :: Robustness against unstable network conditions


The MANTIS system is up and running; Limited bandwidth or unstable network conditions result in slow data transmissions

Response The system does not crash No data is lost Data is transmitted with delay

Priority High

Traces to MPR_0021

Driver MARMQG_009.Robustness :: Robustness against no network conditions


The MANTIS system is up and running; No network conditions result in no data transmissions

Priority High

Traces to MPR_0021



Driver MARMQG_010.Safety :: Monitored processes and sites are not influenced w.r.t. safety


The MANTIS system is up and running; MANTIS system components are changed or new components are integrated into the system

Response The system must not reduce the SIL level of the system monitored

Priority High

Traces to MPR_0074

Driver MARMQG_011.Scalability :: Scaling over large number of data generating devices


The MANTIS system is up and running; One or more sites are integrated into the system

Response The system has to scale in a way that it does not need to comprise its performance. However, this goal may only be reached by upgrade other system components in parallel, like network performance

Priority Medium

Traces to

Driver MARMQG_012.Security :: Separation of views for different MANTIS users


The MANTIS system is up and running; A MANTIS user accesses data from the MANTIS system

Response Depending on the user’s role and location, the MANTIS system adapts the access rights and data views

Priority High

Traces to

Driver MARMQG_013.Security :: Overall data usage control


The MANTIS system is up and running; A MANTIS user accesses data from the MANTIS system

Response Depending on a usage agreement, the MANTIS system adapts the access rights and data views

Priority High

Traces to

Driver MARMQG_014.Security :: Security levels and zones


The MANTIS system is up and running; Data is transmitted between MANTIS components

Response Depending on the transmission paths and zones the MANTIS system adheres to different security rules

Priority Medium

Traces to MPR_0087;MPR_0088



Driver MARMQG_015.Upgradeability :: No data loss during upgrade of system components


The MANTIS system is up and running; Parts of the system are upgraded with new software/firmware versions

Response The system stays up and running No maintenance data is lost No configuration data is lost

Priority Medium

Traces to



2.6 Driver Prioritization

Table 2-8 depicts the prioritization of the MANTIS quality goals. There are 9 goals with high priority, and 6 goals with a medium priority.

Table 2-8: Prioritization of MANTIS quality goals

MARMQG_001.Adaptability Adaptability for different environments High

MARMQG_002.Adaptability Adaptability for new components High

MARMQG_003.Adaptability Use of preferred technologies High

MARMQG_007.Availability Concurrent access on data High

MARMQG_008.Robustness Robustness against unstable network conditions High

MARMQG_009.Robustness Robustness against no network conditions High

MARMQG_010.Safety Monitored processes and sites are not influenced w.r.t. safety High

MARMQG_012.Security Separation of views for different MANTIS users High

MARMQG_013.Security Overall data usage control High

MARMQG_004.Availability Planned downtimes of system parts Medium

MARMQG_005.Availability Unplanned downtimes of system parts (software) Medium

MARMQG_006.Availability Unplanned downtimes of system parts (hardware) Medium

MARMQG_011.Scalability Scaling over large number of data generating devices Medium

MARMQG_014.Security Security levels and zones Medium

MARMQG_015.Upgradeability No data loss during upgrade of system components Medium

2.7 Maintenance Related Standards

Deliverable D1.4 lists all Maintenance related standards. This document especially takes ISO13374, parts 1-4 into account.



3 System Scope, Context and Maintenance Domain

This chapter gives a first overview on the MANTIS system planned. Section 3.1 denotes its context and

environment, and section 3.2 gives an overview on different maintenance types and actions as well as

connected concepts. Section 3.3 introduces a feature model, which provides key concepts to characterize common and varying aspects in the architectures to be derived from the reference architecture. Section 3.4 describes general architecture principles and constraints being considered during the design of the MANTIS reference architecture. Section 3.5 outlines the approach followed to derive the MANTIS reference architecture from the various drivers.



3.1 System Scope and Context

The MANTIS platform must support the interplay between the different stakeholders of a complete maintenance process. Its main purpose is to process and transmit maintenance related data, and to make

controlling and securing of data flows possible.

Figure 3-1 shows activities directly related to the MANTIS platform, components and stakeholders interacting with it.

Figure 3-1: Context of MANTIS and the MANTIS Platform

The MANTIS Platform should support the following main features:

- Extract, Transform, Load (ETL) Architecture Pattern - Root Cause Analysis (RCA) - Remaining Useful Life (RUL) evaluations - On-site and remote edge device management (e.g. calibrate, configure, parameterize sensors)

uc [package] Context [Context]

MANTIS System

«block»Sensor

«block»HMI

«block»Business Case

«block»COM

«block»PdM Data

Storage

«block»Analysis

Maintenenance service provider

HMI technology provider

Data analysis provider

Data provider

Sensor technology provider

Solution provider

RUL RCAETL

DA/DM/SD HA/PA/AG

Data Buffering

Data Management

Maintenance Execution

Security Management

Maintenance Optimisation

Collaborative Decision Making

Event Handling

(Remote) Edge Device Management

PdM Data

«flow»

Analysis Data

«flow»

«include»

Control Data

«flow»

Analysis Data,

PdM Data

«flow»

Analysis Data,

PdM Data

«flow»

PdM Data ,

Visualization

Data

«flow»

Control Data

«flow»

SensedData

«flow»



- ISO13374 function chain: Data Acquisition (DA), Data Manipulation (DM), State Detection (SD), Health Assessment (HA), Prognostic Assessment (PA), and Advisory Generation (AG)

o Incl. Data Buffering

- General Data Management

- Event Handling - Security Management

- Maintenance Optimisation - Maintenance Execution - Collaborative Decision Making

The MANTIS platform will have to interact with the following components:

- Sensors of all kind that send data and receive configuration data, where applicable

- HMIs showing the current maintenance related situation and sending control signals to machinery under control

- Maintenance service providers will have to retrieve and send data via the platform to enable a distributed analysis of data from different origins

- All kind of communication will take place via the platform - The platform must enable new and variable business models, which use data from different origins

in different places for analyses and storing



3.2 Maintenance Domain Model

The main purpose of the MANTIS platform is to make predictive maintenance possible. Therefore, the MANTIS domain on the top level comprises all maintenance related actions, and components needed to

support maintenance this is illustrated in the figure below. The MANTIS platform must be able to connect the components in play, and to support the maintenance actions.

Figure 3-2: Domain Model for MANTIS with Maintenance as central topic

Below a brief explanation on the various maintenance type definitions as they have been used in the context of the architecture.

Reactive maintenance

Maintenance performed when a functional structure, system or component has already entered a fault condition (break/fix) and the user has contacted the support organisation.



Preventive maintenance

Maintenance performed according to a fixed schedule, or according to a prescribed criterion, that detects or prevents degradation of a functional structure, system or component, in order to sustain or extend its

useful life

Proactive maintenance

Maintenance performed when a functional structure, system or component has already entered a fault condition (break/fix) and the user has not contacted the support organisation yet.

Predictive maintenance

Maintenance performed as governed by condition monitoring programmes.



3.3 MANTIS Feature Model

A feature model is a model that defines main characteristics (and their dependencies), typically in the form of a feature diagram. A “ feature” is defined as a “ prominent or distinctive user-visible aspect,

quality, or characteristic of a software system or system” [8]. The focus is on the systematic and efficient reuse of similar system parts.

Several MANTIS scoping workshops have been conducted to identify relevant features in the predictive

and proactive maintenance domain. The resulting feature model has been used to characterize and to compare the different MANTIS use cases for their commonalities and differences. The MANTIS Feature Model is outlined in the following tables. Some rather detailed features are hidden for the sake of brevity.

The features in the Feature Model are hierarchically structured. The indentation level in the Feature

column represents the hierarchy level. Descriptions have been provided where the feature is not self-explaining.

Table 3-1: MANTIS Feature Model Excerpt I

Feature Description

PM Ecosystem Characterisation of the PdM ecosystem

Goal Business goals in the PdM ecosystem

Increased uptime

Improved downtime management

Facilitate maintenance planning

Increased equipment lifespan

Increased performance

Increased operational efficiency

Improved product quality

Increased customer satisfaction

Maintenance as a service

Maintenance cost reduction (operational or end-user)

Advanced Maintenance HMI

Maintenance Assistance Systems

Constraints Pivotal constraints that influence PdM solutions in the application domain

Data usage regulations

Data ownership regulations

Players Players in the PdM ecosystem

Asset Manufacturer

Asset Owner / Asset End User

Asset System Provider

Component Supplier

Data Analyst

Data Provider

HMI Technolology Provider

Maintenance / Asset Service Provider

MANTIS Integrator



Table 3-2: MANTIS Feature Model Excerpt II

Feature Description

SOI System of interest for PdM solution

Kind Kind of system

Product in focus

Production process in focus

No of SOI instances How many SOI instance are monitored?

Different operation environment Does the SOI operation environment differ signficantly? Must the operation environment be considered in PdM?

Location Where are the SOIs operated?

Mobile system Is the SOI mobile?

Fleets Are SOI fleets monitored?

Characteristics of interest Which characteristics in the SOI are of interest?

Function What main functions does the PdM solution comprise?

Data aquisition (DA)

Data sources From which data sources is data acquired?

Sensors

Sensor Data representation

Operation data

Environmental data

Other data

Data manipulation (DM)

Intelligent functions

Self-testing

Self-identification

Self-validation

Self-adaptation

Intelligent firmware update

Fusion

Eliminating noise and erroneous data

Stochastic methods

Kalman filtering

Fuzzy logic

Logical links

Rule-based methods

Normalising

Feature extraction Organising and extracting the important information of the signals in order to perform an adequate analysis of the data

Trend Analysis

Preprocessing

Bayesian Observer



Table 3-3: MANTIS Feature Model Excerpt III

Feature Description

State detection (SD)

Tool measurement

Video-based

Health assessment (HA)

RCA

Pre-Processing

Modelling

RUL

Knowledge based models

Life expectancy models

Artificial Neural Networks

Physical Models

Failure detection and prediction

Maintenance optimization

Collaborative decision making

Prognostic assessment (PA)

Advisory generation (AG)

Maintenance planning (MP)

Maintenance execution (ME)

Collaborative maintenance Is there a distributed decision making the in PdM solution?

Lifecycle Management (LM)

Security Management (SeM)

Safety Management (SaM)

HMI Are there HMI-parts that (could) use MANTIS HMI solutions

Scheduling Maintenance Tasks

Monitoring Production Assets

Data Analysis

Reporting

Communication

Service improvement

Data Processing Style Which data processing styles are applied?

Stream Processing

Batch Processing

Data Management What data is managed how?

SQL Is data managed in a SQL-like manner?

Non-SQL data Is non-SQL data managed?

Representation How is the data represented?

MIMOSA Is MIMOSA used for data storage / exchange?

Other Which other data representations are used?

Bandwidth Optimization



3.4 Architecture Principles and Constraints

This section describes general architecture principles and constraints, which have been considered during the design of the MANTIS reference architecture.

3.4.1.1 Keeping it simple & stupid

While designing the reference architecture, we will prefer simple solutions over complex ones and want to

be as practical as possible. To avoid unnecessary content, we will focus on addressed concerns and added value for architecture stakeholders. Furthermore, we will try to avoid repetition where possible.

3.4.1.2 Re-use instead of re-invention

We consider existing solutions and best practices and reuse them where possible. To this end, we rely on state-of-the-art and – practice approaches, including standards, results of related projects. We try to avoid

the not-invented-here syndrome, be reflecting about the necessity to deviate from existing solutions and provide the underlying rationale where possible to make the design results as understandable as possible.

3.4.1.3 Guidance instead of constraints

The provided reference architecture is to provide guidance on how to instantiate an architecture for a particular PdM domain or specific task within MANTIS and beyond. It raises awareness for critical design

decisions and ensures consistency and applicability of technologies, interoperability mechanisms, data formats and models, data analysis tools, etc. However, it does not intend to enforce a certain solution approach via respective constraints. Use cases should be able to deviate from the reference architecture where needed. Deviations will be identified and analysed and can drive necessary improvements of the

reference architecture in the next iterations.

3.5 Design Approach

This section outlines the approach followed to derive the MANTIS reference architecture from the various

drivers. The overall design approach followed to this end can be characterized as Bottom-Up - Top-Down.

In the Bottom-Up phase, pre-existing solutions (aka. brown field use cases) within the MANTIS partners have been investigated for the relevance of the various drivers, the followed solutions, considered alternative solutions, and the experiences made so far. From this kind of scoping, a first feature model has been created and all use cases have been characterized along the feature model. The necessary

generalization of the different concepts (esp. functions) has been mainly driven by the ISO13374 standard

and MIMOSA.

In the Top-Down phase, reasonable solutions for the key drivers have been designed and documented in three iterations. The conducted design activities in the iterations comprised the selection of key drivers,

the identification of generalized solution alternatives (from the brown field use cases, State-of-The-Practice CPS architectures), the characterization of solution alternatives along the overall goals for trade-offs, taking or constraining design decisions, and the documentation of the considered solutions and design

decisions. New insights from the use cases along the driver solutions have been added over time in later iterations.



4 Reference Model (Functional Perspective)

The MANTIS reference model (functional perspective) constitutes support for the understanding and the

definition of expected functionality and behaviour of system components and related design decisions. The purpose of the functional perspective is to scope and to define functionality and information of

MANTIS-based PdM solutions. This perspective documents the functional structure including key functional elements, the interfaces they expose and the interactions between the functional elements. The included functional blocks have been derived from the key drivers (esp. functional and quality goals) as well as the functions identified in the bottom-up analysis in MANTIS use cases. It is essential that the

functional perspective is limited to the stakeholder needs and requirements, and excludes any

implementation choices.

4.1 Foundations

The international standards “ Condition monitoring and diagnostics of machines – Data processing, communication and presentation – “ ISO 13374 [9], “ Internet of Things Architecture” IoT-A [5] and

“ The Industrial Internet of Things Reference Architecture” IIRA [10] constitute the foundations of the MANTIS reference model.

4.1.1 Condition monitoring and diagnostics of machines

ISO 13374 [9] describes general guidelines related to data processing, communication, and presentation of condition monitoring and diagnostics of machines. Data processing is divided in six processing blocks: Data Acquisition (DA), Data Manipulation (DM), State Detection (SD), Health Assessment (HA),

Prognostic Assessment (PA) and Advisory Generation (AG).

Data Acquisition (DA) converts output from data source to digital parameters representing physical quantities and related information. Data Manipulation (DM) transforms the digital signals to meaningful data. State Detection (SD) controls normal baseline state profiles, searches for abnormalities and determines in which abnormality zone data belongs (e.g. “ alarm” ).

Health Assessment (HA) diagnoses faults and classifies current health of machine equipment and/or

process. Prognostic Assessment (PA) considers current health assessment and projects usage loads on the machine equipment and/or process to predict future health states, failure modes and remaining useful life. Advisory Generation (AG) creates advices for maintenance or operational changes in order to optimize the life of the machine equipment and/or process.

In addition to these six data processing blocks, the standard outlines guideline for information presentation, external systems (e.g. Decision Support), data archiving and block configuration. Its conceptual

information schema defines single integrated guidelines concerning machinery and condition monitoring information.

In the display domain, the standard describes five areas. State detection is the presentation of state

information concerning the observed monitoring situation (Condition Indicator). Health assessment summarizes the results of human or automated current health analyses (current Condition and Health Status). Prognosis contains prognosis-specific information (predicted Condition and Health Status). Recommended actions lists the recommended actions to be taken (Recommendation and Symptom of

Fault). Identification describes the identification of machinery equipment and/or processes.



Figure 4-1 Conceptual Model on condition monitoring

4.1.2 Internet of Things – Architecture

Internet of Things (IoT) describes a conglomeration of solutions related to the world of interconnected and intercommunicating smart objects and devices. The IoT-Architecture (IoT-A) is an established international standard [5] with the goal, to provide a foundation to emerge from purely vertical and

isolated solutions to a full fledge Internet of Things, where IoT applications share common technology and common architectural principles. It enables interoperability capabilities for IoT solutions and covers different application fields with various development cycles and technologies. IoT-A addresses scalability requirements of a future full fledge Internet of Things in terms of communication between and the

manageability of smart objects and devices.

4.1.3 Industrial Internet of Things Reference Architecture

The Industrial Internet of Things Reference Architecture (IIRA) is an international standard [10] and

provides an architectural template to design concrete architectures for Industrial Internet of Things (IIoT) systems. This approach shall guarantee consistent architecture implementations across different use cases

class Conceptual Model (Condition Monitoring)

Condition Monitoring

Health Assessment

Data Acquisition

Data Manipulation

State Detection

Prognostic Assessment

Advisory Generation

Health Status

Stable

Critical

Critical but stable

Fair

Good

Serious

Serious but stable

Condition Indicator

Failure Condition

Fault Condition

Recommendation Symptom of Fault

Undetermined

Data Source Decision Support

Condition«flow»

estimates

detects

«flow»

diagnoses

calculates

«flow»

predicts

generates

«flow»

«flow»

supports

predicts

«flow»



in various industrial sectors. Furthermore, it shall support reaching a common understanding and communication of the overall system among its stakeholders. IIRA constitutes an architecture framework with four viewpoints: business, usage, functional and implementation. The business viewpoint addresses the concerns of the identification of stakeholders, the business vision, values and objectives in the business

context of the IIoT system. The usage viewpoint describes expected system usage. The functional viewpoint is concerned with the functional components of the IIoT system. The implementation viewpoint deals with implementation choices like technology, communication schemes and lifecycle procedures. In complement to the architecture framework, IIRA provides guidelines for functional decomposition by defining functional domains. The control domain addresses functions performed by industrial control

systems (sensing, actuation). The operations domain is responsible for provisioning, management, monitoring and optimization of CPS in the control domain. Functions in the information domain gather

data from various domains in order to acquire intelligence about the overall system. The application domain provides business functionality and functions in the business domain enable integration and end-

to-end operations of IIoT systems like Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), Product Lifecycle Management (PLM) or Manufacturing Execution System (MES).



4.2 Information View [Information Model]

The MANTIS Information Model defines the structure (e.g. relations, attributes, services) of all the information on a conceptual level. The term information is used along to the definitions of the DIKW

(Data, Information, Knowledge, and Wisdom) hierarchy (see [11]) where data is defined as pure values without relevant or useable context. Information adds the right context to data and offers answers to

typical questions like who, what, where and when. The description of the representation of the information (e.g. binary, XML, RDF etc.) and concrete implementations are not part of the MANTIS Information Model.

4.2.1 Definition of the MANTIS Information Model

The following figure gives an overview to relevant data characteristics in the MANTIS information model. The included characteristics have been identified in the course of the MANTIS requirements activities and the scoping of the MANTIS use cases. The characteristics are elaborated in the following

sections.

Figure 4-2 MANTIS Information Model Element Categories

bdd [package] InformationModel [IM_Overview]

PdM Data

DataCharacteristic

+ Accuracy

+ Asynchronous

+ Frequency

+ Latency

+ Resolution

+ Synchronous

DataContent

+ Advice

+ Event

+ GeoData

+ HealthStatus

+ HealthTrend

+ Image

+ MaintenancePlan

+ MaintenanceReport

+ Sound

+ State

+ Video DataProcessing

+ AggregatedData

+ AugmentedData

+ ManipulatedData

+ RawData

DataSource

+ ManualData

+ SensedData

DataStructure

+ Stream

+ Value

+ ValueContainer

DataSubject

+ AssetDescription

+ ERPData

+ MachineData

+ ProcessData

DataKind

+ Information

+ Knowledge

+ MetaData



Data in MANTIS can be differentiated in the following kinds: Data: syntactically treated signals or signs Metadata: data that provides information about other data Information: semantically enriched data, regular structured, aggregated and augmented

Knowledge: observations that have been meaningfully organized, accumulated and embedded in a context through experience, communication, or inference. It is used to interpret situations and to generate activities, behaviour and solutions.

The data will be structured in single values, value containers, or data streams.

Figure 4-3 MANTIS data structures and kinds

The data in MANTIS on the one hand side consists of somehow sensed data for determining the condition

of assets monitored, and on the other hand it comprises manual data from human actors in the maintenance process. Within the process of predicting maintenance issues the data is captured as raw data and successively enriched with syntactic, semantic and experience based increments.

bdd [Package] InformationModel [IM_DataKind&DataStructure]

Data Information Knowledge

MetaData

StreamValue ValueContainer

Structure



Figure 4-4 MANTIS data sources and processed data

Data within MANTIS is covering many differing kinds of content ranging from simple to complex, and comprises four different subjects:

class IM_DataProcessing&DataSource

AggregatedData AugmentedData

Data

ManipulatedData

ManualData

RawData

SensedData

Source

Processing



Figure 4-5 MANTIS data subjects and content

Data handled by MANTIS, depending on its origin, can be assigned a meaningful sample of characteristics, which are shown in the figure below:

Figure 4-6 Data characteristics within MANTIS

bdd [Package] InformationModel [IM_DataContent&DataSubject]

Data

VideoSound MaintenanceReport

MaintenancePlanImage

Image2D Image3D

HealthTrend

HealthStatus

GeoData

Event Advice

State

ERPDataAssetDescription MachineDataProcessData

Content

Subject

class IM_DataCharacteristics

Accuracy Asynchronous

Data

Frequency LatencyResolution Synchronous

Characteristic



4.2.2 Other information-related models in MANTIS

The kind of data to be handled within MANTIS has been elicited for deliverable D1.4 MANTIS Platform Requirements [12]. The following figure (an excerpt from D1.4 input) gives the summarisation from all

platform relevant requirements that deal with data.

Figure 4-7 MANTIS data description as consolidated from partner input in task 1.3



4.3 Functional View

The functional view defines functional capabilities of a MANTIS system. This view describes what a MANTIS system is required to do on an abstract level. Key functions have also been included in the

MANTIS Feature Model.

4.3.1 Functional Concept

In the MANTIS project, Functional Decomposition (FD) refers to the process by which the different Functional Components (FC) that make up the MANTIS Reference Architecture are identified and related to one another. The conceptual functional decomposition is outlined in Figure 4-8. Here each key and

crosscutting function of MANTIS is assigned to a functional domain (Application, Maintenance, Communication, Device, Management, and Security).

The MANTIS functional concept results from the extension of the ISO13374 functional elements with maintenance planning and execution, combined with typical functions of CPS and IoT systems taken from the IoT-A reference model.

Figure 4-8 MANTIS Functional Model

In general, different types of maintenance applications can be built based on the functional components

of MANTIS. The relation of applications types and maintenance functional components is given in the following table.



Table 4-1 Functions vs. Maintenance Type

Reactive Maintenance

Proactive Maintenance

Predictive Maintenance

Adaptive Maintenance

Data Acquisition X X X X

Data Manipulation X X X X

State Detection X X X X

Health Assessment (X) X X X

Prognostic Assessment X X X

Advisory Generation X X

Maintenance Planning X X X

Maintenance Execution X X X X

The various function concepts are explained in more detail in the following section.

4.3.2 Function Tree

The following section lists the key functions of the MANTIS architecture (cf. Figure 4-9). Most of them are derived from ISO13374. A short explanation of the functions is given hereafter to make the reader understand their basic tasks.

Figure 4-9 Function Tree for MANTIS PdM

bdd [package] BlockView [Function Net]

Data Acquisition (DA)

Data Manipulation (DM)

State Detection (SD)

Predictive Maintenance (PdM)

Health Assessment (HA)

Prognostic Assessment (PA)

Advisory Generation (AG)

Maintenance Planning (MP)

Maintenance Execution (ME)

Data Management (DN)






Data Acquisition (DA)

The output which is obtained from the sensor is converted into the digital parameter, which represents a physical quantity and information related to e.g. time, calibration, the quality of the data, the data collector utilized and the sensor configuration [13]

Functionality: The Data Acquisition (DA) function provides system access to digitized data entered automatically or manually.

Inputs: The DA function may represent a specialized data acquisition function that has analogue feeds (e.g. from legacy sensors), or it may collect and consolidate sensor signals from a data bus. Alternatively, it might represent the software interface to a smart sensor.

analogue, manual, and digital data

control, synchronization, and configuration data

historical DA outputs

Outputs: The DA function is basically a server of calibrated/scaled digitized sensor data records. The output of all DA function blocks shall contain the following:

digitized data

time-order/time-reference data, normally referenced with UTC and local time zone

data quality indicator (e.g. “ bad” , “ good” , “ unknown’ ’ , “ under review’ ’ , etc.).

Data Manipulation (DM)

Data manipulation unit performs signal analysis and calculates the relevant descriptors. The end result is a virtual sensor reading from raw data. [14]

Functionality: The Data Manipulation (DM) function processes the digital data from the DA function to convert it to a desired form which characterizes specific descriptors (features) of interest in the machine condition monitoring and diagnostic process. Often the functionality within this layer consists of some signal processing algorithms. DM calculates descriptors (features) from sampled sensor data, other descriptors, or the output of computations. The computation may be characterized as an input-output mapping.

Inputs: Sampled digital data from the DA function, and cascaded data from other DM instances.

Outputs: Descriptors (features) from sampled sensor data, other descriptors, or the output of computations. The computation may be characterized as an input-output mapping.

State Detection (SD)

State Detection (SD) block facilitates the creation and maintenance of normal baseline ”profiles” and searches for abnormalities from the data whenever the new data is collected and determines the abnormality zone to it, if any, for example, an alert or alarm. [13]



Functionality: The State Detection (SD) function (sometimes referred to as “ state awareness” ) is to compare DM and/or DA outputs against expected baseline profile values or operational limits, in order to generate enumerated state indicators with respective boundary exceedances. The SD function generates indicators, which may be utilized by the Health Assessment function to generate alerts and alarms. When appropriate data are available, the SD block should generate assessments based on operational context, sensitive to the current operational state or operational environment.

Inputs:

current DA and DM outputs; cascaded SD output

historical DA and DM outputs

operational data (context, environment, state, external systems data)

configuration data

Outputs:

DA control signals and scheduling commands (for optimizing functional interplay)

DM control signals (for optimizing functional interplay)

Data which will contribute to a diagnosis in the health assessment function

Health Assessment (HA)

The Health Assessment (HA) block diagnoses any faults and rates the current heath of the equipment or process, considering all state information. [13]

Functionality: The Health Assessment (HA) function utilizes expertise from a human or automated agent to determine the current health of the equipment and to diagnose existing fault conditions. It determines the state of health and potential failures by fusing the outputs of the DA, DM, SD and other HA function blocks. HA performs agent-specific assessments of a component’ s or system’ s current health state with the associated diagnoses of discovered abnormal states in the associated operational context. HA results may also include evidence and explanation information.

Inputs:

DA, DM, SD outputs

operational data

configuration data

human expertise

automated agent expertise

Outputs:

component/system’ s current health grade

diagnosed faults and failures with associated likelihood probability

calculation of the current risk priority number (RPN)

modelling of ambiguity groups and multiple hypotheses may be included in the output data

structures

explanation detailing the evidence for a diagnosis or health grade



Prognostic Assessment (PA)

Prognostic Assessment (PA) block determines the future health states and failure modes of the equipment and/or process based on the current health assessment and projected usage loads, as well as the remaining useful life predictions. [13]

Functionality: Prognostic Assessment (PA) performs agent-specific assessments of a component’ s or system’ s future health state with the associated predicted abnormal states and remaining life for a projected operational context. It may also include evidence and explanation information. It uses a combination of prognostic models and their algorithms, including future operational usage model(s). Remaining useful life (RUL) is a typical characteristic to be prognosed. Inputs:

DA, DM, SD, HA and (cascaded) PA outputs

account historical failure data and operational history

projected failure rates related to operational utilization

Outputs:

health grade at a future time

estimation of the remaining life of an asset given its projected usage profile

Advisory Generation (AG)

Advisory Generation (AG) block offers actionable information regarding maintenance or operation of an equipment and/or process so that the lifetime of the equipment and/or process can be optimized. [14]

Functionality: Advisory Generation (AG) integrates information (including safety, environmental, operational goals, financial incentives, etc.) to generate advisories to operations and maintenance and to respond to capability forecast assessment requests.

Inputs:

DA, DM, SD, HA, PA and (cascaded) AG outputs

operational data

configuration data

external constraints (safety, environmental, budgetary, etc.)

operational history (including usage and maintenance)

current and future mission profiles

high-level unit objectives

resource constraints

Outputs:

recommendations, such as

o prioritized operational and maintenance actions

o capability forecast assessments

o modified operational profiles to allow mission completion



maintenance advisories (put as structured work requests)

o verification of monitoring data

o performance of additional monitoring

operational advisories

o immediate (e.g. notification of alerts and action steps)

o strategic (e.g. notification of (high) risk of failure)

capability forecast

Data Management (DN)

Data Management (DN) block offers necessary functions to control, protect, deliver and enhance the value of data and information assets.

Functionality: Data Management (DN) manages all kind of data in a distributed scenario. It adheres to security and safety qualities given.

Inputs:

Distributed, structured, semantically tagged, digitized data

Authentication and authorization rules and patterns

Contracts

Guarantees

Outputs:

Data stored

Data handles

Maintenance Planning (MP)

Maintenance Planning (MP) block offers necessary functions to plan sound maintenance activities.

Functionality: Maintenance Planning (MP) prepares a plan for an optimized immediate and strategic maintenance based on current and historic health data, as well as resource planning data.

Inputs:

AG outputs

Resource planning data

Outputs:

work plan

Maintenance Execution (ME)

Maintenance Execution (ME) block offers necessary functions to execute planned maintenance activities.



Functionality: Maintenance Execution (ME) executes the maintenance plan.

Inputs:

MP outputs

Templates

Guidelines

Constraints

Outputs:

Report on the outcome


Lifecycle Management (LM) block offers functions to manage the different PdM system elements along their lifecycle.

Functionality: Lifecycle Management (LM) manages the lifecycle of the system’ s assets


Security Management (SeM) supports security, trust and privacy in the following scopes [Industrial Internet Consortium (2015)]:

endpoint security

communication security between the endpoints

management and monitoring security (endpoints and communication mechanisms)

data distribution and secure storage

The end-to-end security capability should span functional domains, the information technology and operational technology without interfering with the operational business processes


Safety mechanisms include, but are not limited to [Industrial Internet Consortium (2015)]:

Support for independent functional safety features

Well-defined, verified and documented interfaces

Enforceable separation of disparate functions and fault containment

Runtime monitoring and logging

Safety Management is not in scope of MANTIS and is not further elaborated in this document.



5 Reference Architecture (Logical Perspective)

This chapter decomposes the overall MANTIS system into several logical building blocks or components.

This decomposition is done hierarchically, where necessary. Functionalities are mapped to the logical

building blocks. This chapter provides first a general overview on the reference architecture, where the focus is on the structural decomposition, and then discusses quality-related design aspects which are mapped to the key functional goals as identified in 2.4. This is described for the various elements such as Data processing, Interoperability, Lifecycle Management and security. Furthermore it introduces the reference application concept which helps in understanding key differentiators between use cases.

5.1 Overview

The MANTIS reference architecture has been created by analysing commonalities and variabilities of the specific use case architectures, abstracting realization details and adding solutions for missing key functions

and quality aspects. The architectural concepts which have been explored in order to arrive at the

presented architecture have been captured in the Appendix in the sections on Key Architectural Concepts & Architectural Styles & Patterns

Figure 5-1 provides an overview on the MANTIS reference architecture from the logical, i.e. technology-independent, perspective.

Figure 5-1 Overview on MANTIS Reference Architecture

The following sections describe the various design decisions which are taken into account for the architecture definition. The rationale which is used for these decisions is described in more detail and can

be found in the Appendix: Design Decision Rationales

Design Decision: Main solution structure

MARMKF_002,, MARMKF_009

The architecture follows the Three Tier architecture pattern (edge, platform, enterprise), which is typical for CPS / IOT and big data solutions. It assures a clear separation of concerns: PdM data about the system of interest is collected and pre-processed in the edge tier. Data management and processing

happens mainly in the platform tier and the enterprise tier comprises applications, which work mainly on the data provided by the platform tier. Considered design alternatives include Peer-To-Peer and Pipe-And-Filter.

Edge Tier Platform Tier Enterprise Tier

Physical Entity

Edge broker

Cyber entity

Stream processor

Physical Entity

Cyber entity

Batch processorLocal

storage

Data Exchange

Edge Gateway

Man

age.

Secu

rity

Comm.

P2M

Intell.Sensor Edge

Management

SecurityData

Storage

Data Management

CloudManagement

Analysis App

ManagementApp

Service Management App

Edge IDE

Maint. planner Service Execution

App



Design Decision: Edge-to-Platform Communication

MARMKF_003, MARMKF_007, MARMKF_008

The interaction between the Edge Tier and the Platform Tier is mediated via a dedicated gateway (cf.

Edge-gateway-mediated pattern). Two alternatives have been considered: (i) Edge-gateway-mediated, where sensors are not directly connected to the MANTIS platform and the communications of the data extracted from the physical entities to the cloud and vice-versa are possible by using an edge gateway;

and (ii) cloud-based architectural pattern, where sensing devices are capable to connect themselves to the platform to push data extracted from the physical entities and to eventually receive data from the platform services. Edge-gateway-mediated was used, as a considerable amount of management and data processing functionality has to be rendered in the edge. Putting this on each sensor node would raise the costs of

the solution significantly.

The edge-based architectural pattern addresses the problem on how the edge architecture should be. It is the most suited approach whenever there are resources that cannot be connected to the internet or whenever performance of services is a critical issue. Also in this scenario, there are several patterns that can be used by considering the specific application scenario.

As performance, security, and lifecycle management are identified as key drivers in MANTIS, we follow

the “Mediator” pattern – that belongs to the “Master-Slave” main category. According to this pattern, an

entity is used to manage and handle the way a set of CPS are interacting. This pattern promotes loose coupling by preventing direct communication between CPSs keeping the overall complexity at a low(er)

level. Therefore, the “Mediator” encapsulates the collective behaviour of the edge.

Design Decision: Separate gateway from asset

MARMQG_010

Some of the use cases separate the maintenance functionality from the monitored SOI (System Of Interest) to avoid any side effects of the PdM on the SOI overall performance. This is especially of interest in regulated application domains as health care for instance. This separation can be achieved by coupling

the SOI and the edge gateway via a shared local storage. With this, the gateway only can access data, which is provided by the SOI in a deliberate manner and freedom of interference is assured.

Design Decision: Platform as a cloud

As there were some concerns of the use cases with cloud-based approaches, e.g. data privacy and usage, the MANTIS reference architecture leaves open, if the platform tier is realized as a private cloud under full control of the OEM or an open cloud.

Actually, a cloud-based architecture is intended to provide a common environment for services and

applications to be provisioned and used with minimal management effort. Cloud environments provide an infrastructure of virtual computing resources that allow users and enterprises to design, implement and use services like data storage, data processing (batch and stream processing, etc.). In the same way as

the edge-based architectural pattern, the cloud itself needs to be designed. In the context of the MANTIS

project a “Broker” and/or “Middleware” pattern is proposed and used. This pattern provides a complete

infrastructure for highly distributed applications and it envisions the presence of an entity – “Broker” –

that is responsible for coordinating all the communications between distributed software components and/or modules (in the context of MANTIS, CPSs) such as forwarding requests, as well as for transmitting

results and exceptions. A broker is used to receive and distribute data from the edges (edge broker). In addition, a service broker can be used to coordinate the data distribution between the different

maintenance services.



Design Decision: Maintenance function deployment

MARMKF_002, MARMKF_003, MARMKF_005, MARMKF_009, MARMKF_010

In an Edge-based architectural pattern, “high distribution” is a key concept. As a matter of fact, an edge-

based architecture is intended to distribute software applications, data and services to the logical extremes

of a network near to the source of the data.

The maintenance functionality of the reference model is distributed across the different tiers. To be more specific, data analysis also happens in the edge and not all acquired data is necessarily shared on the platform. This makes the edge a smart component in the overall architecture and not just a rather simple data provider.

Due to communication bandwidth and data usage issues, a substantial amount of data analysis can and

should happen on the edge gateway. Especially in the off-site and mobile use cases, health assessment or even prognostic assessment can be rendered on the gateway. Local pre-processing can also be helpful in the light of data protection, as only aggregated data for a certain purpose is provided on the platform.

If real time analysis is required and the edge to cloud connector does not provide sufficient quality of

service, a strategy could be to deploy the real time analyses at least partially on the edge.

Design Decision: Common functionality groups

MARMKF_002, MARMKF_003, MARMKF_010, MARMKF_011, MARMQG_002, MARMQG_003,

At the current evolution stage of predictive and proactive maintenance solutions, a reuse of data analysis building blocks across different application settings seams not to be feasible, as the data analysis solutions are tailored to the settings in the SOI. Hence, there is no added-value if the MANTIS reference architecture

would try to identify and specify specific data analysis services and interfaces beyond the already existing ISO 13374 [9], which already proposes a functional decomposition.



5.2 Data Processing and Management

Data processing and Management is a key issue in any distributed system and therefore also in MANTIS. Due to the importance of this it is elaborated in an own deliverable (cf. D2.5). Key aspects are listed in

this chapter for an integrated reading experience.

Design Decision: Data processing patterns

MARMKF_001, MARMKF_002, MARMKF_009, MARMKF_010, MARMKF_011

For processing large amounts of data, different architectural patterns like Kappa and Lambda have been

identified. As the different patterns have their pros and cons, the MANTIS reference architecture does not favour a specific pattern to this end.

Design Decision: Data types and storage

MARMKF_004, MARMKF_005

According the information model, MANTIS solutions can process and store data of different type and representation. Besides discrete sensor values also video streams might need to be processed. Different

storage technologies are available to store large amounts of data. Depending on the application setting,

SQL-like and non-SQL databases can be used and are used in the MANTIS use cases.

Design Decision: Data storage organization

MARMKF_004, MARMKF_005, MARMKF_012, MARMKF_018

The MANTIS reference architecture is open to use various options for data storage, but MIMOSA is a good starting point for the organization of the data storage. The MIMOSA data scheme was developed in the light of central SQL-based data storages, and meanwhile technology has evolved. However, the

MIMOSA data scheme is capable of storing the necessary information (as it is based on the ISO-13374 standard) and can be a good inspiration and guidance for defining own data schemes.

Design Decision: Data access via data management facade

MARMKF_005

As the data storage technology is subject to fast change over the last years and most probably will continue to be so in the near future, the MANTIS reference architecture proposes to encapsulate the

actual storage technology behind a respective data management façade in order to mitigate the impact

of technology changes of the data storage on the data analysis functions.



5.3 Interoperability

Interoperability is a key issue in any distributed system and therefore also in MANTIS. Due to the importance of interoperability, interface, protocol and functional interoperability guidance and

specification are elaborated in an own deliverable (cf. D2.10).

In this section we just document the most influential design decisions regarding to interoperability in order

to offer the integrated reading experience.

Design Decision: Collaborative proactive maintenance

In order to support collaborative proactive maintenance, different ways of doing so have been discussed. Collaboration on a shared data exchange and processing platform, would ease collaboration, however, due

to data protection regulations and the plethora of different data processing platforms, such standardization is not adding value to the current existing settings. Therefore, the MANTIS reference architecture proposes to the use dedicated collaboration spaces with standardized data storages for collaboration.

A viable approach to this end is to use MIMOSA-based data storages (cf. Finnish use case) to provide and exchange the shared data. To ease the adoption and usage of MIMOSA-based solutions, respective REST-APIs have been provided and respective training material has been compiled. Some of the use cases

follow the MIMOSA-based collaboration. They will reveal, if MIMOSA-based solutions will also be capable

of handling non-SQL-data.



5.4 Lifecycle Management

PdM solutions are distributed systems with high evolution rates in all tiers. The identified architecture drivers show clearly the demand for a coherent lifecycle management of the overall solution.

Design Decision: Lifecycle management on gateway

MARMQG_001, MARMQG_002, MARMQG_003

As the data analysis on the gateway needs to be evolved, e.g. to add new sensors or data analysis functions

to the edge, or to adapt it to changing environment, the gateway supports a respective lifecycle management for the PdM system elements.

In order to ease the management of the diverse edges an edge management service is provided in the platform tier.

Design Decision: Lifecycle management on platform

MARMQG_001, MARMQG_002, MARMQG_003

Also sound lifecycle management is needed for the platform tier, e.g. to deploy new data analyses.

5.4.1 Application

Different technologies exist to support application lifecycle management, especially Also on resource constrained devices. Among the most prominent ones is OSGi [15]. It was originally developed to support lifecycle management on open service gateways, as for instance the edge gateways.

The OSGi specification describes a modular system and a service platform for the Java programming

language that implements a complete and dynamic component model, something that does not exist in standalone Java/VM environments. Applications or components, coming in the form of bundles for deployment, can be remotely installed, started, stopped, updated, and uninstalled without requiring a reboot; management of Java packages/classes is specified in great detail. Application life cycle

management is implemented via APIs that allow for remote downloading of management policies. The service registry allows bundles to detect the addition of new services, or the removal of services, and adapt accordingly

Figure 5-2 OSGi Service Gateway Architecture [16]

The OSGi specifications have evolved beyond the original focus of service gateways, and are now used in

applications ranging from mobile phones to the open-source Eclipse IDE. Other application areas include automobiles, industrial automation, building automation, PDAs, grid computing, entertainment, fleet management and application servers.

Although most of OSGi implementations are based on Java, reading the OSGi specification can also help to implement similar concepts based on other technology.

https://en.wikipedia.org/wiki/File:Osgi_framework.svg



5.4.2 Interface

Application programming interfaces (APIs) enable two or more different systems to interact with each

other. API lifecycle management comprises the whole lifecycle from concept to deprecation of the interface. Stages in the API lifecycle are planning and designing, developing, testing, deploying, evolution and retiring of the API.

The OpenAPI specification [17] defines a standard interface description for REST APIs to allow systems to discover and understand the capabilities of a service. With a properly defined API, a consumer can understand and interact with the service without access to its source code or additional documentation.

5.4.3 System

System lifecycle management is a concept on administrative level to handle the complexity of information systems more efficiently over the entire lifecycle. Enabler for system lifecycle management are virtual machine and container technologies like Docker [18].

Virtual machines are system hypervisors allowing multiple guest operating systems to run together on a

single host system by translating guest instructions to host instructions using hardware virtualization or binary translation. Containers are lightweight operating systems running inside a single host system. In contrast to a virtual machine, a container is running instructions native to the core CPU of the host

system. Therefore, containers provide savings in resource consumption.

A further important architectural difference between containers and virtual machines is the way in which storage and security are managed. Security of virtual machines depends on the implementation of the hypervisor. Access controls and policy enforcements provide security for container. Virtual machines take

much more storage than containers, as each instance needs the installation of whole operating system

kernel and its associated programs. Containers share their operating system and therefore take lower amount of storage.

In order to support dynamic environments modern container technologies like Docker provide container orchestration, change and configuration management out of the box.

5.4.4 Continuous Automation

Continuous automation enables moving of software applications from idea to deployment at high velocity. It is the practice of automating every aspect of their lifecycle from designing, developing, building, testing,

integrating, deploying to operating. Continuous automation enables changing software and systems at high velocity, consistently and safely by automating infrastructure, development and deployment workflows, applications and systems operations, and compliance management.

Application, interface and system lifecycle management are necessary to implement an effective continuous

automation, which is able to reduce

time from commit to deployment,

mean time to resolve failures and

risk of deploying low quality.



5.5 Security

5.5.1 Introduction

In modern IT applications and services, a large amount of business-critical and personal data is processed and exchanged continuously. This may even happen unintentionally or unnoticed. It is therefore of crucial

importance for both companies and individual users to control the usage (e.g., dissemination, big data analysis, data exchange for a certain time) of sensitive or confidential data in order to prevent any misuse right from the start. This is especially important, when considering new data driven business models, where data sharing needs also the control of the usage. The lack of suitable security measures in this area

can lead to identity theft, disclosure of secret data or official documents, or loss of reputation due to the

violation of customer privacy. According to external studies1, about one in two companies had some kind of attack within the last year. Although external attacks by hackers are still the most common attack scenario with approximately 40 percent, insider attacks are increasing to about 30 percent in recent years.

Data usage control on one hand offers an effective and flexible approach for this situation, but on the

other hand, it is also an enable for new data driven business models. It is an extension of classical access control and allows specifying and enforcing security policies for regulating the usage of data after initial

access has been granted.

The concept of data usage control enables security policies of any degree of granularity that may also

cover time-related and cardinality-based aspects. Granularity can range from strict separation of domains to protective mechanisms regulating the individual usage of specific data, depending on the respective domain and implementation. This allows guaranteeing comprehensive protection of data. At the same time, more rigorous security measures can be established for selected confidential data.

5.5.2 Supporting the vision of MANTIS

Within the project, a massive amount of data will be collected that characterizes the usage of a system, its operational condition, location, movement and other physical properties. Towards the goal of

maintenance optimization, different systems will process the data. In addition, different stakeholders will share the maintenance relevant data in order to collaborate e.g. in data analysis. Within this collaborative maintenance ecosystem, the (probably sensitive) data will be altered, copied and disseminated arbitrarily,

after initial access has been granted. Classical access control cannot provide the mandatory security

mechanisms for today’s business environments and processes. Distributed data usage control allows

controlling the future usage of data beyond this initial access. Data usage control extends established access control solutions and can offer additional value in the area of data security, thus addressing many of possible requirements regarding data privacy and intellectual property protection. In MANTIS, for

example, data usage control can enforce the usage of the collected sensor data according to fine-grained policies.

Such a usage can be (exemplarily):

• Automatic deletion of data after a certain time

• Remove or anonymize parts of the data

• Aggregate data or modify it before/during/after analysis

Data usage control is an adequate instrument for mastering these and similar challenges.

1 See, for example, https://www.brainloop.com/de/corporate-trust-studie-industriespionage-2014.html

https://www.brainloop.com/de/corporate-trust-studie-industriespionage-2014.html



5.5.3 Security Information Model

Traditionally security has been just a commodity or a necessary evil but along the years this perception has being changing to become an integral and an inseparable part of any system. Indeed, nowadays,

security is in fact a functional requirement for interoperability with many existing systems.

MANTIS represents itself a challenge and an opportunity to apply the most modern approaches in the field of security. In this sense, we will focus on the security information model, which is based on the

needs of the project and supported by a physical secure architecture of the system.

In order to facilitate the understanding of the final model is necessary to define previously standard security information models, and then identify the challenges and the threats.

In any system, the security information model is based on three principles:

Integrity. To maintain the completeness and accuracy of data over its entire life-cycle.

Confidentiality. To guarantee the privacy of data over its entire life-cycle to unauthorized individuals, entities or processes.

Availability. To guarantee that the information is available when is needed.

These principles impose some requirements:

- The system must be defined in digital terms, that is, every relevant element (subject, action and

object) must have a digital identity with independence of the nature of the element. This is a very important requirement in order to successful apply effective security policies and prior to use a specific information model.

- A specific information model for managing different levels of confidentiality.

- A specific security policy model.

- Additional requirements associated with industrial environments.

Additionally, it will be necessary to specify the requirements of the Public Key Infrastructure to support elements such as integrity.

Finally, practical considerations will be detailed in order to fulfil with existing security technologies such

advanced threat detection techniques.

5.5.4 MANTIS Security Implementation

Previous concepts are the basis of any system that needs to be secured, although traditionally used in

non-hybrid systems, that is, in classical IT environments but in Industry 4.0, the implementation of these concepts in an effective and useful way requires a deeper analysis of the best theoretical models and especially the technology used to implement them.

In MANTIS a realistic and effective information security management system has been carried out by

involving the following processes:

- Process 1. Digitalization of the system. - Process 2. Information security model. - Process 3. Implementation of the security model.

5.5.4.1 Process 1: Digitalization of the system

Within company environments, there exist end user clients, mobile devices such as smartphones and tablets as well as a dedicated server and network infrastructure. For providing a comprehensive enforcement of security directives throughout the entire company environment, each of these components



needs protection according its potentials for data usage. This includes, for instance, data leakage prevention for end user devices to prevent unintentional copying or mailing of sensitive data. Additionally, mobile devices have to be protected according to their current context, such as usage outside company premises, with more restricting policies compared to usages in the company environment. These security

policies have to be automatically adjusted with respect to the current context of the user and the mobile device. Finally, the server and network infrastructure has to be protected against both insider attacks, such as unauthorized mass data retrieval by an employee, and attacks from the outside.

In order to accomplish this control, a previous and fundamental step is necessary: the digitalization. In essence, the idea is that just every object has a digital reflect, that is, a unique identification of every

element that allows to control every action and every subject involved in the information management.

This process of giving an identity can be performed in many ways, in MANTIS the best approach has been followed, that is, by using Public Key Infrastructure generating digital certificates not only for every object providing as an unique identifier but for every significant operation. In this sense, in order to

consume services based on REST APIs require a digital certificate for the server but also for the client.

#923045#

Digital Certificate

Unique Identifier

Subject

Figure 5-3 Digital Identity

In MANTIS the digital identity will be the key to be able to establish security policies. Without a digital identity it is not possible to control the behaviour of the system. With this aim in mind, in MANTIS we will associate every element of the platform with one of the following categories:

- Subject. It is the element in charge of requesting operations (actions) with objects. These are the

actors of the system. In many situations the subjects are processes intermediated by users but

there are other situations where the processes are not associated with users. - Action. It is the definition of an operation; every operation must be defined in order to control the

behaviour of the system. - Object. These are the elements which receive the actions. In this category, certain elements can

be subjects and/or objects such as processes.

Based on this definition the digitalization of the system requires a specification and classification of the

elements of the system, which can be involved in the processing of the information.

5.5.4.2 Process 2: Information security model

Information usage control is a generalization of access control and is concerned with requirements that pertain to handle data after access to it has been granted. It is about the definition and enforcement of policies regulating what must (not) happen to data after its distribution. Hence, usage control extends

access control by the consideration of future use and specifies obligations for data users. Among other

domains, usage control is relevant in the context of intellectual property protection, compliance with regulations, and digital rights management.

In general, data is processed on several abstraction layers of a system in parallel. For example, when receiving content over the network, a temporary file is written, the content is rendered in some application window and is stored in some application-specific data structure. In order to protect sensitive data from



undesired usage throughout an entire system or infrastructure, data usage control must therefore be enforced on several layers of abstraction.

Besides this, it is necessary to define the security classification of the information but taking into account

the potential of the platform, under two challenging situations:

- Industrial environments. The companies try to maintain as confidential the information of the processes (KPI), the model of machines, the technology used, etc. Additionally, two factors are very important:

o The availability and the integrity of the industrial processes are very important for taking

right decisions on processes.

o The system must respect operational safety systems. MANTIS cannot interfere on real-

time systems which may produce personal injuries or catastrophic losses. - Medical environments. The personal health information is one of the most critical assets; indeed,

there exists a very restrictive legislation in many countries. MANTIS should implement a model which guarantees this confidentiality of the information.

The assessment of these two challenging environment leads the use of restrictive security information models. A priori, the first candidate is the most restrictive information model known as Bell-Lapadula Model (BLP). This model is used in government and military applications and it is focused in enforcing

the access control to confidential data. This model has the following properties:

- The simple security property. This establishes that a subject of a specific level cannot read information at a higher security level.

- The *(star) property. This establishes that a subject of a specific level cannot write to any object at a lower security level.

- The discretionary security property. In this the specification of the discretionary access control is

made by means an access matrix.

Figure 5-4 Bel Lapadullas

This model is accompanied by the following definition of security levels: Top Secret, Secret, Confidential and Unclassified.

On the other hand, there is no perfect model and this is not an exception, BLP has a set of theoretical limitations, but the most relevant in MANTIS platform are the following:

This model was designed for system where the security levels do not change dynamically.

The state-transition model does not contain any state invariants.

The information model requires an access control policy to rule the security of the system. With this aim in mind, the original model specified (BLP) establishes the policies as a matrix where the access to every

element is specified.



The main drawback of the original specification is the inherent complexity of MANTIS, which requires a more complex specification of the security and at the same time more a way to facilitate the management of the security. The direct application of this matrix will conclude with a huge matrix and some problems to manage the policy.

With the aim of overcome this drawback, an initial approach to facilitate this is the use of the concept of “ Role” , that is, a mechanism for grouping sets of subjects with the same level of security but with some interesting properties in terms of manageability:

These are suitable means for encapsulating the organizational functions/duties of a user.

Different roles can be defined, each for different types of competences, which are assigned to

users.

RBAC realizes the security principle of “ least privilege” .

It is consistent with BLP model.

Therefore, in MANTIS the first approach to manage the security will be the use of roles and the BLP for establishing the security policy.

An RBAC model can formally be described by the tuple RBAC=<U,R,P,O,…>, the most important

elements of this tuple are:

U: User.

R: Role.

P: Permission.

O: Object.

The National Institute of Standards (NIST) establishes some subdivisions of the original model such as:

Core RBAC, Hierarchical RBAC, Constraint RBAC and Consolidated Model.

The model that best suits to MANTIS is the Hierarchical and Constrained RBAC model, which supports the majority of the challenging environments and situations. The joint use with the BLP model facilitates the management of the security of the system.

This model has been used and it is being used in common or standard environments, that is, in non-hybrid environments. In this sense, MANTIS represents a challenge because a HC-RBAC system could fulfil all

the needs in normal situations, however, in industrial environments there are some events and situations such as maintenance periods or failures of components that are not correctly controlled by the security system.

For example, a policy can specify that a system engineer has access to command an actuator because is part of the group of authorized personnel but it the system has failed this operation can become in a high-risk operation if someone is trying to repair the machine. The main drawback of an HC-RBAC is that it

does not consider context information.

In MANTIS, the most effective and recommended approach has been adopted, that is, to adopt a HC-RBAC as the core policy but including context information by using attributes, concluding with an hybrid model known as RBAC-ABAC model. With this model previous situations are covered while maintaining intact the general security policy, or in general terms, it is possible to consider dynamic environments such in industrial environments and at the same time static policies for classical environments where the context

makes no sense.

This hybrid model has concluded in a set of standards and security specifications known as “ eXtensible Access Control Markup Language” (XACML). A precise definition from Wikipedia is:



The standard defines a declarative fine-grained, attribute-based access

control policy language,[2] an architecture, and a processing model describing how to

evaluate access requests according to the rules defined in policies.

As a published standard specification, one of the goals of XACML is to promote common

terminology and interoperability between access control implementations by multiple

vendors. XACML is primarily an Attribute-Based Access Control system (ABAC), where

attributes (bits of data) associated with a user or action or resource are inputs

into the decision of whether a given user may access a given resource in a particular

way. Role-based access control (RBAC) can also be implemented in XACML as a

specialization of ABAC.

The XACML model supports and encourages the separation of the access decision from

the point of use. When access decisions are baked into client applications (or based

on local machine userids and Access Control Lists (ACLs)), it is very difficult to

update the decision criteria when the governing policy changes. When the client is

decoupled from the access decision, authorization policies can be updated on the fly

and affect all clients immediately.

This standard provides the following:

Policy Language.

Request and Response Language.

Standard data-types, functions, combining algorithms.

Extensibility.

Privacy profile, RBAC profile.

An architecture defining the major components in an implementation.

The general terms for XACML are the following:

Resource: Data, system component or service.

Subject: An actor who requests to access certain Resources.

Action: An action on resource

Environment: The set of attributes that are relevant to an authorization decision and are independent of a particular subject, resource or action.

Attributes: Characteristics of a subject, resource, action or environment.

Target: Defines conditions that determine whether policy applies to request

The structure of a security policy is specified as follows:

https://en.wikipedia.org/wiki/Access_control

https://en.wikipedia.org/wiki/Access_control

https://en.wikipedia.org/wiki/XACML#cite_note-2

https://en.wikipedia.org/wiki/Attribute-Based_Access_Control

https://en.wikipedia.org/wiki/Role-based_access_control

https://en.wikipedia.org/wiki/Access_Control_Lists

https://en.wikipedia.org/wiki/Authorization



Policy Set

PolicyPolicy

TARGET

Policy

Policy Set

Policy

RuleRule

TARGET

Rule

Rule

Effect

TARGET

CONDITIONS

Figure 5-5 Security Policy Structure

On the other hand, it is necessary to include a suitable and right government of the security policy by separating functional tasks into different components. That is, it is necessary to separate the function of

intercepting any request of operation with data from the decision point or from the place where the policy is being stored, all these issues needs to be considered in a realistic way.

5.5.4.3 Implementation of the security model: Integrated Distributed Data Usage Control Enforcement (IND2UCE)

Policy Specification

Security policies are an established instrument for expressing security demands. Such security policies and the corresponding security mechanisms for enforcing them can become very complex. In general, different stakeholders have manifold security demands that they need to express and specify properly. These

stakeholders have different skill levels, areas of expertise and background knowledge with respect to security and the application domain. Thus, it is obvious that less experienced users can lose track of the effects and consequences that the enforced security policies entail. To prevent misconceptions, interfaces for specifying security policies should be informative, easy to understand and intuitive to use.

Misspecification can lead to massive damage due to unintended data leakage, which can even be

aggravated by careless behaviour caused by a wrongly perceived security due to wrongly specified security policies.

IND2UCE’s methodology for eliciting the security policies supports the derivation of a complete and correct

catalogue of security policies from an application domain. This includes the identification of assets, threats

and appropriate countermeasures in the application domain, which are elicited from various stakeholders. Additionally, domain and legislative regulations and company-specific directives might have to be

incorporated into the catalogue. Usually, these policies are dynamically adjusted over time according to

changing requirements and adapted to a specific user’s demands. Due to the different skill levels and areas

of expertise, policies have to be specifiable by various user groups without compromising the overall security catalogue.

Policy Decision

Based on the specified and installed policies, decisions are drawn upon incoming requests, that is, policy-

related events. IND2UCE’s evaluation engine supports several temporal and cardinal operators. This allows

the specification of policies incorporating, for instance, the frequency of a certain event within a given time interval or interdependencies between several, potentially unrelated events.

IND2UCE security policies are based on the Event-Condition-Action paradigm (ECA). Basically they consist of the following parts:



Event

Condition

Authorization decision and execute actions

Incoming events are matched against the specified event. There are several measures in place to limit the performance overhead in the monitored system. This includes, for instance, that only those events are

intercepted and evaluated that are necessary for the decision making. In case of a successful event matching, the condition is evaluated.

This condition is expressed in the Obligation Specification Language OSL, a first-order temporal logic language. It supports common propositional logic operators such as true, false, not, and, or and implies.

Furthermore, it provides temporal operators for specifying not only common access control directives, but

additionally for restricting the usage of data in terms of temporal intervals or depending on any previously occurred events. These operators include always, during, within and since. Finally, cardinality-based

operators provide support to specify restrictions in terms of general frequencies and —in combination with

temporal operators—also frequencies within given time intervals.

If the specified condition is satisfied, the appropriate authorization decision (permit or deny) is enforced. Furthermore, additional actions, so-called execute actions, are invoked. This includes both mandatory

actions (e.g., allow the event only if a persistent log entry was written successfully), as well as optional

actions, independent of the final decision. If the intercepted event does not match the specified events of any active policy or the appropriate condition is not satisfied, a default answer will be returned. Depending on the situation, this can be a default inhibit in case of a white-listing approach or a default allow for a black-listing approach.

The policy decision may also depend on additional information that is not present in the intercepted event

itself or the current policy state. This includes information about contextual information such as the geographical location of a mobile device or an abstract context of an entity.

Policy Enforcement

For enforcing usage control policies, system events need to be detected and potentially intercepted. These events depend on the system domain for which the policy is specified, for example, virtual machine

migration events for cloud environments or application communication of an app on a mobile device.

Therefore, policy enforcement must be tailored to the specific technology for understanding the semantics of the events. The monitored and intercepted events are given to the decision engine for requesting

permission or denial of the event. In addition to just allowing or inhibiting the event’s execution, the

decision can also require a modification of the event.

In addition to the enforcement decision on the intercepted event—permission or denial—a decision can

be bound to a successful execution of some specified action, such as the writing of a permanent log entry.

Only after a successful execution response, the final decision will be enforced. Depending on the installed scenario, a security policy can incorporate an entire chain of potential decisions. In this case, the execution status of additional actions, whether they failed or succeeded, determines the final decision that will be enforced.

Policy Management

In general, policy management is concerned with the policy life cycle. This includes the instantiation,

negotiation, deployment, and revocation of security policies, as well as conflict detection and resolution. As these tasks involve several different components (potentially on different systems), policy management also includes the management of the IND2UCE components required for policy enforcement.

Policy Management becomes especially important when exchanging data across system boundaries. Every

time data crosses system boundaries, the target system must be prepared for the protection of incoming data, that is, the corresponding policies need to be deployed. These policies need to be instantiated for a



specific target system, because enforcement mechanisms can work differently (e.g., work on different events) on different systems or technologies. Policies can have a major impact on the system they are deployed on. Moreover, policies may also affect other systems. For example, a client policy might trigger a server action in case something unwanted happens. If the policy scope is not limited to one system, all

involved parties need to be identified and asked for permission. If all affected parties agreed on the policy, it can be deployed on the target system.

Technical Framework

The policy enforcement framework IND2UCE is a technical solution for data usage control. It provides

mandatory components (see Figure 5-6) for providing a comprehensive enforcement of data usage policies

in different technical infrastructures and application domains. The framework is divided in three layers:

„Manage“, „Decide“, and „Enforce“.

Figure 5-6: Policy enforcement framework IND2UCE

In the following, the components of the IND2UCE framework will be briefly explained in an exemplary sequence.

First, the Policy Administration Point PAP provides a user interface for security administrators to specify new policies or to alter and manage existing ones. These policies are sent to the Policy Management Point

PMP for storing them in a Policy Retrieval Point PRP or for deploying at a Policy Decision Point PDP.

The Policy Retrieval Point PRP stores policies securely so that malicious entities cannot modify existing policies or upload unauthorized policies.

provideinformation

instantiateprovidepolicies

deploy/revoke

inquire decide

store

execute

Decide

Man

age

Enfo

rce



The Policy Management Point PMP initiates policy deployment or revocation requests. Additionally, all components of the IND2UCE framework are required to register with their appropriate communication interfaces and capabilities at an appropriate PMP. Using this information, other components can query

the PMP for mandatory interaction partners. For instance, a Policy Decision Point PDP can dynamically

query the PMP for an appropriate Policy Information Point PIP providing demanded additional

information. The PMP’s component registry provides a dynamic enforcement infrastructure with

potentially changing components during runtime.

The Policy Enforcement Point PEP is responsible for intercepting relevant events and requesting a decision how to proceed with the event. Depending on this decision, the event may have to be inhibited, or specific event parameters may have to be modified by the PEP.

The decision for intercepted events is determined by the Policy Decision Point PDP based on installed policies. Policy evaluation may require additional information provided by a Policy Information Point PIP.

This information, not present in the intercepted event itself, ranges from supplements from a directory services to data flow information of the concerned data item. Furthermore, this includes contextual information such as geographical location of a mobile device or an abstract context of an entity (e.g. an employee is at the working premises in a meeting without wireless network available) or external service

information. A decision can be bound to a successful execution of a specified action, such as the writing

of a permanent log entry. In this case, the PDP triggers the execution of specified actions at a Policy Execution Point PXP. Only after a successful execution response, the PDP will return the appropriate decision to the PEP; otherwise, a fall-back action, if specified, will be processed.

In the following, each component will be described in more detail with existing exemplary instantiations.

Management-Layer

The management layer contains the Policy Administration Point, the Policy Management Point and the Policy Retrieval Point.

Policy Administration Point

The Policy Administration Point (PAP) represents the graphical interface between users, for example, the responsible security administrators, and the IND2UCE framework. The security policy specification process

needs to be usable for the intended user group. As users differ in their usability requirements (e.g., efficient usage versus detailed guidance), different specification paradigms must be supported. In addition, a tailoring of the PAP to the application domain by, for example, providing domain-specific security policy

templates and terminology, can increase user acceptance of PAPs as well as correctness and completeness of the specified policies. A PAP framework allows the tailoring of PAPs to the application domain, to the policy specification paradigm, and to the target platform on which the PAP runs. Figure 5-7 shows an exemplary PAP instantiation on an Android platform for the cloud domain and for non-policy-experts, but

for users with strong domain knowledge. This PAP instantiation uses security policy templates as an adequate way to adapt to individual aspects of the policy, without considering the final low-level and machine-readable policy.

Figure 5-7: Policy Administration Point (PAP)



Policy Management Point

The Policy Management Point (PMP) is a technology-independent component and can consist of multiple

PMP clients and a PMP server. To enable cross-system enforcement, each PMP client registers at a

central PMP server. The PMP server offers central services and a global registry for other enforcement infrastructures. The PMP client provides a central registry for every IND2UCE component running within its local system. Each component registers at the PMP, thus providing a dynamic enforcement infrastructure. Besides that, a PMP offers services like policy deployment and revocation. Depending on the types of components, additional information about their interfaces and capabilities are registered. For

example, the PDP can request a certain information source (PIP) or enforcement component (PEP) via

a PMP that is needed for the decision making process. Finally, in case of a cross-system enforcement, policy templates are instantiated for a potentially remote target system prior to the final negotiation and deployment. Policy templates are used to cover enforcement mechanisms, which contain variables that

are instantiated depending on platform information.

Figure 5-8: Policy Management Point PMP

Policy Retrieval Point

The Policy Retrieval Point (PRP) provides a secure and tamperproof storage for security policies and policy templates. In a policy workflow, a user specifies a security policy using the Policy Administration Point and passes it to the PMP for further processing. This can involve a direct deployment at the

corresponding PDP or a storing in the PRP for later use. Moreover, the policy can contain variables and placeholders, which are instantiated later at runtime. The corresponding policy templates are stored in the PRP and instantiated and deployed automatically depending on the concrete scenario.

Figure 5-9: Policy Retrieval Point PRP

Decision-Layer

Policy Decision Point

The Policy Decision Point (PDP) is a technology-independent reasoning component in the IND2UCE framework. Based on the IND2UCE policy language using the Event-Condition-Action paradigm, the PDP

supports several logical, temporal and cardinal operators. Due to its technology independence, the PDP can be easily instantiated on most common platforms and integrated into most systems and



infrastructures, only depending on available communication interfaces. Currently, PDP instantiations for Linux, Android, and Windows operating systems are available.

Figure 5-10: Policy Decision Point (PDP)

Policy Information Point

The Policy Information Point (PIP) allows the retrieval of additional information such as contextual

information during the decision-making. For instance, our Context-PIP for Android provides an easy access

to aggregated sensor information from the Android operating system to detect current situations and user activities. Such contextual information include the current location of the mobile device, connectivity information, etc.

Figure 5-11: Policy Information Point (PIP)

Enforcement-Layer

Policy Enforcement Point

Conceptually, the Policy Enforcement Point (PEP) is the only technology-dependent component in the IND2UCE enforcement framework. Its task is to intercept relevant events occurring in its observation domain, that is, in its layer of abstraction. For instance, the PEP can control the system layer, in which file access events can be intercepted, or the graphical infrastructure layer, where creation of windows or drag and drop actions can be observed. Therefore, PEPs need to be tailored to their specific runtime

technology. After retrieving a response from the Policy Decision Point, this decision needs to be enforced by the PEP.

Figure 5-12: Policy Enforcement Point (PEP)



Various PEP instantiations and prototypes exist from several projects. These include, but are not limited to system-wide enforcement on Android mobile devices, controlling of cloud environments based on VMware, and cross-company enforcement on the Enterprise Service Bus.

Policy Execution Point

The Policy Execution Point (PXP) is a supplementary component in the IND2UCE framework for execution of additional actions. Exemplary instantiations include, but are not limited to writing permanent log entries, sending notification emails, and wiping of an Android mobile device. Depending on the application scenario basically any common user and system action can be executed by a PXP. These actions do not need to refer to any specific intercepted event, but they can comprise those if necessary.

Figure 5-13: Policy Execution Point (PXP)

5.5.4.4 Practical considerations

5.5.4.4.1 Policy compliance

One of the most difficult parts in any system while maintaining the security posture of every element of the platform. In this sense, a significant effort has been made, especially from EEUU. One of the most important agencies of this country created a specification for security compliance known as Security Content Automation Protocol (SCAP).

SCAP gives us a transparent, interoperable, repeatable, and ultimately automated way to assess security software flaws and misconfiguration in the enterprise. This standard has the following features:

It is based on open standards

It uses configuration and asset management standards

It adapts to a wide array of use cases.

It enables the repeatability across products and services of various manufacturers.

…

Based on this protocol we can specify the security posture of any system, of any nature.

5.5.4.4.2 Threat detection

Traditionally the threat detection was an easy task and it was primarily based on signatures or changes in the configuration of the systems. Although the emergence of the new threats known as Advanced

Persistent Threats has made this task extremely difficult. Modern malware tries to be hidden as much as possible making virtually impossible the detection of this kind of threats.

In order to overcome this situation a new concept was created named “ Indicators of compromise” (IOC). The aim of this concept is to identify and share threat information. MANTIS will take into account this concept and it will use in those cases where the exposition is very high or in those situations where the threat level is considerable.



5.6 Reference Applications

This chapter outlines the various deployment scenarios which can be achieved by using the MANTIS Reference Architecture.

In this chapter we present a means to define and visualize reference applications (RA) for MANTIS. The reference applications are intended to

ease the classification of MANTIS use cases

raise the understanding for crucial architectural issues

guide the design of the MANTIS ARM

The reference applications are denoted in the context of actors (from stakeholders list), their concerns,

as well as the main maintenance functions (cf. 4.3). The here proposed approach is to define reference applications along key features influencing the system structure.

As a starting point we use the following feature X reference application matrix with the 4 features ( which

are identified as most differentiating) “Number of ecosystem players”, “Distributed Data Processing”,

“Mobile Machine”, and “Number of Machines in Monitored System”. The 3 reference applications are

characterised by differing feature configurations. The matrix and its key features are matter of evolution and therefore are planned to change in future versions of this document.

Table 5-1 ‘Feature’ X ‘Reference Application’ matrix

Feature RA1 RA2 RA3

# of Ecosystem Players 1 2 2

Distributed Data Processing No Yes Yes

Mobile Machine No Yes Yes

# of Machines in Monitored System 1 1 1

In the following 3 sections we denote the interplay between involved actors, ecosystem players, concerns addressed, and the main MANTIS functions for the listed reference applications.



Figure 5-14 Reference Application 1

Reference Application 1 describes the MANTIS entry-level scenario where there is only 1 player which controls the entire ecosystem (ranging from machine ownership all the way up to providing maintenance services). This scenario is for example applicable when a machine owner cannot or does not want to outsource the service functions.


Reference Application 2 outlines a slightly expanded scenario where there is a machine owner and a separate service provider. In scenarios like this, additional concerns, which need to be addressed, are for



example on data usage controls. (It is possible that the machine owner does not want to share all the machine data with the service provider)


Reference Application 3 depicts an even more complex scenario where there are 3 ecosystem players in the mix with many machines in a machine line. Here can be seen that different functions of the MANTIS ecosystem can reside on multiple places and each with their own scope. In such scenarios it is evident

that interoperability, data ownership and -usage are of utmost importance.



6 Reference Technologies and Solutions

6.1 Reference Technologies

Based on the reference solutions that have been developed by the various partners (and that are listed in section 6.2) we can suggest some technologies of interest which can be used to implement the architecture.

Table 6-1 Reference Technologies

Area Technology

Messaging RabbitMQ (AMQP, MQTT) [19]

Data Streaming Apache Kafka [20]

Data Processing Apache Storm [21]

Data Storage Apache Hadoop HDFS [22], MIMOSA [23]

Application Lifecycle Management OSGi [15]

Interface Lifecycle Management OpenAPI [17]

System Lifecycle Management Docker [18], Kubernetes [24]

Continuous Automation Chef [25]



6.2 MANTIS Reference Solutions

The reference solutions presented in this chapter detail out how various partners have implemented the MANTIS reference architecture using different technologies.

6.2.1 Use Case: Conventional Power Production

Characteristics: Stationary equipment, collaboration between multiple partners

The partners working with the use case 3.3 are Fortum, Nome, Wapice, VTT and LUAS, and the use

case handles the topic of operation and maintenance of machines (O&M) in conventional power

production. The pilot target is a flue gas recirculation blower which is equipped with different sensors of multiple partners in the phase 1 (see Figure 6-1). The gathered data from the sensors is transferred further to a MIMOSA database provided by LUAS. The MIMOSA database is a free open source database solution dedicated to O&M needs in manufacturing, fleet and facility environments [23]. LUAS provides a REST

interface for the MIMOSA and thus data gathered and stored to the MIMOSA are available to all partners. All partners can store and get data from the MIMOSA all around the world as long as there is an internet connection available and thus the phase 2 “ Data access and display” is quite successfully covered.

Figure 6-1. Phases in the conventional power production use case 3.3



The used information transfer protocol is the mentioned http Representational State Transfer (REST) protocol. The RESTful interface is implemented in JAVA, operates on a JAVA server and it offers CRUD (Create, Read, Update, Delete) functionalities. The data posted and retrieved via REST is formatted as JavaScript Object Notation (JSON). Recently, the first test version of REST, that supports all the tables

found in the MIMOSA database, has been presented by LUAS. The REST-MIMOSA combination has been used for a while already but at first stage the REST support was made only for the main tables used in the use case 3.3. The REST application programming interface (API) is not only used for storing new data to MIMOSA, but it is used also for integration of different existing data storage systems with MIMOSA, such as systems of Nome, Wapice and Fortum.

The MIMOSA accepts all kind of data to be stored there, such as time waveform data, spectrum data,

single measurements (e.g. temperature measurements), processed data (e.g. calculated statistical factors), amplitude peaks, test information (e.g. ferrography info), images, character data (e.g. notes), binary data and much more. Measurement location related data, such as where the measurement was done (in which

enterprise, to which component/asset, from which direction, the GPS Location etc.) and what kind of

sensor was used (e.g. accelerometer, output in gs or m/𝑠2) can be stored to MIMOSA as well. MIMOSA

supports also storage for alarms related data, engineering study data (e.g. RCM, FMECA), prognostic and diagnostic related data, health state detection data, remaining useful life data, maintenance

recommendations data, maintenance work management data and data related to logistic resources. More

accurate description about storable data can be found from the documentation of the MIMOSA.

MIMOSA is really versatile and supports all kind of data and in that way makes the collaboration between different sections of a company/organisation or between different companies/organisations easier. MIMOSA is standardized and unifies the ways how data is stored and recovered from the database and thus all the data is available for everyone through the same kind of procedures. This wide variety of

available data makes it easier to test, verify and develop analysis tools for the condition monitoring needs

(phase 3). The available data makes it easier also to develop different kind of user interfaces for specific needs because there are lot of different data presented in the same source from which you can pick up the needed data and info.

In this use case Nome and Wapice provide data from the flue gas recirculation blower to the MIMOSA database from which the data can be gathered to various user interfaces and for example plotted by other partners. Used interfaces include for instance web pages from which the data can be easily examined with

any web browser and with a mobile phone too. Data can also be further manipulated and stored back to MIMOSA by other partners. Used post-manipulation techniques involve for example statistical parameter

calculation among other techniques.

In the Figure 6-2 can be seen an illustration of a path from stored data to a web user interface by using the architecture provided by the MIMOSA. Firstly data is stored to the MIMOSA database that follows the Open System Architecture for Enterprise Application Integration (OSA-EAI) MIMOSA standard. The

complete database structure can be downloaded from the MIMOSA webpage for free and it is ready to

be used as such when accompanied with a database engine (VTT has shared an instruction video of how to do it). Then another free MIMOSA architecture called Open System Architecture for Condition-Based Maintenance (OSA-CBM) is used to take care about how the information is handled in condition based maintenance processes. OSA-CBM divides the condition based maintenance into six steps: data

acquisition, data manipulation, state detection, health assessment, prognostics assessment and advisory generation. The needed intelligence is programmed to these layers. To all these layers are connected through an OSA-CBM web service which is the same for all the layers. This common web service makes

it easier to interact between different layers. In the end the user interface connects to these layers through

the web services and then the OSA-CBM blocks uses the data from the MIMOSA database to present/calculate the needed outcome to the user interface. The presented user interface can do calculation to raw data (statistical features, FFT etc.), present triggered alarms etc. In this presented user interface the data can be gathered from two different MIMOSA databases located in separated places (other provided by LUAS and other is VTT’ s local database).



Prognostics

Assessment

OSA-CBM

Web Service

(SOAP)

Health

Assessment

OSA-CBM

Web Service

(SOAP)

State

Detection

OSA-CBM

Web Service

(SOAP)

Data

Manipulation

OSA-CBM

Web Service

(SOAP)

Data

Acquisition

OSA-CBM

Web Service

(SOAP)

Other

Consumers

Web Service

(REST)

Advisory

Generation

OSA-CBM

Web Service

(SOAP)

MIMOSA OSA-EAI

User Interface

MIMOSA OSA-EAI

SQL

Server

LUAS

Web Service

(REST)

Python

algorithms

Figure 6-2. Data from MIMOSA database (based on OSA-EAI) and through OSA-CBM to the web user interface.

Common architecture has led also to a positive interconnection of multiple systems. Nome, Wapice, LUAS

and VTT all have their own kind of sensor systems that feed data to the MIMOSA database from various

locations. The cheapest connected system in the use case is based on a Raspberry Pi 3 model B. A low-cost ADXL001 accelerometer is connected to the Raspberry Pi through a signal chain that involves an amplifier, various filters to reduce the noise, and an analog-to-digital converter. The Raspberry Pi has an integrated Wi-Fi antenna which enables the connection to the Internet and, consequently, to the common MIMOSA database. The Raspberry has also its own local MIMOSA database that can be used to store

data gathered from the sensor and also different parameters that could be helpful when setting up the

system such as, bearing model, rolling element number, outside diameter, and others. It is possible to connect numerous different devices/sensors to the Raspberry like temperature sensors, humidity sensors, moisture sensors, air pressure sensors, gas sensors, motion sensors (gyroscopes, accelerometers, PIR etc.),

GPS receivers, analogue sensors (via ADC), cameras and so on [26]. Another characteristic of the Raspberry is that it can do various calculations to data before uploading it to the database. Some of these calculations are statistical functions like kurtosis, skewness, root mean square, standard deviation, etc. and also an analysis in the frequency domain such as FFT (Fast Fourier Transform) and envelope analysis

among others. This way, the raw data can be saved in the local database, and only the data after the

analysis uploaded. This process helps reduce the necessary processing power and storage in the external database. It is also worth mentioning that as well as the Raspberry sends data to the external database, its local database can be accessed externally if it is necessary to get the raw data.

As a conclusion, it can be said that the platform used in the use case 3.3 supports wide range of

collaboration between partners through MIMOSA and REST and that various devices/sensors can be



connected to the platform for example by using the low-cost Cyber Physical System (CPS) solution that is built upon Raspberry.

6.2.2 Use Case: STILL & Ansaldo

Characteristics: mobile equipment

Our edge device is an STM32F4 series microcontroller and a GSM modem. Another edge device will be the onboard communication unit from the STILL truck itself.

Our RCA engine will be based on fault trees and Petri-net based execution. Our stream processor will

implement log pattern recognition of STILL’ s vehicular error codes.

Our RUL engine is based on proportional hazard method, to be implemented as a stream processing unit (the run time estimation of the current RUL), while the parameter estimation is supposed to be a batch

processing entity running periodically to re-calculate the parameters (if necessary).

The HMI should interact via REST with the stream processor and batch processors. It should also access the data distributor (MQTT broker) to fetch the online measurement data coming from our edge devices.

Figure 6-3 STILL & Ansaldo Architecture

Legacy Physical Entity (static use

cases)

"Dumb" nodes

Sensor Read-out Server

MIMOSA

Apache Spark cluster

RUL

RCA

Vehicles and mobile use cases

IoT-A CEP events

IoT-A CEP events

IoT-A CEP events tailored for comm constrained case

Create and use: MIMOSA contexts

publish

PUSH or request further data or RCA meas

Authorization

System

Service

Registry Orchestration

System

Register its Services or Request Orchestration

Request Orchestration

MANTIS Edge gateways

Trigger

External stakeholders

ERP dummy

BME

AITIA

Register

Register

Stormstore

Trigger

measure

ADIRA, ISEP, INESC

ADIRA, ISEP, INESC

Fetch

Fetch

Java Jersey Grizzly server Hibernate ORM Jersey SSE

STM32 microcontroller,GSM modem, persistent TCP

VTT,runs in MS SQL in Azure,

has UI



6.2.3 Use Case: Adira

Characteristics: stationary equipment

Figure 6-4 Adira reference architecture

6.2.4 Use Case: Goizper

Characteristics: stationary equipment

This section presents the ICT technological architecture used by the “ Early detection of internal wear of

a GOIZPER clutch-brake” use case (Use-Case 1.3, Scenario 1.3.3) in MANTIS. Partners involved in this use case are Goizper, Tekniker and MGEP. Many of the components for the architecture presented in this

section have been agreed upon with other uses cases. The aim of this decision is to provide a similar/common backbone of technologies that will support a third party into building a similar solution

or even provide a solution (MANTIS architecture) to companies without cloud infrastructure. That is, to build a use-case independent implementation of the MANTIS platform.

The overall objective seek by GOIZPER is to early detect internal wear of a GOIZPER clutch-brake. To do that, the moving parts of the clutch-brake are sensorized. By continuously monitoring the system conditions proper operation of the clutch-brake can be ensured. Moreover, the most critical operating

variables can be registered in the platform in order to analyze the working process and prevent misuses. Those data are uploaded enabling the holistic analysis of the clutch-brake system. The idea is to determine/detect the main causes of failure and the components’ remaining useful life.

GOIZPER, as user of the platform, wants to investigate different Cloud platform alternatives. The main

objective is to find the most robust and reliable architecture that fits with GOIZPER’ s business. For this reason, GOIZPER has developed two different data fluxes in order to send the Clutch Brakes information

to the cloud. One of the architectures has been coordinated and developed by MGEP and the second architecture has been coordinated by TEKNIKER. These two different approaches (at edge/sensor and platform level) are shown in detail during the next sections.

6.2.4.1 System Architecture for open source platform (GOIZPER-MGEP)

The architecture implementation [27]agrees with the three-tier architecture presented in [28][2]. There are three levels; Edge tier, Platform tier and Enterprise tier. The Edge Tier is concerned with the



technological solution deployed on the sites where the Press Machines (including the Clutch Brake component) are located. At this level, data acquisition systems to extract the data from the different sensors and SCADA systems connected to the machines are deployed. Figure 6-5 depicts the elements of this tier. There is an Industrial PLC based on the B&R X20CP1382 module. This module is connected to

the sensors attached to the Clutch-Brake (including the intelligent soft-sensors). It collects all the measurements from the sensors and runs local code to pre-process the signals and produce a set of parameters that are able to characterize the overall status of the Clutch Brake. The module stores these parameters in a local file. This module acts as a Datalogger for the cyber physical system. A second embedded computer (Edge Gateway) is attached to the Datalogger. The Edge Gateway retrieves the

parameter files and creates IoT-A CEP Events [29]that are sent to the cloud platform as messages.

Edge gateway

IoT-A CEP Events

Local storage

FTP(poll + pull)

Very lowfreq. updates

Goizper Clutch Brake

Physical Entity

Cyber entity

ºB&R X20CP1382

Figure 6-5: Edge Tier

At platform level, data coming from the different sources is persisted and different applications that allow

analyzing this data are available. The specific modules for the Platform Tier are presented in Figure 6-6:

Edge Brokers: The Edge Broker maintains the connection with edge devices (edge tier) and includes a data distributor. The distributor is a message-oriented component to collect and redistribute the in- and outbound messages between components. In this use case, this module is a publish/subscribe system that receives the data from edge tier in different queues and publish the message received to the modules in the platform that have subscribed to each queue. The technological solution for the Edge Broker is RabbitMQ. RabbitMQ [19] is an open source message broker that supports multiple messaging

protocols such as Advanced Message Queuing Protocol (AMQP) [30].

Converters: Software components or modules to translate edge cloud interface (IoTA CEP Events) into database interface (MIMOSA API Rest [23]) or files system interface (HDFS API). The converter is

implemented using the translation capabilities of an Enterprise Service Bus (ESB) named WSO2 [31].

Data Storage systems store the information coming from Cyber Physical Systems (CPS) and results obtained from data analysis maintenance actions and algorithms. Two storage systems:

a) MIMOSA DB: is a database compliant with the ISO-13374 Standard (Condition Monitoring and Diagnostic of Machines) [13]. One of the main objectives of the MIMOSA CBM architecture is to

standardize the information flow between the various blocks, so that equipment from different vendors

could be interoperable. The MIMOSA database is deployed in SQL Server and API REST is used to access data from applications.

b) Hadoop Distributed File System (HDFS): is a distributed file system designed to run on commodity

hardware. Designed to be deployed on low-cost hardware, HDFS is highly fault-tolerant and provides high throughput access to application data, which makes suitable for applications that have large data sets.



Batch Processing: data analysis and processor mechanisms to enable the management of large volumes of data, fetched from storage systems and process on demand. Implemented using Apache Spark [32] [ref]. The batch processor units implement the offline analytics capabilities of the platform. These technologies are (1) Root Cause Analysis (RCA) powered by Attribute Oriented Induction Clustering [33] and (2) Remaining Useful Life (RUL) powered by Time Series Forecasting.

API or WS: To interact with the Enterprise Tier an API offering services is provided. This component provides information and functionality (for example RUL) to components external to the platform such as HMI, applications (ERP) or even other platforms that lack certain maintenance algorithms.

Figure 6-6: Platform Tier open source alternative

The enterprise level is concerned with the applications that integrate information from one/several sites to enhance the global decision-making process using monitoring through Human Machine Interfaces (HMI)

and data aggregation and analysis.

6.2.4.2 System Architecture for AZURE BASED CLOUD platform

The architecture implementation also agrees with the three-tier architecture followed in Mantis (Edge tier, Platform tier and Enterprise tier). The Edge Tier is common to that of the previous alternative (see Figure 6-5).

The platform is built using the Microsoft Azure cloud services.

The system has been designed using a typical pattern in big data scenarios known as “ lambda architecture” , with uses three layers to solve the computing problem: speed layer, batch layer and serving layer.

The batch layer holds an immutable, read-only master database, and it pre-computes a batch view with

indicators and aggregated data.

The speed layer deals with recent data only and executes quick algorithms (rules and machine learning algorithms) to produce a speed view with alarms and predictions.

The serving layer is composed of the batch view and the speed view mentioned before.

Exploitation and visualization of data relies on the Microsoft Power BI capabilities to show aggregated information, indicators and transient raw data of the monitored assets.



Application of big data techniques, combined with machine learning for pattern identification, and complex event processing for the detection in real time of the learned patterns, is the approach for reaching a high ratio of availability and operational performance.

The Azure services used to build the platform are described below:

Azure Event Hubs: It is a highly scalable publish-subscribe message broker for event ingestion, with a partition-based approach to support the ingestion of millions of events per second.

Azure Stream Analytics: This is an event data processing service for real-time analysis of streaming

data. It uses a SQL-like language to create rules, and can sent requests to the Azure Machine Learning Service to execute algorithms in real time.

Azure Machine Learning: It is a cloud service for the implementation of predictive analytical

solutions. It provides a big number of built-in packages, and allows the customization of new ones.

Azure HDInsight: This is highly scalable solution used to prepare the data in the batch layer. It allows the combination of data and statistical equations providing many possibilities for data

enrichment.

Azure Blob Storage: It is a cloud service to store unstructured data as blobs. A variety of data files can be stored, for example binary, text, documents, multimedia files, etc.

Azure SQL Database: This is a relational database which is used to store alarms and aggregated

values.

Additionally, a business intelligence tool (Microsoft Power BI) is used for reporting and visualization of data. This tool facilitates the representation of information in attractive panels and reports that can be

customized in a very flexible manner.

Figure 6-7: Platform Tier Azure based alternative



6.2.5 Use Case: Philips Healthcare

Characteristics: stationary equipment, single partner

Figure 6-8 PHC Architecture

6.2.5.1 Data Processing Layer

Data Processing can be classified into two major groups namely Batch Data Processing and Real-time Data Processing.

Batch Data Processing

The rationale for Batch Data processing is due to the inherent complexity of unstructured data in volumes and that a correlated/aggregated meaning or results of a complex set of analysis is not needed real-time. A Data Lake implemented over a volume distributed storage technology like Hadoop will be the primary storage of a variety of unstructured data. Data from various sources land on Data Landing Zone from where they are ingested into the Data Lake. Data landing zone is a transient storage to classify data before storing them on specific types of storages optimized for analysis. E.g. Search stores, Query stores.

Data that requires no pre-processing, and utilized as is, are stored directly in a Data Warehouse. Based on the data needs, data extractor/ETL jobs are scheduled in batches to retrieve structured data to be stored in the Data Warehouse. Historic data is maintained in the Data Lake for longer period of time.

Real-time Processing

Real-time processing is the choice for handling data velocity and when the results of analysis are deemed necessary for real-time monitoring and diagnostics. With real-time data feeds, the complexity is to manage a workflow of Data extraction and analytics sinks in the processing pipeline. The rules are stored in a knowledge repository to manage and evolve them independently. It also allows for relatively stateless processing pipes that can be scaled seamlessly. Data streamed into the processing pipeline gets into the Data Lake eventually for historical analysis later. Structured data extracted enroute are stored in Data Warehouse for trending, training and analysis.

Data Landing

Zone

HDFS

Knowledge Repository

Real-time Processing Pipelines

ETL

ETLMapReduce

Data Warehouse

Cubes & Dimensions

Models & Predictions

Enterprise Queuing

Workflow Management

Visualizations & Reports

Real-time Data Processing Layer

Batch Data Processing Layer

Log Analysis Data Extraction

Serving Layer

BI & Analytics Layer

Presentation Layer

Application LayerRemote Monitoring Workspot

BI Rules

Data Lake



6.2.5.2 Serving Layer

The serving layer consists of structured storages optimized for different type of analytics E.g. search, query. The data store is loosely termed as Data Warehouse as it houses sufficiently high quality structured data. Having relevant well-structured data available lays a solid foundation for many different activities including data analysis and monitoring. Access to Data Warehouse including store/retrieve are designed around data abstractions.

6.2.5.3 BI & Analytics Layer

BI and Analytics layer encompasses Business rules, Statistical models and pre-defined data dimensions. This layer is the Brain of entire architecture. This layer also supports sandboxing to develop and test the entities before being deployed into production. Sandboxing provides the capability to test the models in different sizes of data sets.

Business Rules

Business rules can be simple such as parameter value exceeding threshold or could be complex rule sets such as parameter value trends in the last sixty days. The business rules are evaluated at the end of the processing in the pipeline and can result in the creation of an alert needing a service action. Results of rule evaluation can also be used to flag certain exceptional conditions in the Dashboards.

Cubes and Dimensions

Data for analytical processing and visualization are stored in pre-defined and optimized multi-dimensional formats. Cubes and dimensions are handy in multi-dimensional dashboards/reports using slice, dice, drill down/up, roll-up and pivoting.

Models and Predictions

Data analysis is an iterative way of processing data in a way to detect statistical patterns. Models emerging out of these patterns are trained over and over to improve the quality and value of the output. Industry standard machine learning and statistical package like Rapid Miner is used on the top of existing toolkits or using standard data interfaces JDBC/ODBC.

6.2.5.4 Presentation Layer

The primary goal of Presentation layer is to provide detailed and sharp views of end results to a multitude of end-users. The trade-off for the complexity in BI and Analytics layer is the simplicity in the Presentation layer. The characteristics of visualizations include detailed yet crisp views to facilitate mission critical decisions. The choice of technology shall support variety of end-users clients including mobile apps, notebooks and large high-resolution monitors in data centres. The presentation layer also provides a rich toolkit for integration/inclusion/embedding of the visuals in different applications.

6.2.5.5 Application Layer

While a number of applications can be built on the top of the stack, in the purview of this project a single unified Remote Work spot will be the end result.



7 Conclusions

This deliverable is the entry point to all architecture documentation in the MANTIS project, which

describes all aspects of the MANTIS Reference Architecture, Interfaces and Information Model. D2.9

traces its content to the MANTIS requirements elicited in WP1 of the project. The intent of this deliverable is to provide guidance to partners on how to establish a PdM solution, which fits their business characteristics. It does this by identifying key architecture drivers and based on these drivers suggests architecture styles, patterns and topics to consider (such as security concerns). Through the help of reference applications & use-cases, it offers a more concrete view on how one can implement a PdM

solution.

This is the final iteration out of three iterations. The main gap, which is being addressed, is the relationship of the requirements and how they map on the architecture and its decisions. This is a typical phenomenon with several proposed reference architectures: An architecture is presented but it is hard to assess, if and

how it meets the overall objectives. Detailed validation of this is performed and documented in Task 2.4. For the MANTIS reference architecture we wanted to take a different approach: we present the reference architecture, but also the underlying design decisions and rationale. We believe that this will provide more guidance to the intended readers. To this end, we have explained the reference architecture through

different perspectives (mainly functional and logical) connected through architecture drivers with the

platform requirements.



8 List of Tables

Table 2-1: Business Drivers of the Asset Manufacturer ..................................................................... 15

Table 2-2: Business Drivers of the Asset Service Provider ................................................................. 16

Table 2-3: Business Drivers of the Asset End User ............................................................................ 16

Table 2-4 MANTIS Stakeholders ..................................................................................................... 18

Table 2-5: Requirements, directly related to the MANTIS platform ................................................... 19

Table 2-6: Key Functional Goals of MANTIS ARM .......................................................................... 25

Table 2-7: Quality Goals for MANTIS.............................................................................................. 27

Table 2-8: Prioritization of MANTIS quality goals ............................................................................ 31

Table 3-1: MANTIS Feature Model Excerpt I ................................................................................... 37

Table 3-2: MANTIS Feature Model Excerpt II .................................................................................. 38

Table 3-3: MANTIS Feature Model Excerpt III ................................................................................. 39

Table 4-1 Functions vs. Maintenance Type ...................................................................................... 50

Table 5-1 ‘Feature’ X ‘Reference Application’ matrix ........................................................................ 76

Table 6-1 Reference Technologies .................................................................................................... 79



9 List of Figures

Figure 1-1 MANTIS overview ............................................................................................................ 6

Figure 1-2 WP2 structure.................................................................................................................. 8

Figure 1-3 The purpose of a common MANTIS architecture ............................................................. 10

Figure 1-4 MANTIS Architectural Reference Model building blocks .................................................... 13

Figure 2-1 Relationship between architecture Drivers and Architecture Design. .................................. 14

Figure 2-2 Stakeholder overview ...................................................................................................... 17

Figure 3-1: Context of MANTIS and the MANTIS Platform ............................................................. 33

Figure 3-2: Domain Model for MANTIS with Maintenance as central topic ........................................ 35

Figure 4-1 Conceptual Model on condition monitoring ...................................................................... 42

Figure 4-2 MANTIS Information Model Element Categories .............................................................. 44

Figure 4-3 MANTIS data structures and kinds ................................................................................. 45

Figure 4-4 MANTIS data sources and processed data ....................................................................... 46

Figure 4-5 MANTIS data subjects and content ................................................................................. 47

Figure 4-6 Data characteristics within MANTIS ............................................................................... 47

Figure 4-7 MANTIS data description as consolidated from partner input in task 1.3 .......................... 48

Figure 4-8 MANTIS Functional Model ............................................................................................. 49

Figure 4-9 Function Tree for MANTIS PdM .................................................................................... 50

Figure 5-1 Overview on MANTIS Reference Architecture .................................................................. 56

Figure 5-2 OSGi Service Gateway Architecture [16] .......................................................................... 61

Figure 5-3 Digital Identity ............................................................................................................... 65

Figure 5-4 Bel Lapadullas ................................................................................................................ 66

Figure 5-5 Security Policy Structure ................................................................................................ 69

Figure 5-6: Policy enforcement framework IND2UCE......................................................................... 71

Figure 5-7: Policy Administration Point (PAP)................................................................................. 72

Figure 5-8: Policy Management Point PMP ..................................................................................... 73

Figure 5-9: Policy Retrieval Point PRP ............................................................................................ 73

Figure 5-10: Policy Decision Point (PDP) ........................................................................................ 74

Figure 5-11: Policy Information Point (PIP) ..................................................................................... 74

Figure 5-12: Policy Enforcement Point (PEP) .................................................................................. 74

Figure 5-13: Policy Execution Point (PXP) ...................................................................................... 75

Figure 5-14 Reference Application 1 ................................................................................................ 77



Figure 6-1. Phases in the conventional power production use case 3.3 ............................................... 80



Figure 6-2. Data from MIMOSA database (based on OSA-EAI) and through OSA-CBM to the web user interface. ................................................................................................................................. 82

Figure 6-3 STILL & Ansaldo Architecture ........................................................................................ 83

Figure 6-4 Adira reference architecture ............................................................................................ 84

Figure 6-5: Edge Tier ...................................................................................................................... 85

Figure 6-6: Platform Tier open source alternative ............................................................................. 86

Figure 6-7: Platform Tier Azure based alternative ............................................................................ 87

Figure 6-8 PHC Architecture ........................................................................................................... 88

Figure 11-1 Three-tier architecture .................................................................................................. 98

Figure 11-2 Gateway-Mediated Edge Connectivity and Management Pattern ..................................... 99

Figure 11-3 The producer and consumer of an (Arrowhead) Service ................................................. 100

Figure 11-4 A MANTIS defined local cloud ..................................................................................... 102

Figure 11-5 The interactions between Local (automation) Clouds in MANTIS .................................. 103

Figure 11-6 Microservice service taxonomy ...................................................................................... 104

Figure 11-7 Microservices architecture topology............................................................................... 104

Figure 11-8: Lambda architecture ................................................................................................... 105

Figure 11-9 Kappa Architecture ...................................................................................................... 106

Figure 11-10: Concept view: ECNs, development frameworks, and product implementation ............... 108

Figure 11-11: Type of analytics based on its applications ................................................................. 109

Figure 11-12: A deployment pattern of analytics during its different lifecycle stages .......................... 109

Figure 11-13: Function view: full life-cycle data service .................................................................... 110

Figure 11-14: Industrial Analytics Design Considerations .................................................................. 112

Figure 11-15: Basic architecture of state of the art for smart manufacturing solutions ....................... 113

Figure 11-16: Points of collaboration between Edge Computing and Cloud Computing ...................... 113

Figure 11-17: Lightweight OS architecture [source: Huawei] ............................................................. 115

Figure 11-18: Edge computing and Microservices............................................................................. 116



10 Literature and References

[1] EABOK, “Enterprise Architecture Body of Knowledge (EABOK),” [Online]. Available:

http://www2.mitre.org/public/eabok/developing_an_ea/reference.html. [Accessed 24 03 2016].

[2] ISO/IEC, ISO/IEC/IEEE 42010:2011 "Systems and software engineering - Architecture Description", ISO/IEC, 2011.

[3] Wikipedia, “Wikipedia: Reference Architecture,” [Online]. Available:

https://en.wikipedia.org/wiki/Reference_architecture. [Accessed 22 02 2016].

[4] S. Martinez-Fernandez, P. Medeiros Dos Santos, C. Ayala, X. Franch and G. Travassos, “Aggregating

Empirical Evidence about the Benefits and Drawbacks of Software Reference Architectures,” in

ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Beijing, 2015.

[5] “Internet of Things Architecture (IoT-A),” [Online]. Available: http://www.iot-a.eu/public.

[6] Wikipedia, “Wikipedia: OASIS (organization),” [Online]. Available:

https://en.wikipedia.org/wiki/OASIS_%28organization%29. [Accessed 03 03 2016].

[7] “Wikipedia Reference Model,” [Online]. Available: https://en.wikipedia.org/wiki/Reference_model.

[8] P. F. Brown, R. Metz and B. A. Hamilton, Reference Model for Service Oriented Architecture 1.0,

2006.

[9] ISO, “ISO 13374,” [Online]. Available: http://www.iso.org/iso/catalogue_detail?csnumber=21832.

[10] “INDUSTRIAL INTERNET REFERENCE ARCHITECTURE,” Industrial Internet Consortium,

[Online]. Available: http://www.iiconsortium.org/IIRA.htm.

[11] J. Rowley, “The wisdom hierarchy: Representations of the DIKW hierarchy,” Journal of Information Science, vol. 33, no. 2, pp. 163-180, 2007.

[12] MANTIS Consortium, D1.4 MANTIS Platform Requirements V1, MANTIS Consortium, 2015.

[13] ISO, ISO 13374-1 - Condition monitoring and diagnostics of machines - Data processing, communication and presentation - Part1: General guidelines.

[14] ISO, ISO 13374-1 Condition monitoring and diagnostics of machines - Data processing, communication and presentation - Part1: General guidelines.

[15] “OSGi,” [Online]. Available: www.osgi.org.



[16] “Wikipedia - OSGi,” Wikipedia, [Online]. Available: https://en.wikipedia.org/wiki/OSGi.

[17] “OpenAPI Initiative,” [Online]. Available: https://www.openapis.org/.

[18] “Docker,” [Online]. Available: https://www.docker.com/.

[19] “RabbitMQ,” [Online]. Available: https://www.rabbitmq.com/.

[20] “Apache Kafka,” [Online]. Available: https://kafka.apache.org/.

[21] “Apache Storm,” [Online]. Available: https://storm.apache.org/.

[22] “Apache Hadoop,” [Online]. Available: http://hadoop.apache.org/.

[23] MIMOSA, “MIMOSA An Operations and Maintenance Information Open System Alliance,” 2017.

[Online]. Available: http://www.mimosa.org/mimosa/. [Accessed 17 May 2017].

[24] “Kubernetes,” [Online]. Available: https://kubernetes.io/.

[25] “Chef,” [Online]. Available: https://www.chef.io/.

[26] Felix, “50 of the most important Raspberry Pi Sensors and Components,” 2016. [Online]. Available:

https://tutorials-raspberrypi.com/raspberry-pi-sensors-overview-50-important-components/. [Accessed 18 May 2017].

[27] J. Fernandez-Anakabe, E. Zugasti, I. Garitano, U. Zurutuza, M. Anasagasti, M. Mondragon, and F.

Larrinaga, ““Implementation of a Reference Architecture for Cyber Physical Systems to support

Condition Based Maintenance “,” 5th International Conference on Control, Decision and Information

Technologies CODIT, Thessaloniki , 2018.

[28] P. V. a. I. M. C. Hegedus, “The MANTIS architecture for proactive maintenance",” 5th International

Conference on Control, Decision and Information Technologies CODIT, Thessaloniki, 2018.

[29] “http://www.iot-a.eu/public,” [Online].

[30] S. Vinoski, “Advanced Message Queuing Protocol 10: 87-89,” IEEE Internet Computing, 2006.

[31] “wso2,” [Online]. Available: https://wso2.com/.

[32] “Spark,” [Online]. Available: https://spark.apache.org/.

[33] J. C. Y. a. C. N. Han, “Knowledge Discovery in Databases: An Attribute-Oriented Approach. In

Proc. 18th Int. Conf. Very Large Data Bases,” Vancouver, 1992.



[34] Arrowhead, “The Arrowhead Project,” [Online]. Available: http://www.arrowhead.eu/. [Accessed 10

10 2015].

[35] J. Lewis and M. Fowler, “Microservices - a definition of this new architectural term,” 2014. [Online].

Available: https://martinfowler.com/articles/microservices.html.

[36] J. Forgeat, “Data processing architectures – Lambda and Kappa,” ericsson, 19 Nov 2015. [Online].

Available: https://www.ericsson.com/research-blog/data-knowledge/data-processing-architectures-lambda-and-kappa/.

[37] Edge Computing Consortium, “Edge Computing Reference Architecture 2.0,” Edge Computing

Consortium, 2017.

[38] Industrial Internet Consortium, “The Industrial Internet of Things, Volume T3: Analytics

Framework,” Industrial Internet Consortium, 2017.

[39] IEC, “Edge Intelligence,” IEC, 2017.

[40] J. Knodel and M. Naab, Pragmatic Evaluation of Software Architectures, Springer Publishing Company, 2016.

[41] Mondragon University, "Wireframes Web App," 2015. [Online]. Available:

http://teamsite.youris.com/sites/cityfied/WP2/WP2%20Tasks/Task%202.2/ST%202.2.2/DesignSolutionReport%20(Spanish).pdf.

[42] Mondragon University, “Design Solution Report,” 2015. [Online]. Available:

http://www.mondragon.edu.

[43] ISO, “ISO/IEC 25000 Software engineering – Software product Quality Requirements and Evaluation

(SQuaRE),” ISO, 2011.

[44] N. Rozanski and E. Woods, Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives, Addison-Wesley Professional, 2011.

[45] N. Kazman, L. Bass, M. Webb and G. Abowd, “SAAM: a method for analyzing the properties of

software architectures,” in ICSE 94, Los Alamitos, CA, USA, 1994.

[46] R. Kazman and M. Klein, “Performing architecture tradeoff analysis,” in Proceedings of the third international workshop on Software architecture (ISAW '98), 1998.

[47] P. Kruchten, “The 4+1 View Model of Architecture,” IEEE Software, pp. 42-50, 1995.

[48] D. Garlan, F. Bachmann, J. Ivers, J. Stafford, L. Bass, P. Clements and P. Merson, Documenting Software Architectures: Views and Beyond, Addison-Wesley Professional, 2010.



[49] K. Pohl, H. Hönninger, R. Achatz and M. Broy, Model-Based Engineering of Embedded Systems:

The SPES 2020 Methodology, Springer Publishing Company, 2012.



11 Appendix

11.1 Key Architectural Concepts

This chapter describes central solutions concepts that are used in the overall MANTIS system. Respective styles, patterns, tactics, custom solutions are documented here.

11.1.1 Architectural Styles & Patterns

An architecture pattern is a simplified and abstracted view of a subset of system architectures that is recurrent across a set of system architectures in a domain, yet allowing for variants.

IoT-A and the industrial internet consortium [10] have identified a set of architecture patterns for IoT and industrial internet systems, which hold also for MANTIS solutions, as they are CPS-based and with that also in the scope of IoT and industrial internet. The most relevant patterns presented in the following.

11.1.1.1 Three-Tier Architecture

According to the IIC-RA [10] three-tier architecture pattern comprises edge, platform and enterprise tiers.

These tiers play specific roles in processing the data flows and control flows involved in usage activities. They are connected by three networks, as shown in Figure 11-1 Three-tier architecture:

Figure 11-1 Three-tier architecture

The edge tier collects data from the edge nodes, using the proximity network. The architectural characteristics of this tier, breadth of distribution, location, governance scope and the nature of the proximity network, vary depending on the specific use cases.

The platform tier receives, processes and forwards control commands from the enterprise tier to the edge tier. It consolidates processes and analyses data flows from the edge tier. It provides management functions for devices and assets. It also offers non-domain specific services such as data query and analytics.

The enterprise tier implements domain-specific applications, decision support systems and provides interfaces to end-users including operation specialists. The enterprise tier receives data flows from the edge and platform tier. It also originates control commands to the platform tier and edge tier.



Different networks connect the tiers: The proximity network connects the sensors, actuators, devices, control systems and assets, collectively called edge nodes. It typically connects these edge nodes, as one or more clusters related to a gateway that bridges to other networks. The access network enables connectivity for data and control flows between the edge and the platform tiers. It may be a corporate network, or an overlay private network over the public Internet or a 4G/5G network.

11.1.1.2 Gateway-mediated edge connectivity and management According to the IIC-RA [10] the gateway-mediated edge connectivity and management architecture pattern comprises a local connectivity solution for the edge of an Industrial Internet System (IIS), with a gateway that bridges to a wide area network as shown in Figure 11-2. The gateway acts as an endpoint for the wide area network while isolating the local network of edge nodes. This architecture pattern allows for localizing operations and controls (edge analytics and computing). Its main benefit is in breaking down the complexity of CPS systems, so that they may scale up both in numbers of managed assets as well as in networking. However, it may not be suited to systems where assets are mobile in a way that does not allow for stable clusters within the local network boundaries.

Figure 11-2 Gateway-Mediated Edge Connectivity and Management Pattern

The edge gateway may also be used as a management point for devices and assets and data aggregation point where some data processing and analytics, and control logic are locally deployed. The local network may use different topologies: In a hub-and-spoke topology, an edge gateway acts as a hub for connecting a cluster of edge nodes to each other and to a wide area network. It has a direct connection to each edge entity in the cluster allowing in-flow data from the edge nodes, and out-flow control commands to the edge nodes. In a mesh network (or peer-to-peer) topology, an edge gateway also acts as a hub for connecting a cluster of edge nodes to a wide area network. In this topology, however, some of the edge nodes have routing capability. As result, the routing paths from an edge node to another and to the edge gateway vary and may change dynamically. This topology is best suited to provide broad area coverage for low-power and low-data rate applications on resource-constrained devices that are geographically distributed. In both topologies, the edge nodes are not directly accessible from the wide area network. The edge gateway acts as the single entry point to the edge nodes and as management point providing routing and address translation.



11.1.1.3 Edge-to-Cloud The Edge-to-Cloud architecture pattern contrasts with the gateway-mediated pattern as it assumes a wide-area connectivity and addressability for devices and assets. This means that the edges are directly accessed from the cloud. While this allows a direct connection to the edges and therefore results in a lower latency, this pattern has several disadvantages including:

Higher computational demands. The edge must be able to communicate with the cloud directly; this requires higher computational power and wide-area access for all edges.

Stronger protocol dependence. All edges must adhere to the communication protocol with the cloud. If a different protocol or technology should be used, all edge nodes must be upgraded or replaced.

More difficult security management. All edge nodes now represent an attack surface and entry point for the whole system.

11.1.1.4 The service oriented approach

The MANTIS approach can be based on the solutions explored in the Arrowhead. The Arrowhead Framework utilizes the previously discussed SOA approach tailored to current use case (see 6.2.1).

The Arrowhead project aims nothing less than to create a full-scoped framework enabling collaborative automation by networked embedded devices. The grand challenges it faces are the creation of interoperability and integrability of almost any device [34].

The Arrowhead Framework created in the project uses standard UML external interface description symbols when representing elements and their services. However, in the context of Arrowhead, elements can represent a wide range of entities and not just software components. Figure 11-3 depicts the generalization of a service consumption.

Figure 11-3 The producer and consumer of an (Arrowhead) Service

In this context, an Arrowhead System can be a lot of things. For example:

Cyber-Physical Systems (CPS): sensors, actuators

Cloud-based environments: data storages

Logical units: distributed algorithms

Various (human) stakeholders: system operators, developers, etc.

The main idea here is that such systems (functionally contained abstractions) provide and consume

services from one another. This way, a static architecture (”big picture”) can be drawn that describes the various entities involved. We can establish services between these systems. An Arrowhead Service can be from a wide range of functionalities:

Data storage or database querying

HMI visualization

Service Discovery

Event subscription, event notification

System configuration

Etc.

This is also in accordance with the previously explored relevant frameworks and technologies. Services are defined by their information model (that is exchanged during servicing a client) as defined in 4.2.



Services can be realized by an arbitrary number of service providers and consumers. The idea is that the systems involved only have to speak the same language (communication, semantic and data model). Services are created to provide certain functionality by multiple entities.

This way, services are the building blocks derived from the functional decomposition process.

11.1.1.4.1 The risks of NOT using the SOA approach

The following disadvantages have been identified in the Arrowhead project for not using the SOA

approach:

Hard to enable a common understanding

Every integration will be unique and dependent, no reuse

Integration will be heavily time consuming compared to a clean service approach proposed in the Arrowhead project

Several adoptions (continuously increasing)

Several technologies (continuously increasing)

Increased product dependency

Increased vendor dependency

Very hard to achieve Interoperability and Integrability

11.1.1.4.2 Local Automation Clouds

The Arrowhead Framework has defined closed entities that can be instantiated anywhere for any purpose. These entities are called Arrowhead Local Clouds and their purpose is to bring the IoT world into the industrial automation world with specific restrictions stemming from the nature of the industry.

Security and safety is still the primary objective (as declared in the MANTIS vision in section 1.2.1). To achieve this, we should follow the architectural concept of creating local, closed environments (as also stated in the MANTIS project proposal, e.g. in the STILL use-case).

In these local clouds, we define various functionalities (services) provided by numerous elements (that are complex systems on their own).

What makes a Local Cloud?

In the Arrowhead Framework, Local Clouds are defined by an abstraction with the following principles:

Local Clouds have a well-defined purpose

They have their own set of stakeholders: operators, designers, developers

It is a System-of-systems

Logically closed entities: they have their functional, network, physical borders

Can be governed by their own instance of the Arrowhead Core Systems

As an example, we can define local clouds for a MANTIS use-case, where local processing is done on local data, see Figure 11-4. In this visualization, the arrows show the services and the rectangles represent the functional blocks (systems).



Figure 11-4 A MANTIS defined local cloud

If we follow this architectural approach, we will have to define the elements (“systems”), describe them. This way, we are defining Systems (in the Arrowhead sense).

After this process, we also have to define the services (with their information and functional model). The last step is creating the servicing capabilities between the defined systems.

In this sense, a global MANTIS cloud (defined in the project proposal and assumed in the other WPs) is also a Local Cloud but its systems are highly distributed compared to other (truly local) Local Clouds. Figure 11-5 depicts how the Local Clouds can be presented as such (the data flow is consistent with the preliminary WP4 architecture).



Figure 11-5 The interactions between Local (automation) Clouds in MANTIS

11.1.1.4.3 Microservices

The microservice software architecture is a style for service-oriented development, whose popularity is increasing now because of growing adoption of distributed computing. It refers to a method for “ designing software applications as suites of independently deployable services” [35]. Compared with the monolithic style of building an application the microservice architecture enables changes of modules without the

necessity of rebuilding and re-implementing the whole system. Microservices are independently created and managed services/objects. They can be included in various applications. The application programming interfaces (APIs) expose their functionality and serve as access points. Each one implements a set of narrowly, context bound functions with a high level of freedom.

Microservices architectures have a limited service taxonomy as illustrated in Figure 11-6. Functional services support specific business operations or functions. Infrastructure services support non-functional

tasks such as security including authentication and authorization, auditing, logging and monitoring. Infrastructure services do not expose a public interface. The microservices architecture treats them as private shared services only available internally to other services. Externally clients can request functional

services through the API layer.



Figure 11-6 Microservice service taxonomy

A microservice architecture decomposes the system into smaller, modular, collaboratively working components, which are much easier to build and upgrade than a monolithic application as a whole. Each microservice is an API-accessible process with its own database. There are several models for deployment available such as serverless deployment, service instance per container, service instance per virtual machine or multiple service instances per host.

Figure 11-7 Microservices architecture topology

Microservice architecture is an enabler for the development of IoT platforms:

1. Microservices are developed with modern, flexible project management methodologies such as

agile and DevOps methods. These methods enable designing, changing and verifying the

microservices in a continuous delivery process. 2. Microservices provide scalability for distributed systems. A system is composed of multiple

independent microservices working in loosely coupled collaboration. A microservice can run on a single machine or as multiple instances on different machines.

3. Microservice architecture enables the realization of microservices in different technologies. 4. Microservices offer flexibility for the construction of evolving models.

The microservice architecture style addresses many issues commonly found in large monolithic applications and complex SOA architectures. One of its fundamental concepts is a share-as-little-as-possible paradigm that depends strongly on the concept of a bounded context. In contrast, SOA is a share-as-much-as-

possible architecture style that emphasises abstraction and business functionality reuse.



11.1.1.5 Lambda & Kappa

Over the past few years, there have been many discussions about how to design a good real-time data processing architecture [36]. A good real-time data processing architecture needs to be fault-tolerant and scalable; it needs to support batch and incremental updates, and must be extensible.

One important milestone in these discussions was Nathan Marz, creator of Apache Storm, describing what we have come to know as the Lambda architecture. The Lambda architecture has proven to be relevant to many use-cases and is indeed used by a lot of companies, for example Yahoo and Netflix. But of course, Lambda is not a silver bullet and has received some fair criticism on the coding overhead it can create.

In the summer of 2014, Jay Kreps from LinkedIn posted an article describing what he called the Kappa architecture, which addresses some of the pitfalls associated with Lambda. Kappa is not a replacement for Lambda, though, as some use-cases deployed using the Lambda architecture cannot be migrated.

It can be challenging to accurately evaluate which architecture is best for a given use-case and making a wrong design decision can have serious consequences for the implementation of a data analytics project. Now let’ s get into greater detail about the two data processing architectures.

Figure 11-8: Lambda architecture

The Lambda Architecture, shown in Figure 1, is composed of three layers: batch, speed, and serving.

The batch layer has two major tasks: (a) managing historical data; and (b) recomputing results such as machine learning models. Specifically, the batch layer receives arriving data, combines it with historical data and recomputes results by iterating over the entire combined data set. The batch layer operates on the full data and thus allows the system to produce the most accurate results. However, the results come at the cost of high latency due to high computation time.



The speed layer is used in order to provide results in a low-latency, near real-time fashion. The speed layer receives the arriving data and performs incremental updates to the batch layer results. Thanks to the incremental algorithms implemented at the speed layer, computation cost is significantly reduced.

Finally, the serving layer enables various queries of the results sent from the batch and speed layers.

Figure 11-9 Kappa Architecture

The Kappa architecture is shown in Figure 2. One of the important motivations for inventing the Kappa architecture was to avoid maintaining two separate code bases for the batch and speed layers. The key idea is to handle both real-time data processing and continuous data reprocessing using a single stream processing engine. Data reprocessing is an important requirement for making visible the effects of code changes on the results. As a consequence, the Kappa architecture is composed of only two layers: stream processing and serving. The stream processing layer runs the stream processing jobs. Normally, a single stream processing job is run to enable real-time data processing. Data reprocessing is only done when some code of the stream processing job needs to be modified. This is achieved by running another modified stream processing job and replying all previous data. Finally, similarly to the Lambda architecture, the serving layer is used to query the results.

So when should we use one architecture or the other? As is often the case, it depends on some characteristics of the application that is to be implemented. Let’ s go through a few common examples:

A very simple case to consider is when the algorithms applied to the real-time data and to the historical data are identical. Then it is clearly very beneficial to use the same code base to process historical and real-time data, and therefore to implement the use-case using the Kappa architecture.

Now, the algorithms used to process historical data and real-time data are not always identical. In some cases, the batch algorithm can be optimized thanks to the fact that it has access to the complete historical dataset, and then outperform the implementation of the real-time algorithm. Here, choosing between Lambda and Kappa becomes a choice between favouring batch execution performance over code base simplicity.

Finally, there are even more complex use-cases, in which even the outputs of the real-time and batch algorithm are different. For example, a machine learning application where generation of the batch model requires so much time and resources that the best result achievable in real-time is computing and approximated updates of that model. In such cases, the batch and real-time layers cannot be merged, and the Lambda architecture must be used.



11.1.2 Edge Computing

As with many new technologies, Edge computing is on the verge of rapid breakthrough. Mostly driven by

manufacturing, Edge is an answer to the ever-growing adoption of cloud technologies, while maintaining

processing speed. As cloud-based computing is particularly well suited for processing of large amount of information, speed is not its main strong suit. One aspect of this is the transportation of on premise data to the cloud itself. For applications which have to rely on time-critical decision in the sub-milliseconds range, this delay may not acceptable.

This is where Edge computing comes in, where the main idea is to process data as closely to the source as possible. Rather than sending all data to the cloud, you first process the data locally. This processing

can be as simple as filtering relevant data, but can also include virtualized services, like machine learning application and real-time visualizations.

The advantages of edge can be distinguished in three main topics:

Latency, if reaction speed is critical, latency is very important. An example might be the analysis on in-process sensors which need to be able to trigger a valve in sub-milliseconds. This can never be achieved if the system relies on a cloud-connection only.

The second topic is security and privacy. The Edge device can be used to set-up hardware encrypted data

connections to the cloud, to enable better protected pathways to the cloud. But it also enables to process data locally, preventing the requirement to send all data to the cloud, either solving privacy regulation or the potential risk of losing operation-critical data.

The final aspect is that with a full cloud deployment, the connection to the cloud is critical. An unreliable

connection to the cloud would mean that (business critical) data processing halts or would be unreliable

at best. With Edge computing, this reliance on the connecting to the cloud lowers, because data is processed near the source.

11.1.2.1 Where to do the processing in the EDGE

Make a decision about where to perform a specific processing in the Edge is nothing obvious, there are several aspects to be taken into consideration, such as the device/product where the deployment will be

performed, characteristics of the data at edge level, type of analytics to apply, the life-cycle of analytics, the functional perspective and design considerations. This section is a reflection on the subject in the spirit of providing some clues on how to approach the challenge.

Smart edge computing assets, systems, and gateways are digitalized, network-based, and intelligent. They provide ICT resources such as networks, computing, and storage, and can be logically abstracted as Edge Computing Nodes (ECNs). To suit the typical application scenarios of ECNs, the architecture defines four

types of ECN development frameworks. Each framework includes an operating system, functional modules,

and integrated development environment to meet the needs of various scenarios. Based on the ECN development frameworks combined with the specific hardware platform required by ECNs, six types of products can be built, as outlined in Figure 11-10 [37] : Embedded Controllers, Independent Controllers, Terminal Perception, Smart Gateways and Edge Distributed Servers.



Figure 11-10: Concept view: ECNs, development frameworks, and product implementation

Figure 1: Concept view: ECNs, development frameworks, and product implementation

Edge data has specific characteristics. It is generated at the network edge and includes machine running

data, environment data, and information system data. It features high throughput, fast flow movement,

diversity, strong correlation, and high requirements on real-time analysis and processing. Compared with business big data scenarios such as Internet, smart analysis of edge data has the following characteristics and differences:

Causal relationship vs. association relationship: Edge data is mainly targeted at smart assets that generally run with explicit input and output causal relationships, whereas business big data focuses on data association relationships.

High reliability vs. low reliability: Industries such as manufacturing have high requirements on the accuracy and reliability of models. If the accuracy or reliability is low, property loss or even personal injury may occur. In contrast, business big data analysis generally has low requirements on

reliability. It is required that the edge data analysis result be explainable. Therefore, black-box deep learning is restricted in some application scenarios. The combination of the traditional mechanism models and data analysis methods is the innovation and application direction of smart analysis.

Small data vs. big data: Assets such as machine tools and vehicles are designed and manufactured by people. Most data in their running process is predictable, and only data generated in abnormal or critical conditions is truly valuable. On the other hand, business big data analysis requires mass data.

Another aspect to consider is the different types of analytics that can be performed and the types of data

required to perform the analytics. Figure 11-11 [38].



Figure 11-11: Type of analytics based on its applications

Figure 2: Type of analytics based on its applications

Baseline analytics detect irregular behavior of the asset within milliseconds of the actual event. The data used to perform these analytics is usually local to the asset under consideration and relies on data acquired from the asset when it was working normally. Diagnostic analytics that identify the root cause of the anomaly such as a failing bearing in a motor requires previous knowledge of fault states. Diagnostics

results can be returned in the order of minutes. Prognostic analytics that tell you the remaining useful life of a bearing can take in the order of hours to return a result and requires access to multiple types of data and from multiple sources to make the prediction.

Nor should we forget that analytics has its own life cycle. Deployment of analytics typically consists of three steps: 1) train a (predictive) analytics model, b) test and validate the model on previously unseen

data, and c) deploy the model to make predictions on real (streaming) data. Training and deployment of analytics models can be broadly categorized into the following three workflows: 1) training of the model and the deployment of the model is done in the cloud, 2) training of the model is done in the cloud, while the model is deployed at the edge, and 3) training and deployment of the model is done at the edge.

Figure 11-12 [38] illustrates the three workflows for the training and deployment of analytic models.

Figure 11-12: A deployment pattern of analytics during its different lifecycle stages

Figure 3: A deployment pattern of analytics during its different lifecycle stages

For the first workflow, once the model is trained in the cloud, the model is usually deployed as a web service. Data acquired from sensors at the edge is sent to the model via the web service. Predictions from the model can be returned to the edge through the web service if required. In the second workflow, once

Edge Edge/Cloud Cloud



the model is trained in the cloud the model is deployed to the edge. For deployment, the model is usually exchanged between the cloud and analytic algorithms running at the edge using a standard interchange formal such as PMML1 or JSON2. Analytic models can be deployed at the edge on heterogeneous

computing elements, such as a CPU, GPU or FPGA. For example, FPGAs are ideal deployment targets

for very low latency applications.

From a functional point of view (Figure 11-13) [37], the entire data analytics process consists of a series of specific modules that can, or will not, be deployed on the same node. Where to deploy data preprocessing, data analysis, data distribution and policy execution, and visualization and storage

functionalities will be a decision to make.

Figure 11-13: Function view: full life-cycle data service

Figure 4: Function view: full life-cycle data service

Data preprocessing includes filter, clean, aggregate, and optimize (including dirty data elimination) raw

data and perform semantic parsing. Data analysis relates to streaming data analysis, process data in real time so that events and ever-changing service conditions and requirements can be responded to quickly, accelerating the continuous analysis of data. It also provides the common statistical model library, and supports the algorithm integration of models such as statistical models and mechanism models, and

support model training methods such as lightweight deep learning. Data distribution and policy execution executes policies locally based on predefined rules and data analysis results, or forward data to the cloud

or other ECNs for processing. Data visualization and storage, using technologies such as TSDB can significantly conserve storage space and meet the requirements on high-speed read and write operations.

Finally the following design considerations [38] should be taken into account in order to decide where to

do the processing:

1. Scope: Ultimately, it is the derived information (not the raw data) and how it can be acted on that determines what kinds of analytics are deployed, and where. For instance, if the goal is to optimize

machine uptime at one site, then analytics performed on data gathered there may be sufficient. In

this case, the analytics can be performed anywhere, provided that, if done remotely, the normal local operation is not critically dependent on network latency and the availability of the analytics results. On the other hand, if the value proposition is to optimize production across sites requiring comparison of factory efficiencies then analytics needs to be performed on data gathered from these sites and thus be available in a higher tier of the system architecture.



2. Response time and reliability: In an industrial setting, some problems require deterministic analysis, computation and response; others can be done after the fact. The former almost always requires analytics to be local for reliability and performance.

3. Bandwidth: The total amount of data generated by these sensors, together with data gathered from

the control systems can be huge. The increased network and infrastructure required to fuse data from one domain with others will be compensated by the creation of valuable insights.

4. Capacity: In some cases, it may be optimal to perform the analytics at a particular tier in a system architecture, but the existing infrastructure may not be able to support it, so a different tier is selected. Key properties of the ICT infrastructure include latency, bandwidth and computational capacity.

5. Security: The value from moving data must be balanced with concerns for transferring raw data outside of controlled areas, and the associated costs. It may be more efficient to perform some

analytics locally and share necessary summary, redacted or anonymized information with other domains.

6. Volume: All data needs to be stored, at least temporarily. The storage required depends on the application.

7. Velocity: Industrial measurements are typically captured cyclically. High-frequency data, such as vibration or acoustics data from aircraft engines and wind turbines, can significantly increase the speed of data processing needed. Another consideration is transient events where readings must be captured

with accurate time recording to determine order of occurrence, causality and root cause. With these low latency requirements, analytics using high velocity data is better performed close to where the data is measured.

8. Variety: When many pieces of equipment have been acquired over the years with dissimilar controls,

interfaces and available data, effective analytics depends on shared information models to understand the data highly varying in both format (syntax) and content (semantics) to deliver the expected insights. Appropriate levels of Interoperability are required to deal with this challenge.

9. Analytics maturity: Analytics involves the processing of raw data (measurements) into descriptions (information) and then contextualizing the results (knowledge) and benefiting from historical

experience (wisdom). The maturity of this process is not limited to where the analytics can be performed and as a design consideration is equally valid regardless of where the analytics are performed.

10. Temporal correlation: One of the common issues in IIoT systems is correlating data between multiple

sensors and process control states, including the temporal order at which the data is generated.

Applying analytics closer to where the data is generated reduces the burden of correlation when the analytics is applied.

11. Provenance: Performing analytics at a lower architecture tier maintains the genuine sources of the data as they progress through the IIoT system.

12. Compliance: To illustrate how compliance may impact the analytics as a design consideration, national security is used as an example. National security concerns may place restriction on the architectural decision about data management and sharing with government regulations in industries like aerospace

and defense. This will influence where the analytics must be placed to meet the regulatory

requirements, for example, possibly preventing performing large-scale computation in a public cloud facility to lower the cost.

The combination of all these factors determines what capabilities are required and where the analytics will be deployed. Generally speaking, which analytics and where the processing is located depends on the maximum acceptable network latency and jitter in response to events, criticality of the analytics in the

normal operations and the cost of uploading large amounts of data. From the deterministic response,

reliability and resilience perspectives, it is optimal to perform analytics close to the sources of the data, and the analytics results need to be accessible for decision-making. In most systems, some form of hybrid approach with local and centralized analytics will be required.



Figure 11-14: Industrial Analytics Design Considerations

Figure 5: Industrial Analytics Design Considerations

Figure 11-14 [39]shows the basic architecture of state of the art for smart manufacturing solutions and

the nodes where the different processes required by the edge ecosystem could be deployed. Sensors and actuators (e.g. robot, numeric control (NC)) are connected to programmable logic controllers (PLCs) or distributed control systems (DCSs), which communicate with them through a field network or field bus

and control them in real-time. Some sensors (e.g. a video camera with a video analytics function) and actuators (e.g. an intelligent robot) have computing and/or storage capabilities. Usually only limited computing and/or storage capabilities are supported by PLCs and/ or DCSs. ECNs (e.g. industrial PCs, gateways) are connected to PLCs/DCSs and sensors/actuators directly through a field network/bus or

indirectly through an information network (e.g. Ethernet). ECNs usually have richer computing and/or storage capabilities and some edge computers may act as a gateway between OT and IT. An IT system can be implemented on premise or in the cloud.



Figure 11-15: Basic architecture of state of the art for smart manufacturing solutions

Figure 5: Basic architecture of state of the art for smart manufacturing solutions

11.1.2.2 Collaboration of Edge Computing and Cloud Computing

Cloud computing is suitable for non-real-time, long-period data and business decision-making scenarios,

while edge computing plays an irreplaceable role in scenarios such as real-time, short-period data and local decision-making.

Edge and cloud computing are two important foundations for the digital transformation of industries. The collaboration between them in respect of network, service, application, and intelligence will help support more scenarios and unleash greater value in industry digitalization.

The network edge side needs to support multiple types of network interfaces, protocols, and topologies,

real-time service processing, fixed latency, data processing and analysis, distributed intelligence, and security and privacy protection. The cloud side cannot meet such requirements. Edge computing needs to collaborate with cloud computing in networks, services, applications, and intelligence. Figure 11-16 [37]

Figure 11-16: Points of collaboration between Edge Computing and Cloud Computing

Figure 8: Points of collaboration between Edge Computing and Cloud Computing

Edge Clouds are not here to replace data centers but to complement them. Edge clouds are useful in

reducing latency and improving user experience, but the amount of processing power and storage is orders of magnitude below traditional clouds. So, for example, an application that needs to support very low



end-to-end delay can have one component running in the Edge Cloud and other components running in the distant Cloud.

In edge computing, data processing and compute power is distributed and decentralized to the extreme

ends of the network. This is not an entirely new concept. From an architecture principle perspective, we

have seen such distributed models before when it came to geographically distributed grid computing and peer-to-peer networking.

Edge computing has its obvious advantages over centralized computing. It provides for improved Quality of Service (QOS) given that lower data volumes lead to decreased latency and improved overall

transmission costs. It also decreases dependency on the core computing environment, be it a cloud or an on-premise environment, therefore eliminating the centralized server/cloud as a potential bottleneck.

Additionally, a distributed architecture provides for greater scalability as it is easier to virtualize processing capacity on an as-needed and real-time basis.

However, the need for edge computing today is not driven by these advantages but to address the need to perform specialized computing on local IIoT devices. For example, think about intelligent robots who while still connected to a centralized network will still need to be able to independently act. It is also the same for sensors and physical equipment at a manufacturing unit, which should be able to self-heal or regulate themselves as possible.

Edge computing has also become possible today because of the ability of smaller devices to now have immense processing power. This doesn't mean that edge computing will blow over centralized cloud computing. These worlds will need to co-exist and a hybrid will form. There were good reasons for businesses to move from on-premise computing to cloud computing. Cloud computing allows for better

overall management of applications and data given its centralized architecture model. They are also simpler to create and support. The intelligent robot and the intelligent sensor will still need to

communicate with a centralized cloud to receive patches and also send useful updates that could further be distributed by the centralized cloud to other connected devices. The future will be a hybrid model where edge-based processing and cloud-based modeling come together.

Lightweight OS, Microservices and Virtualization

In Edge Computing, data will be processed, analyzed and aggregated at the network edge near things or data sources. The IoT gateway is the perfect host of all these capabilities. The main edge computing

functions of IoT gateway include cloud offloading, private data filtering, data aggregation, etc. The OS running on IoT gateways is usually a general purpose OS such as Linux. Horizontal decoupling brings

openness to gateways. Third party applications can be deployed on gateway OS via a Host OS, a container (LXC, Docker) or a virtual machine.

It is hard to develop intelligent hardware and the IoT applications due to the hardware resource constraint, the shortage of development platforms, the complex communication protocol, etc. The innovative and

business effective way is to provide an open and customizable platform.



Figure 11-17: Lightweight OS architecture [source: Huawei]

Figure 9: Lightweight OS architecture [source: Huawei]

The core parts of the platform, depicted in Figure 11-17 [39], include:

1. Lightweight kernel. The OS kernel running on system-on-chips in tiny devices will hide the chipset

difference between silicon companies, and this OS provides the drivers and reacts to events happening around the hardware. Considering the resource-constrained applications, the kernel has to have a light footprint, high startup speed, low power consumption and fast response.

2. Customizable services. Provides key services such as connectivity service, sensor manager, security

engine, etc. which support agile development of new applications. The algorithm for each service is flexible and customizable.

3. Open API. The point of the open API is to abstract the chaotic world of system on a chip (SoC) design and complexity of key services (such as connectivity) away from developers – leaving a cleaner, common interface to work with.

Today’ s IoT largely exists in isolation, and it has been impossible to realize a truly interconnected world

where devices are interoperable. The import of the lightweight OS platform to the vertical Edge ecosystem will open a new chapter. Since the platform provides the consistent APIs over the connectivity, security,

application and other such domains, which hides the vendor’ s implementation differences, it is evident that it will fundamentally solve the interoperability issues.

A key architectural goal for edge computing is to have functionality isolation in the form of services which

can be deployed anywhere, e.g. on a smart device, on an IoT gateway, in a micro data centre, inside a telecommunications network, in a company’ s data centre or in a public or private cloud. This isolation can be achieved through the service concept, specifically microservices (which are totally self-contained)

or pods (a Kubernetes concept designating a group of services sharing resources, e.g. a database).

An industry-wide trend is emerging to package edge computing capabilities into microservices and deploy them within containers on IoT Edge Computing Nodes (ECNs), as illustrated in Figure 11-18 [37]. Containers provide security through isolation; they also serve as deployment units that simplify lifecycle management through less interdependency and complexity.



Figure 11-18: Edge computing and Microservices

Figure 10: Edge computing and Microservices

The deployment of services requires a runtime environment that is the same across all possible deployment

locations. There are three major options, as follows: a) Open Service Gateway initiative (OSGi), a standardized runtime environment for Java. A micro-service will be represented by an OSGi bundle, 2) Virtual Machines, as a hardware abstraction. A micro-service will be represented by the service functionality inside its own guest OSs, and c) Containers, as an OS abstraction. It provides all the

dependencies, e.g. runtime libraries and components, of micro-services. The industry trend is favouring containers, as they are more lightweight than virtual machines (less resources consumption and faster start-up) and not tied to a specific programming language like OSGi.

Just as virtualization exists to abstract away areas of memory, processing power, networking control and other computing resources -- we see that micro-virtualization essentially works to abstract applications

away from the hardware they run on. This means that those applications can then be run in isolated

environments -- and this makes a lot of logical sense for security, obviously.

Virtual machines provide a software abstraction of computer hardware. This allows running multiple OS instances on a single piece of hardware. A virtual machine provides the mapping between the software

interfaces and the actual hardware. It also manages the resources among these instances. Initially, virtual machines where used for consolidating servers on a smaller number of hardware systems. With the rise of cloud computing virtual machines played an important role for providing scalability and isolation.

Containers provide virtualization on the OS level. Containers are user space instances executed in the user space of an OS kernel providing strong isolation between them. The OS kernel can provide resources

management between different containers. Unlike virtual machines, containers impose little or no overhead. However, containers are less flexible than virtualization, as only one OS is used, Linux being the most popular one.

Today containers are predominantly used within cloud platforms, providing horizontal scalability, isolation

and resource and lifecycle management. Containers also play an important role in software engineering

processes, as they allow developers to build software in a container which can be deployed in a production environment along with the container itself, simplifying, installing, providing maintenance and enhancing application security.

Containers have become more popular in cloud computing as they have less overhead than virtual

machines. They provide less flexibility, but cloud data centres typically provide standardized hardware running Linux, which removes the need for flexibility.



11.2 Design Decision Rationales

Design decisions can have positive or negative effects on architecture drivers as given in chapter 2. In this appendix chapter, the rationales for design decisions and their alternatives are formulated [40]. Each

decision is described and explained. The pros summarize reasons for making the decision, and the cons & risks summarize reasons against the decision. Assumptions express what was not definitely know when making the decision. Trade-offs describe quality attributes, drivers, and other decisions that are influencing

and competing with this decision. Manifestation links show the manifestation of the design decision in the architecture and their viewpoints.

11.2.1 Main solution structure

Design Decision Main solution structure

Three Tier architecture pattern

Edge, Platform and Enterprise Tier

PdM data is collected and pre-processed in the edge tier

Data management and processing mainly in the platform tier

Applications on the enterprise tier work on data provided by the platform tier

Pros Proven for CPS / IOT and big data solutions

Clear separation of concerns

Allows independent local processing on the edge

Cons & risks Changes in SOI or overall goals requires consistent changes in all three tiers

Overhead and risk in overall lifecycle management

Limited edge computational resources

Alternative Peer-To-Peer structure

Peer-To-Peer architecture pattern

PdM data is collected and pre-processed on the peer

Data management and processing is distributed on the peers

Pros Distribution of workload

Cons & risks Overhead for distributed coordination

Alternative Pipe-And-Filter structure

Pipe-And-Filter architecture pattern

Pros Easy to change and extend by reconfiguration

High reusability of processing functions (filters)

Cons & risks Interactive transformations are difficult

No filter cooperation

Assumptions Trend to move more data management and processing functionality to edge tier

Trade-offs Overhead for changes compared to pipe-and-filter structure

Less coordination effort compared to peer-to-peer structure

Workload on the edge is bound to the resources of edge. In case of overload, it is not possible to move easily workload to platform tier.

Manifestation links Figure 5-1 Overview on MANTIS Reference Architecture



11.2.2 Edge-to-Platform Communication

Design Decision Edge-to-Platform Communication

Edge-gateway-mediated pattern

Pros Support security and privacy for data access

Aggregate data from different sources

Communication protocol from edge to platform is easy to change

Mediator encapsulates the collective behaviour of the edge

Loose coupling by preventing direct communication between CPSs

Keep overall complexity at a low(er) level

Cons & risks Gateway is single point of failure

Gateway node required

Alternative Push-to-Platform Communication

Sensing devices are capable to connect themselves to the platform

Push data extracted from the physical entities to platform

Receive data from the platform services

Pros Low latency

No single point of failure

Cons & risks Platform has to tackle with different communication protocols

Security needs to be implemented on each edge node

Either higher costs or security can only be partially implemented

Assumptions Gateway nodes which are easy to connect to SOI and platform become available

Trade-offs Gateway as mediator is single point of failure


11.2.3 Separate gateway from asset

Design Decision Separate gateway from asset

The gateway can only access data, which is provided by the SOI

Pros Avoid any side effects of the PdM on the SOI overall performance

PdM can be changed without affecting the SOI (impact on safety certification)

Cons & risks Additional costs for gateway

SOI to gateway communication introduces latency

Alternative Gateway on asset

Asset renders gateway functionality

Pros Low latency

Low costs for hardware

Cons & risks Side effects of the PdM on the SOI overall performance

Changing gateway has impact on the SOI (safety certification)

Assumptions None

Trade-offs Separated gateway has higher costs

Separated gateway introduces additional latency in edge tier communication




11.2.4 Platform as a cloud

Design Decision Platform as a cloud

Platform tier is realized as cloud

Private cloud under full control of the OEM or an open cloud

Broker and/or Middleware pattern

Broker is responsible for coordinating all the communications between distributed software components and/or modules

Edge broker is used to receive and distribute data from the edges

Service broker is used to coordinate the data distribution between the different maintenance services

Pros Common environment for services and applications to be provisioned

Minimal management effort

Infrastructure of virtual computing resources

Cons & risks Security and privacy risks

Vulnerability to attack

Limited control and flexibility

Cloud computing platform dependencies (vendor lock-in)

Cloud computing costs

Assumptions Cloud computing becomes widely available at lower costs

Trade-offs None




11.2.5 Maintenance function deployment

Design Decision Maintenance function deployment

Distribution of maintenance functionality across tiers

Data and services are deployed near to the source of the data

Data analysis also happens in the edge

Not all acquired data is necessarily shared on the platform

Local pre-processing on the edge

Only aggregated data for a certain purpose is provided on the platform

Render health assessment or prognostic assessment on the edge gateway

Pros Reduction of communication bandwidth and data usage

Loosen coupling across tiers

Hide CPS specific context information on the edge

Enables off-site and mobile use cases

Enables data protection

Enable real time analyses if cloud based platform does not provide sufficient quality of service

Cons & risks Additional computational resources required on edge

Additional complexity in lifecycle management

Alternative Edge as pure data provider

Maintenance functionality is deployed on platform and enterprise tier

Platform tier collects data from edge tier

Edge is primarily data provider

All acquired data is shared on the platform

Few local pre-processing on the edge

Data provided on the platform is primarily raw data

Render health assessment or prognostic assessment on the platform tier

Pros No computational resources required on edge

Reduced complexity in lifecycle management

Cons & risks High communication bandwidth required

Platform tier is strongly coupled to edge

Risk of losing operation-critical data on the platform tier

Higher latency in communication pathway

Assumptions None

Trade-offs Additional computational resources for lower latency and lower communication bandwidth

Additional complexity in lifecycle management for loosen coupling across tiers




11.2.6 Common functionality groups

Design Decision Common functionality groups

Functional decomposition according to ISO 13374 [9]

Pros Accepted industry standard

Connectional reuse possible

Enables common understanding between stakeholders

Cons & risks No computational resources required on edgeTODOReduced complexity in lifecycle management

Alternative Reusable data processing services

Reuse of data analysis building blocks across different application settings

Pros Save long term costs by reusing software components in well-understood domains

Cons & risks Domain of predictive and proactive maintenance solutions is not well-understood at the current evolution stage

Low commonality as data analysis solutions are tailored to the settings in the SOI

Assumptions None

Trade-offs None


11.2.7 Data processing patterns

Design Decision Data processing patterns

Lambda & Kappa architectural patterns

Pros Process high data volumes

Process high velocity of data

Process high variety of data

Highly scalable, fault-tolerant and predictable data processing

Resilient against data stream imperfections

Algorithmic flexibility: exact algorithm on the batch layer and approximate algorithm on the real-time layer (eventual accuracy)

Data schema migrations are easy

Immutable data in dataset records its own history (self-auditing dataset)

Cons & risks Lambda requires maintaining algorithms in batch and real-time layer

Programming in distributed frameworks is complex

Assumptions None

Trade-offs None




11.2.8 Data types and storage

Design Decision Data types and storage

Process and store data of different type and representation

Support SQL and non-SQL databases

Pros Enable Big Data architectures

Support continuous data streams

Cons & risks High complexity

Alternative Pure SQL storage

Process and store data in structured SQL relational model

Support only SQL databases

Pros Simple data structure in relational model

Accessing data in SQL storage is easy

Strong data integrity protection

Cons & risks Inflexibility in data schema

Expensive for handling high velocity and high volume data

Assumptions None

Trade-offs None


11.2.9 Data storage organization

Design Decision Data storage organization

Organize data storage conceptually aligned to MIMOSA Common Conceptual Object Model (CCOM)

Pros Enable data interoperability with MIMOSA on conceptual semantic level

Cons & risks No guarantee for interoperability on syntactical and structural level

Alternative Relational data storage organization

Organize data storage in an entity-relationship model aligned to Common Relational Information Schema (CRIS)

Pros Enable data interoperability with MIMOSE on semantic, syntactical and structural level

Cons & risks Limits flexibility in data storage organization

Alternative Proprietary data storage organization

Organize data storage conceptually aligned to own business standards (not MIMOSA)

Pros High flexibility in data storage organization

Cons & risks Additional interoperability layers and mappings required

Assumptions None

Trade-offs None




11.2.10 Data access via data management facade

Design Decision Data access via data management facade

Encapsulate the actual storage technology behind a data management facade

Pros Resilient to future changes in data storage technology

High degree of abstraction and agility

Cons & risks Implementation overhead

Performance overhead

Alternative Direct access to database

Functions directly access data on the database

Pros Optimize function implementation on database technology

Cons & risks Changes in data storage technology require extensive changes in function implementation

Assumptions Data storage technology will change multiple times in the future

Trade-offs Accept implementation and performance overhead for reduced change costs in future


11.2.11 Collaborative proactive maintenance

Design Decision Collaborative proactive maintenance

Dedicated collaboration spaces with standardized data storages

MIMOSA-based data storages

Pros Comply with data protection regulations

Reduce communication bandwidth

Fast local decision making through taking advantage of knowledge acquired elsewhere

Cons & risks Complexity in different layers of the decision chain in order to bring the right information to the right place in the right time

Alternative Shared data exchange and processing platform

Shared cloud with data exchange interfaces

Pros Low complexity in implementation

Cons & risks High communication bandwidth required

Non-compliance with data protection regulations

Assumptions Collaboration reduces overall proactive maintenance costs

Trade-offs None




11.2.12 Lifecycle management on gateway

Design Decision Lifecycle management on gateway

Modularisation with dynamic reconfiguration of function implementation

Manage module dependencies

Pros Enable dynamic addition of new sensors or data analysis functions on the edge

Enable dynamic allocation of data processing functions from the platform to the edge or vice versa

Enable dynamic fixing of bugs

Adapt data analysis on the gateway to changing environment during runtime


Dependency on lifecycle management technology like OSGi

Alternative No lifecycle management on gateway

Static reconfiguration of function implementation

Pros Simple implementation

Cons & risks Reboot or shutdown of services, nodes or devices required (downtime)

Assumptions None

Trade-offs None


11.2.13 Lifecycle management on platform

Design Decision Lifecycle management on platform

Modularisation with dynamic reconfiguration of function implementation

Manage module dependencies

Pros Enable dynamic addition of data analysis functions on the platform

Enable dynamic allocation of data processing functions from the edge to the platform or vice versa

Enable dynamic fixing of bugs

Adapt data analysis on the platform to changing environment during runtime


Dependency on lifecycle management technology like OSGi

Alternative No lifecycle management on platform

Static reconfiguration of function implementation

Pros Simple implementation

Cons & risks Reboot or shutdown of nodes and devices required (downtime)

Assumptions None

Trade-offs None




11.3 Architecture Driver Solutions

The architecture driver solutions summarize the architecture decisions that are contributing to the architecture drivers or have impact on drivers.

11.3.1 Main solution

Main

solu

tion

stru

cture

Edge-

to-

Pla

tform

Com

munication

Sep

ara

te

gate

way

fro

m

ass

et

Pla

tform

as

a

cloud

Main

tenan

ce

funct

ion

dep

loym

ent

Com

mon

funct

ionality

gro

ups

MARMKF_002:

Transform, manipulate and aggregate maintenance related data on each tier of the architecture

X X X

MARMKF_003:


X X X

MARMKF_005: Store and back-up maintenance related data on each tier of the architecture

X

MARMKF_007:

Secure maintenance related data on each tier of the architecture

X

MARMKF_008:

Secure maintenance related data on all transmission paths

X

MARMKF_009:

Monitor maintenance related data on each tier of the architecture locally and remotely

X X

MARMKF_010:

Analyse distributed maintenance related data for real time scenarios

X X

MARMKF_011:

Analyse distributed maintenance related data for non-real time scenarios

X

MARMQG_003:

Adaptability :: Use of preferred technologies X

MARMQG_010:

Safety :: Monitored processes and sites are not influenced w.r.t. safety

X



11.3.2 Data Processing and Management

Data

pro

cess

ing

patt

erns

Data

typ

es a

nd

stora

ge

Data

sto

rage

org

anizat

ion

Data

acc

ess

via

data

m

anagem

ent

faca

de

MARMKF_001:

Extract, collect, and measure maintenance related data of system of interest, processes and environment

X

MARMKF_002:


X

MARMKF_004:

Retain and log all maintenance related data X X

MARMKF_005:

Store and back-up maintenance related data on each tier of the architecture X X X

MARMKF_009:

Monitor maintenance related data on each tier of the architecture locally and remotely

X

MARMKF_010:

Analyse distributed maintenance related data for real time scenarios X

MARMKF_011:

Analyse distributed maintenance related data for non-real time scenarios X

MARMKF_012:

Control all kind of access and usage of maintenance related data on each tier of the architecture locally and remotely

X

MARMKF_018:

Check maintenance related data compliance X



11.3.3 Lifecycle Management

Lifec

ycle

m

anagem

ent

on

gate

way

Lifec

ycle

m

anagem

ent

on

pla

tform

MARMQG_001:

Adaptability :: Adaptability for different environments X X

MARMQG_002:

Adaptability :: Adaptability for new components X X

MARMQG_003:

Adaptability :: Use of preferred technologies X X

mantis · mantis d2.9 reference architecture and design specification final version work package...

Documents