Ope

nSpl

ice

DD

S

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

PrismTechangelo.corsaro@prismtech.com

Interoperable, Extensible and Efficient System Architectures

Ope

nSpl

ice

DD

S

Motivating Examples

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

SESAR: The Single Sky☐ Enable operational interoperability

across ATM/ATC in EU

☐ Allow information to flow seamlessly within the ATC/ATM ecosystem and across Europe

☐ Ensure the system is incrementally extensible and evolvable

☐ Ensure the system makes efficient use of resources such as network

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Smart City / Smart Grid

☐ Make relevant information available in real-time to an open ended number of consumers

☐ Ensure interoperability with third parties, efficient use of resources and extensibility

Ope

nSpl

ice

DD

S

Interoperability

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Defining Interoperability

The ability of two or more systems or components to exchange information and to use the information that has been exchanged.

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Syntactic Interoperability

☐ The ability of two or more systems to communicate and exchange data

☐ Requires specified data formats and communication protocols

☐ Syntactical Interoperability is a necessary condition for higher-level of interoperability

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Semantic Interoperability

☐ The ability to meaningfully and accurately interpret exchanged information

☐ Semantic interoperability requires communication parties to agree on a common information model

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Interoperability in Summary

☐ Interoperability requires agreements at a syntactic and semantic level

☐ Syntactic Interoperability is commonly achieved by agreeing a communication protocol and a data representation standard

☐ Semantic Interoperability is commonly achieved by agreeing on a Common Information Model

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Measuring Interoperability

Based on the previous definitions we can define 3 levels of interoperability:

☐ Level 0: No Interoperability (closed interfaces)

☐ Level 1: Syntactic Interoperability (open protocol/data format)

☐ Level 2: Semantic Interoperability (common information model)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

☐ Suppose that we define legal messages as any word created over alphabet, more formally:☐ ∑ = {a, b, ..., z, ?, !}☐ words := { x ∈ ∑+}☐ sentence = words+

☐ Example:☐ Can Talk and hear w/o problems☐ Don’t necessarily “understand”

Example: Syntactic Interoperability

Che ora?

Sapresti dirmi che ore sono? What?

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Semantic Interoperability

☐ Suppose now that we constrain our grammar to a subset of legal English phrases:

Thanks!

Excuse me, do you know the time?

Sure, it’s 11 o’clock.

Ope

nSpl

ice

DD

S

Extensibility

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Defining Extensibility

Extensibility is a systemic measure of the ability to extend a system and the level of effort required to implement the extension

Extensions can be through the addition of new functionality or the modification of an existing functionality

Modularity and loose coupling foster extensibility

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Forward Compatibility

☐ Forward compatibility (at times considered as part of extensibility) aims at the ability of a system to gracefully accept input intended for a later version of the system

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Measuring Extensibility☐ Extensibility can be measured w.r.t. the cost associated with

adding a new functionality to the system

☐ Where:☐ CDNF: Cost of developing the new feature☐ CINF: Cost of integration for the new feature

☐ For any system: 0 < Extensibility ≤ 1

Extensibility    =  CDNF/(CINF  +  CDNF)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Measuring Forward Compatibility

☐ For simplicity we will assume that forward compatibility is binary, thus either is supported or not

☐ In general, many systems support forward compatibility under certain conditions. Yet for our discussion it is OK to consider it as a binary property.

Ope

nSpl

ice

DD

S

Efficiency

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Defining Efficiency

In general terms, efficiency describes the extent to which a valuable resource, such as time, space, network bandwidth, etc., is well used for the intended task of purpose

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

☐ Efficiency can be measured as follows:

☐ Where:☐ CO: Optimal Cost to perform the given task/operation ☐ CA: Actual Cost to perform the given task/action

☐ For any system: 0 < Efficiency ≤ 1

Measuring Efficiency

Efficiency  =  CO/CA

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Data Encoding☐ As an example of efficiency, let’s consider different data

encoding for a relatively simple Temperature Sensor

☐ Let’s assume a very simple temperature sensor defined as follows:

struct  TempSensor  {      short  temp;      short  hum;};

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Data Encoding

0 23 0 4B

TempSensor  t  =  {35,  75};

{ “ t ebyte 0 1 2 30 1 2 3

m p “ :

3 5 , “

h u m “

: 7 5 }

byte

CDR JSON

4  bytes16  bytes

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Data Encoding☐ Let’s compute now the efficiency. If we ignore variable length

encoding, then the cost of encoding two short integers is 4 bytes, thus CO = 4 bytes

☐ CDR Efficiency. The actual cost of encoding our temperature sensor in CDR was CA = 4 bytes this leads to EfficiencyCDR = 4/4 = 1

☐ JSON Efficiency. The actual cost of encoding our temperature sensor in JSON was CA = 16 bytes this leads to EfficiencyJSON = 4/16 = 0.25

☐ Note: We have taken a few short cuts here since a formally sound derivation of CO should have considered the entropy of the data source. Instead we have silently assumed a binary entropy function.

Ope

nSpl

ice

DD

S

Data Centric Architectures

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data-Centric Architecture (DCA)Abstract DefinitionA Data Centric Architecture consists of:☐ Abstract Data Types: A, B, C, ... ☐ Arrows: f, g, h, ...where arrows satisfy the following properties:

Given f: A → B, g: B → C, and h: C → D

then the following holds true:

∃ g ∘ f: A → C [Composition]

∃ 1A : A → A [Identity Arrow]

h ∘ (g ∘ f) = (h ∘ g) ∘ f [Associativity]

f ∘ 1A = f = 1B ∘ f [Unit]

A Data Centric Architecture is a Category

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data-Centric Architecture (DCA)

The abstract definition of a Data-Centric Architecture formally captures its key properties:

☐ The Abstraction Barrier is defined in terms of Abstract Data Types

☐ The System Behaviour is specified in terms of transformations (functions or arrows) over these Abstract Data Types

Note: From this emerges a natural relationship between DCA and Functional Programming Languages

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Space Traveller☐ Suppose we want to create a

distributed game in which we have (some) UFOs that are trying to destroy our spaceship

☐ UFO can be our allies/enemies and can be played by other players or computers

☐ Some UFOs, might just be space travellers and should not be destroyed

☐ The goal of the game is to destroy the UFOs that have been classified as threats

[1/3]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Space Traveller

Data Types☐ FlyingObject☐ Dynamics☐ Classification☐ Collision

Arrows☐ animate: Dynamics → Dynamics☐ classify: FlyingObject x Dynamics → Classification☐ collide: [FlyingObject x Dynamics] → [Collision]☐ pilot: IO → Dynamics x FlyingObject☐ fire: IO → FlyingObject x Dynamics☐ display: FlyingObject x Dynamics x Classification x Collision

→ IO

☐ The DCA for this game could be described as follows (keeping it simple...)

[2/3]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Space Traveller

pilot

fire

animate classify

collide display

IODynamics

Dynamics DynamicsDynamics

Classification

Dynamics

Classification[Dynamics][Collision]

Collision

IO

FlyingObject

FlyingObject

IODynamics

FlyingObject

FlyingObject

[FlyingObject]

[3/3]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data-Centric vs. Service-CentricData-Centric and Service-Centric architectures, such as SOA, Distributed Objects, etc., differ in several dimensions:

☐ Abstraction Barrier:☐ Data-Centric architectures rely on data abstractions☐ Service-Centric architectures rely on service abstractions

☐ Level of Coupling☐ Data-Centric architectures rely solely on the shared knowledge of abstract data

types (agree on the “what”)☐ Service-Centric architectures rely on the shared knowledge of operational

interfaces along with the knowledge of the service provider and of its existence (agree on “what”, “who”, and “when”)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DCA Benefits☐ Loose Coupling

☐ Anonymicity (e.g. only things to be known are the data types and not who produces/consumes them)

☐ Composability (especially when combined with functional languages)

☐ Extensibility

☐ Ease of Integration

☐ Performance (easier to parallelize)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DCA Interoperability

☐ To enable Interoperability DCA should rely on middleware technology that provides a Level 2 Interoperability

☐ In other terms, middleware technologies that define☐ open/standard communication protocols☐ open/standard data representation format, and ☐ open/standard ways of describing a Common Information Model

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DCA Extensibility☐ Extensibility is enabled by DCA through modularity, composability

and loose coupling

☐ Modularity is tied to the granularity of the “arrows/functions” that defined in the system

☐ Composability follows from the properties (refer to definition) of DCA

☐ Loose coupling follows from the fact that the only “contract” between the different element of a system are the abstract data types and nothing else. Neither topological nor existential informations, e.g. it does not matter if there are consumer for a given type T nor how many of those are there and where they are located

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DCA Forward Compatibility

☐ Forward compatibility depends entirely from the technology used to implement the DCA

☐ However, the presence of a typed Common Information Model can facilitate the implementation of efficient and safe “forward compatibility” mechanism

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Efficiency

☐ The efficiency of a DCA depends on ☐ The definition of the Common Information Model (e.g. its level of

normalization), and☐ The efficiency of the infrastructure used to implement the DCA

Ope

nSpl

ice

DD

S

DDS and DCA

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Core Standard

☐ DDS is a family of standards addressing the communication needs of Data Centric Architectures

☐ The two standards at the core are:☐ DDS v1.2 -- API☐ DDSI v2.1 -- Wire Protocol

Ownership Durability Content Subscription

Minimum Profile

Data Centric Publish Subscriber (DCPS)

DDS Interoperability Wire Protocol - DDSI-RTPS

Application

ApplicationUDP/IP

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Standard EcosystemApplication

DDS RMI

Java-5ISO C++ ScalaANSI C

DDS

DDS RMI

X-Ty

pes

DDSI-RTPS

Secu

rity

Secu

rity

DDSI-RTPS

X-Ty

pes

network

2004

2006

2012 2012

20102012 2010 2012

2006

2004 2010 2010 201x

ApplicationAPI

Wire Protocol

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DCAs with DDS

☐ DDS provides first class support for DCAs

☐ DCAs Abstract Data Types are captured in DDS via Topics

☐ DDS does not provide primitive support for specifying DCAs Arrows. These are defined by the architect using her/his favorite formalism / programming language

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Topics☐ A Topic defines a typed data

stream☐ A Topic has associated a data

type and QoS☐ The Topic name, type and QoS

defines the key functional and non-functional invariants

☐ Topics can be discovered or locally defined

DURABILITY,DEADLINE,PRIORITY,…

“net.gva.VehiclePosition”

struct Position2D { long vid; //@Key long x; long y;};

TopicTypeName

QoS

Ope

nSpl

ice

DD

S

DDS & Interoperability

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS: Level 2 Interoperability

DDSI Wire Interoperability

Data ModelCommon “Language”

+QoS Requirements

Interoperability

Semantic Interoperability

Syntactic Interoperability

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS: Portability

DDS API Standard

Application

Portability

Ope

nSpl

ice

DD

S

DDS: Extensibility & Forward Compatibility

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Extensibility

☐ DDS-based applications feature all the extensibility benefits deriving from DCA

☐ In addition, due to the built-in dynamic discovery, DDS-based Systems are completely decoupled from deployment details

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Forward Compatibility

☐ DDS supports Forward Compatibility, specifically, it allows for both monotonic as well as generalized type extensions

☐ Monotonic type extensions allow old systems to consume new types as far as changes consist of additions to the tail of the type

☐ Generalized type extensions allows attribute reordering and removal

Ope

nSpl

ice

DD

S

Efficiency

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Efficiency☐ The DDS standard was

designed in such a way to maximize time and space efficiency

☐ OpenSplice DDS has been designed to minimize resource usage while maximizing performance and scalability

Latency☐ 15-20 usec Inter-Core Latency

☐ 75 usec over GBps Ethernet

Throughput☐ Up to 10+M msg/sec inter-core

☐ Up to 5M msg/sec inter node

OpenSplice DDS Performance Metrics

Ope

nSpl

ice

DD

S

Back to the Space Traveller

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example: Space Traveller

pilot

fire

animate classify

collide display

IODynamics

Dynamics DynamicsDynamics

Classification

Dynamics

Classification[Dynamics][Collision]

Collision

IO

FlyingObject

FlyingObject

IODynamics

FlyingObject

FlyingObject

[FlyingObject]

[3/3]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Space Traveller Topic Types//@Nestedstruct Bounds { long width; long height;};

struct FlyingObject { long oid; //@Key long kind; Bounds bounds;};

//@Nestedstruct Vector { long x; long y;};

struct Dynamics { long oid; //@Key Vector pos; Vector speed;};

enum Classifier { UNKNOWN, FRIEND, THREAT};

struct Classification { long oid; // @key Classifier kind;};

struct Collision { long oid; long coid;};

Ope

nSpl

ice

DD

S

Real World Interoperability

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

SESAR: The Single Sky

☐ Adopted the DDS standard as the foundation of their DCA

☐ Defined and standardized a domain specific Common Information Model concerning flights and their management

City  Service  -­‐  Architecture

Urbio2ca  M

ESH  network

Urbio&ca  

City  Message  BUS  –  Opensplice  DDS

Exis2ng  system

s

Esper  –  Park  Control

Data  warehouse

APPLICATIONSUser,  City  agent

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

UK Generic Vehicle Architecture

☐ Adopted the DDS standard as the technology at the foundation of next generation military vehicles

☐ Defined a domain specific Common Information Model -- the GVA Information Model

Ope

nSpl

ice

DD

S

Summing Up

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Concluding Remarks

☐ Interoperability, Extensibility and Efficiency is key for an increasing number of applications

☐ Data Centric Architectures (DCAs) provide an architectural framework that facilitates the design of systems that are interoperable, extensible and efficient

☐ DDS is the middleware infrastructure that most naturally supports DCAs

Ope

nSpl

ice

DD

S

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

References

¥Fastest growing JVM Language¥Open Source¥www.scala-lang.org

¥ #1 OMG DDS Implementation¥ Open Source¥ www.opensplice.org

OpenSplice | DDS¥Scala API for OpenSplice DDS¥Open Source¥github.com/kydos/escalier

Escalier

¥Simple C++ API for DDS¥Open Source¥github.com/kydos/simd-cxx

¥DDS-PSM-Java for OpenSplice DDS¥Open Source¥github.com/kydos/simd-java

¥ DDS-based Advanced Distributed Algorithms Toolkit

¥Open Source¥github.com/kydos/dada

Ope

nSpl

ice

DD

S ¥@prismtech

¥@acorsaro

¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro

¥ crc@prismtech.com¥sales@prismtech.com

¥opensplice.com ¥forums.opensplice.org

¥opensplice.org ¥opensplicedds@prismtech.com

:: Connect with Us ::