extensible information models with dds
Post on 19-Jul-2015
1.393 Views
Preview:
TRANSCRIPT
Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com!
海 賊 天 使
DDS Overview
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Data Distribution in a Nutshell
¨ Data distribution is about making application defined data available where needed and when needed, while preserving and end-to-end type contract
¨ Data is a first class concept, it can be Created, Read, Updated, and eventually Disposed (CRUD)
¨ The last value (or last N-values) of a Data is always available to applications
Id Temp Scale
101 25 C
202 78 F
303 299 K
DW
DW
DW
DR
DR
DR
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
Fully Distributed Global Data Space
enum TScale { ! C, F, K !}; !
DW: DataWriter DR: DataReader
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS Topics ¨ A Topic defines the subject of
publications and subscriptions
¨ A Topic has associated a user defined type and QoS
¨ The Topic name, type and QoS have a well defined role in matching subscriptions
¨ Topics can be discovered or locally defined
Topic
Name
QoS Type
DURABILITY, DEADLINE, PRIORITY, …
[1/2]
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
“com.myco.TTempSensor”
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS Topics
¨ DDS Topic types can have associated keys
¨ Each unique key-value identify a Topic Instance – a specific stream of values
[2/2]
Topic
Name
QoS Type
“com.myco.TTempSensor”
DURABILITY, DEADLINE, PRIORITY, …
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
DDS Type System
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS Type System The DDS Type System is defined by means of:
¨ Abstract Type System ¨ The definition of types supported by DDS
¨ Concrete Syntax(es) to express types ¨ Syntax for specifying DDS Topic Types
¨ Serialization Format ¨ Format for representing topics types as a byte-stream
Limitations in DDS v1.2
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS v1.2 Type System ¨ Abstract Type System
¨ Monomorphic Nominal Type System
¨ Not explicitly defined by the specification
¨ Concrete Syntax(es) to express types ¨ IDL Subset (e.g. struct, enum, union, primitive types)
¨ Serialization Format ¨ CDR (Common Data Representation)
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Limitations ¨ The monomorphic nominal type system makes it
hard to, transparently extend, information models ¨ For a given topic, DataWriters and DataReaders need to
use exactly the same type definition thus limiting independent evolutions of the reading/writing side
¨ The type system is not formally defined
[Abstract Type System]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Example ¨ Suppose that a new Temperature
Sensor was available that produced humidity estimates along with temperature
¨ In DDS v1.2 it would be impossible to deploy new sensors along with old sensors while using the same topic
¨ So what could we do to incrementally update the system?
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; ! !} !
Old Sensor
New Sensor
[1/2]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
TTempSensor
Inconsistent Topic Types ¨ The scenario below would lead, in DDS v1.2, to an Inconsistent Topic Type error
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; !} !
TTempSensor struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
TTempSensor
DW DR
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Example ¨ The solution to the problem is to introduce
and Extension Topic/Type
¨ This extension topic/type shares the same key as the original topic and contains the new attributes
¨ This gets the job done. Yet it adds complexity to the applications that will want to take advantage of the capabilities provided by new sensors. These applications, will now have to deal with two topics, or rely on Multi-Topics
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
struct TempSensorExt { ! @Key long Id; ! float hum; !} !
Old Sensor
Sensor Extension
[2/2]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
struct TempSensorEx { ! @Key long Id; ! float temp; !} !
TTempSensorEx
TTempSensor
Topic Extension ¨ The Legacy DR continues to use the old topic, while new DR are made
aware of the extension topic
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
TTempSensor struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
TTempSensor DR
struct TempSensorEx { ! @Key long Id; ! float temp; !} !
TTempSensorEx
DW DR
TTempSensorEx
Legacy DR
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Limitations ¨ DDS v1.2 specifies IDL as the only concrete syntax
for specifying Topic Types
¨ In addition, no standard/portable way of specifying keys is defined
¨ Not everyone is happy with using IDL. Some users would like to use UML to define topic types, other, XML, etc.
[Concrete Syntax]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Limitations
¨ CDR is a very efficient binary serialization format, its only limitation is the lack of support for extensible types
¨ This stems from the fact that CDR pre-supposes the exact knowledge of types and assumes that the order of type attributes is invariant
[Serialization Format]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Limitations Recap
¨ The main limitations present in the DDS v1.2 type system are the limited support for information model extensibility and evolution
¨ Under the current model, information model evolution or extensions are either non-transparent or require complete update-redeploy cycles
DDS Extensible and Dynamic Topic Types
[DDS-Xtypes]
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS-XTypes Overview The DDS-XTypes Specification defines:
¨ A polymorphic structural type system for DDS topic types – which formalizes extends in several ways the DDS v1.2 type system
¨ A set of standard concrete syntaxes for representing topic types
¨ A set of serialization formats supporting extensible types
¨ A Dynamic API for defining Topic Types and DR/DW operating over these types
DDS-XTypes Type System
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Type System
¨ The DDS-Xtypes type system defines the rules and the type constructors that can be used to create valid DDS Topic Types
¨ The type system also defines a structural sub-type relation that is used to determine assignability of topic types
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Primitive Types ¨ The DDS-XTypes defines the following Primitive Types
¨ The Type mapping defines how DDS Types are mapped to native Programming Language Types (e.g. C99 int32_t, float, etc.)
Primitive Types
Integral Types
Byte Int16 Int32 Int64 Char8 Char16
Floating Point Type
Float32 Float64 Float128
Boolean
Bool
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Constructed Types ¨ The DDS-Xtypes introduces some new constructed types not
originally available in DDS v1.2 (marked in red in the figure below)
Constructed Type
Collection
Map Sequence String Array
BitSet Enumeration Aggregation
Union Structure Annotation
Alias
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Annotations ¨ DDS-Xtypes introduces annotation as a way to
attach meta-data to , and control the semantics of, types and type members
¨ Annotations are aggregated types whose members are restricted to being: ¨ Primitive Types ¨ String
¨ Enumeration
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Annotations in IDL ¨ Some built-in annotations, such as
@ID, @Key, @Optional, etc., are defined by DDS-XTypes specification
¨ User can define their own annotations
¨ Annotation can also be added as //@ after the annotated type member
@Annotation!local interface MyAnnotation { ! attribute long attr1; ! attribute string attr2 default “zzz”; !} !!@MyAnnotation!struct MyStructA { ! long member1; ! string member2; !} !!struct MyStructZ { ! @MyAnnotation (attr1 = 10, ! attr2 = “yyy”)! long memberX; ! string memberY; //@Optional!} !!
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Structure ¨ Structures are the type used to associate user-defined types with
Topics
¨ Structures defined by the DDS-Xtypes Specification support Single Inheritance
¨ Structures members have associated some properties, namely: ¨ ID. To uniquely identify an attribute ¨ Key. To identify whether the member is part of the key ¨ Optionality. To express the fact that an attributed is optional and thus
might not be set ¨ Must Understand. To enforce that a given field is not discarded ¨ Shared. To allow for non-inline data-storage of the attribute
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
DDS-XTypes Structures ¨ Built-in annotations can
be used to control the semantic associated with type members
¨ Annotations, as explained next, are also used to control the kind of extensibility/mutability supported
struct TempSensor { ! @Key @ID(10) ! long Id; ! @MustUnderstand @ID(11) ! float temp; ! @Optional @ID(12) ! float hum; !}; !!!Struct InfraRedTempSensor : TempSensor { ! @Shared @ID(20) ! byte spectrum[1024]; !}; !!
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Type Extensibility & Mutability To better support incremental system updates, the DDS-XTypes Specification defines 3 different kinds of types. The @Extensibility annotation controls the kind attached to a given type
¨ Final ¨ The type is sealed and nothing can be changed w/o breaking compatibility
¨ Extensible (Default) ¨ The Type is monotonically extensible, meaning that new attributes can be added
while preserving the compatibility previous versions of the same type
¨ Mutable ¨ The Type can be mutated by adding, reordering and removing fields. The type
compatibility depends on structural compatibility
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Assignability ¨ The compatibility between different types, is expressed in
terms of assignability
¨ Given two types, T1 and T2, then based on the kind of extensibility supported, T1 is-assignable-from T2 if the type T1 can be “projected/widened” from T2
¨ Example
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; ! !} !
T1 T2
Projection
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Final Types Assignability Rules ¨ Given two final types T1 and T2, we say that “T1 is-
assignable-from T2”, iff: ¨ T1 is-assignable-from T2 iff the two types have exactly the
same members – meaning exactly same names and types as well as @ID and @Key annotations
¨ Example:
@Extensibility(FINAL_EXTENSIBILITY) !struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
@Extensibility(FINAL_EXTENSIBILITY) !struct TSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
T2 T1
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Extensible Types Assignability Rule
¨ Given two extensible types, T1 and T2, we say that T1 is-assignable-from T2 iff: ¨ T2 is a monotonic extension of type T1, e.g., T1 is a “prefix”
of T2
¨ Example: @Extensibility(EXTENSIBLE_EXTENSIBILITY) !struct TSensor { ! @Key long Id; ! float temp; ! TScale scale; !}; !
T1 @Extensibility(EXTENSIBLE_EXTENSIBILITY) !struct TSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; !}; !
T2
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Mutable Types Assignability Rule
¨ Given two mutable types, T1 and T2, we say that T1 is-assignable-from T2 iff: ¨ T1 and T2 share the same keys (e.g. same members
sharing name and ID) ¨ Homonymous members in T1 and T2 share the same ID
and vice-versa ¨ For each member x of T1 (T1.x) and y of T2 (T2.y) sharing a
common ID then T1.x is-assignable-from T2.y ¨ All @MustUnderstand member of T2 also exist in T1
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Mutable Types Example
@Extensibility(MUTABLE_EXTENSIBILITY) !struct HumSensor{ ! @Key @ID(10) long Id; ! @ID(13) float hum; !}; !
T1 @Extensibility(MUTABLE_EXTENSIBILITY) !struct TempSensor { ! @Key @ID(10) long Id; ! @ID(11)float temp; ! @ID(12)TScale scale; ! @ID(13)float hum; !}; !
T2
T1 is-assignable-from T2
@Extensibility(MUTABLE_EXTENSIBILITY) !struct HumSensor{ ! @Key @ID(10) long Id; ! @ID(13) float hum; ! @ID(20) long status; !}; !
T1 @Extensibility(MUTABLE_EXTENSIBILITY) !struct TempSensor { ! @Key @ID(10) long Id; ! @ID(11)float temp; ! @ID(12)TScale scale; ! @ID(13)float hum; !}; !
T2
T1 is-assignable-from T2
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Type Consistency QoS
¨ The DDS-XTypes introduces a the Type Consistency Enforcement Policy to control legal type conversion
¨ This Policy defines the following possibilities: ¨ EXACT_TYPE. As DDS today (but with checks) ¨ EXACT_NAME. What DDSI requires today
¨ DECLARED. Compatible type signatures + assignability ¨ ASSIGNABLE. Assignability is what matters
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Topic Model ¨ The DDS-XTypes
specification extends the topic definition by introducing the concept of a type signature
¨ Type signature are used to constrain the set of types that might be considered assignable
Topic
Name
QoS
Type
Type Signature
TYPE_CONSISTENCY DURABILITY, DEADLINE, PRIORITY, … struct TempSensor { !
@Key long Id; ! float temp; ! TScale scale; !} !
“com.myco.TTempSensor”
“TempSensor, TSensor”
Information Model Evolution Example
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Example – Reloaded ¨ Suppose that a new
Temperature Sensor was available that produced humidity estimates along with temperature
¨ So what could we do to incrementally update the system?
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; ! !} !
Old Sensor
New Sensor
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
TTempSensor
Assignable Topic Types ¨ With DDS-XTypes the system evolution does not introduce any burden
¨ Types are projected through the assignability rules
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; !} !
TTempSensor struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; !} !
TTempSensor
DW
DR
struct TempSensor { ! @Key long Id; ! float temp; ! TScale scale; ! float hum; !} !
TTempSensor
DR
Guidelines
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Picking your “Extensibility” ¨ Final Types should be used to express universal invariant of the system.
Something that is so fundamental it should never-ever change w/o requiring a full system update
¨ Extensible Types are the default in the specification because their semantic is pretty straightforward and safe. Thus, if you don’t want to get into type refactoring you might stick with this default
¨ Mutable Types give you maximum flexibility, however you should be designing your information model around them. When using Mutable Types ensure that you carefully use the @MustUndertand and @Optional annotations to set structural lower bounds on your type as well as nicely dealing with absence of members
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Type Signature ¨ If you design your information model properly, then
you are safe in ignoring the type signature and using the ASSIGNABLE Type Consistency policy
¨ However, when integrating existing systems, the type-signature is key to scope what you might match
Summing Up
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
Concluding Remarks
¨ The DDS-XTypes specification provides a set of powerful extensions to deal with system extensibility and evolution
¨ The Specification is adopted and under finalization
¨ The OpenSplice DDS team is already implementing it!
Copyrig
ht 2010, PrismTech – A
ll Rights Reserved.
http://www.opensplice.com/
http://www.opensplice.org/
emailto:opensplicedds@prismtech.com
http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com
http://bit.ly/1Sreg
http://www.slideshare.net/angelo.corsaro
http://twitter.com/acorsaro/
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
top related