i need the data and i need it 1ms ago!
DESCRIPTION
This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as its programming.TRANSCRIPT
![Page 1: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/1.jpg)
Angelo Corsaro, Ph.D.Chief Technology Officer
OMG DDS SIG [email protected]
Delivering Performance, Openness, and Freedom
OpenSplice DDS
I Need Market Data and I need it 1ms Ago!
![Page 2: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/2.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Challenges
![Page 3: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/3.jpg)
© 2009, PrismTech. All Rights Reserved
Financial FirmsTrends
‣ Direct connection to exchanges is increasingly chosen over data aggregators to reduce trading latency
‣ Market data is growing steadily
‣ Milliseconds speedup over competitors can be worth millions of dollars
Challenges
‣ Process several hundreds-of-thousands of messages, quickly approaching millions messages, per second
‣ Minimize latency and jitter, while maintaining the system stability and reliability
Deployed middleware solutions were not designed to address these challenges!
![Page 4: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/4.jpg)
© 2009, PrismTech. All Rights Reserved
Exchanges
Trends
‣ More and more firms connecting directly
‣ Increasing number of sources providing the same quotes
‣ Competitive advantage obtained by reducing latency on data feeds
Challenges
‣ Ensuring fairness of data distribution
‣ Minimizing latency
‣ Scaling with the number of destinations
Deployed middleware solutions were not designed to address these challenges!
![Page 5: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/5.jpg)
© 2009, PrismTech. All Rights Reserved
OPRA PredictionsTrends
‣ One order of magnitude growth on market data from 2005 to 2008
‣ OPRA message rates predicted to reach ~2M msgs/sec by early 2009
‣ Traffic bursts predicted to further increase by 20%
Challenges
‣ Can your messaging infrastructure keep-up with this trend?
0
500000
1000000
1500000
2000000
88000
190000
359000
701000
1807000
Jan 13, 2005
May 16, 2006
Jan 9, 2007Jan 8, 2008
Jan 6, 2009
Msgs/sec
OPRA Market Data Rates Forecasts
![Page 6: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/6.jpg)
© 2009, PrismTech. All Rights Reserved
Trading with Pareto‣ Usually 20% of the trades
provides 80% of the gains
‣ Under overload condition how can you ensure you keep trading at least 20% of your target?
‣ How can you ensure that you keep trading the the right 20% of data?
Vilfredo Pareto, 1848-1923
Average
Time
Msgs/sec
System Message Rate
Incoming Message Rate
![Page 7: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/7.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
The Data Distribution Service for Real-Time Systems (DDS)
![Page 8: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/8.jpg)
© 2009, PrismTech. All Rights Reserved
Addressing Data Distribution Challenges
The OMG DDS Standard‣ Introduced in 2004 to address the Data
Distribution challenges faced by a wide class of Defense and Aerospace Applications
‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra-large-scale deployments
‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc.
DDS is standard designed to address the data-distribution challenges across a wide class of Defense and Aerospace Applications
![Page 9: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/9.jpg)
© 2009, PrismTech. All Rights Reserved
The OMG Data Distribution Service (DDS)
DDS v1.2 API Standard‣ Language Independent, OS and HW architecture
independent
‣ DCPS. Standard API for Data-Centric, Topic-Based, Real-Time Publish/Subscribe
‣ DLRL. Standard API for creating Object Views out of collection of Topics
DDSI/RTPS v2.1 Wire Protocol Standard‣ Standard wire protocol allowing interoperability
between different implementations of the DDS standard
‣ Interoperability demonstrated among key DDS vendors in March 2009
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
![Page 10: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/10.jpg)
© 2009, PrismTech. All Rights Reserved
Scope of Standardization
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
DD
SI
v2
.1D
DS
v1
.2
Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP
Advanced Message Queuing Protocol
AM
QP
v0
.10
API (?)
Application
AMQP OMG DDS
![Page 11: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/11.jpg)
© 2009, PrismTech. All Rights Reserved
Scope of Standardization
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
DD
SI
v2
.1D
DS
v1
.2
Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP
Advanced Message Queuing Protocol
AM
QP
v0
.10
API (?)
Application
Wire Protocol Standard
AMQP OMG DDS
![Page 12: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/12.jpg)
© 2009, PrismTech. All Rights Reserved
Scope of Standardization
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
DD
SI
v2
.1D
DS
v1
.2
Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP
Advanced Message Queuing Protocol
AM
QP
v0
.10
API (?)
Application
Wire Protocol Standard
Pub/Sub Standard API
AMQP OMG DDS
![Page 13: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/13.jpg)
© 2009, PrismTech. All Rights Reserved
Scope of Standardization
Object/Relational Mapping
Ownership DurabilityContent
Subscription
Minimum Profile
Data Centric Publish/Subscribe (DCPS)
Data Local Reconstruction Layer (DLRL)
DDS Interoperability Wire Protocol
Application
UDP/IP
Real-Time Publish/Subscribe Protocol
DD
SI
v2
.1D
DS
v1
.2
Interoperability Wire Protocol
(TCP, SCTP, UDP)/IP
Advanced Message Queuing Protocol
AM
QP
v0
.10
API (?)
Application
Wire Protocol Standard
Pub/Sub Standard API
Object Relational Mapping Standard API
AMQP OMG DDS
![Page 14: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/14.jpg)
© 2009, PrismTech. All Rights Reserved
Architectural Style
Publisher
SubscriberPublisher
Publisher
Subscriber
Subscriber
Broker
OMG DDS
Publisher
SubscriberPublisher
Publisher
Subscriber
Subscriber
Broker
Broker
Broker
Broker
Broker
AMQP
![Page 15: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/15.jpg)
© 2009, PrismTech. All Rights Reserved
Key Differences in Summary
‣ Peer-to-Peer Communication Architecture
‣ Wire Protocol + API Standard‣ Interoperability across DDS implementations
‣ Application Portability across DDS Implementation
‣ Content-Based Filtering/Quering
‣ Designed bottom-up for maximum performance and determinism
DDS‣ Brokered Communication
Architecture
AMQP
‣ Wire Protocol Standard‣ Interoperability across AMQP
implementations
‣ Header Based Routing
‣ Not geared for High-Performance
![Page 16: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/16.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
DDS and Open Source
![Page 17: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/17.jpg)
Enterprise Ed.
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice DDS
‣ Product reorganized into Editions‣ Community Edition
‣ Compact Edition
‣ Professional Edition
‣ Enterprise Edition
‣ Product Editions provide a a growing set of functionality to address the needs of increasingly more sophisticated users
‣ The Community Edition is Open Source
‣ Compact, Enterprise and Professional Edition are available only through Commercial Subscriptions
Enterprise Ed.
Community Ed.
Compact Ed.
Professional Ed.
![Page 18: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/18.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Community EditionFeatures
‣ OMG DDS v1.2 DCPS‣Minimum Profile
‣ Content Subscription Profile
‣ Durability Profile
‣ Ownership Profile
‣ Networking‣ DDSI v2.1 Implementation
‣ Real-Time Networking Implementation
Ownership DurabilityContent
Subscription
Minimum Profile
Real-Time Pub/Sub (DCPS)
Networking Technology
Application
UDP/IP
Interoperable Wire Protocol (DDSI)
DD
SI v
2.1
DD
S v
1.2
Real-Time Networking
Open Source LGPL Licensed Edition -- The Best Way to Get Started with DDS
Licensing
‣ LGPLv3
![Page 19: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/19.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Performance on Commodity HW
!"#$%&#'
()(!%(*'
&")&%+)'
&!(,%!)'
),,%$#'
!,*%))'(("%!"'
&&)%*$' $"%"$' ,+%($',($%!+'
,*+%&)'
$&"%,,'
*!#%,&'
)$,%$*'
)"#%*)'
"!+%++'
"*(%)+' ")(%&!' ""&%,&'
,++%++'
!++%++'
$++%++'
#++%++'
*++%++'
)++%++'
"++%++'
&+++%++'
&&++%++'
(+%++'
&+(+%++'
(+(+%++'
,+(+%++'
!+(+%++'
$+(+%++'
#+(+%++'
)' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#'
!"#$%
&'$($)$*+%
!*$$,(*%-./*%
-./0/1/23' 456/'
Test Scenario
‣ Single Threaded Application (multi-threaded networking service)
‣ 8192 bit message batches
Throughput Latency
Inter-Node Latency
‣ 60 usec
Inter-Core Read-Latency
‣ 2 usec
Inter-Core Latency
‣ <10 usec
HW:
‣ Dell blade-server
‣ Dual-core, Dual-CPU, AMD Opteron 2.4 GhzOS
‣ Linux 2.6.21-1.3194.fc7Network
‣ Gigabit Ethernet cards
‣ Dell PowerConnect 5324 switch
![Page 20: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/20.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice DDS AdoptionDefense
‣ Combat Management Systems
‣ Tactical Systems
‣ Drones (land/air)
‣ Communication Systems
‣ Radar Processing
‣ Submarine Systems
‣ Future Combat Systems
SCADA/Utilities
‣ Industrial Automation
‣ Power Grids (Smart Grids)
‣ Complex Telemetry (e.g. NASA Rocket Launch System)
![Page 21: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/21.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
OpenSplice DDS Adoption
Transportation
‣ Air Traffic Management/Control
‣ Metropolitan Traffic Management
‣ Underground Metropolitan Transportation
Financial Services
‣ Automated Trading
‣ Risk Management
‣ Option Trading
![Page 22: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/22.jpg)
© 2009, PrismTech. All Rights Reserved
Some OpenSplice DDS Users
!"#$%&'()('*+'##,-+.+/($$,012"0
!"(-/'0&3+4"25&0+-2",6+"78+29:;<=7>?
@A6BC+/DEF=7>>;>
4"25&0
'0,G+HII+JCKA+B@+CA+LI+L6
4DM+G+HII+JCKA+B@+CA+LI+LL
10NG!""#$%%&&&'#()*+",-!'. .-/+. .
Marcoussis, le 28/08/09
Expertise pour la RATP
réalisée par Bernard MAUDRY les 29 et 29 juillet 2009
La RATP va utiliser OpenSplice/DDS pour remplacer son système AGS de gestion des équipements de
station.
Dans ce cadre, elle a demandé à PrismTech d'effectuer une expertise de son ébauche de conception du
nouveau système et de fournir des informations complémentaires et des conseils afin d'exploiter au
mieux les possibilités d'OpenSplice/DDS.
La première journée a été consacrée à la présentation de la conception préliminaire, à sa correction et
aux suggestions concernant son évolution. La conception initiale a été fortement impactée et les
modifications nécessaires impliqueront probablement une nouvelle revue afin de valider les derniers
détails.
La deuxième journée a principalement été dédiée à la réponse aux questions de comportement
d'OpenSplice/DDS dans les situations les plus délicates que rencontrera le nouveau système, en tenant
compte de l'architecture choisie.
Cette architecture comprend les éléments suivants:
! le système est structuré en «lignes» comprenant un centre de contrôle de ligne (niveau 2) et des
stations (niveau 1). Une ligne de métro ou de RER peut être décomposée en plusieurs «lignes»,
auquel cas un des centre de contrôle peut être amené à supplanter un ou plusieurs des autres de
cette ligne.
! Des centres de niveau 3 peuvent intervenir sur n'importe quelle ligne
? Le centre de maintenance (Toscane)
? Le centre de gestion de la ventilation
? Des postes d'intervention (action à partir de n'importe quel endroit du réseau)
! Une adresse IP classe A est associée à chaque ligne
! Une station comprend une noeud de gestion (redondé) chargé de la gestion des équipements de
la station et un poste opérateur
0123.14..-5#) "56 .7,.89.:;;<=>[email protected](C.D.EFG.H=H.9GG.>>>E8.I.+5)*/J .+K(, . $ [email protected]<.2ARM0OA2AQ.MS.ASR31SQ.$!ST.G==9E=H'. .2ARM0OA2AQ.TUUM@A.$ [email protected]<.HO P.1WASVA.DV0MSA00.L12X< .OA1N.W133AY<.R1OA0PA1Q< .SA::.>SR
![Page 23: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/23.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Understanding DDS
![Page 24: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/24.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
Global Data Space
DDS
Brokers
![Page 25: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/25.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
Global Data Space
DDS
Brokers
![Page 26: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/26.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
Global Data Space
DDS
Subscriber
Subscriber
Subscriber
Publisher
Publisher
Publisher
Brokers
![Page 27: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/27.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
‣ Publishers and Subscribers express their intent to produce/consume specific type of data, e.g., Topics
Global Data Space
DDS
Subscriber
Subscriber
Subscriber
Publisher
Publisher
Publisher
Brokers
![Page 28: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/28.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
‣ Publishers and Subscribers express their intent to produce/consume specific type of data, e.g., Topics
Global Data Space
DDS
Subscriber
Subscriber
Subscriber
Publisher
Publisher
Publisher
Brokers
![Page 29: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/29.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
‣ Publishers and Subscribers express their intent to produce/consume specific type of data, e.g., Topics
‣ Data flows from Publisher to Subscribers
Global Data Space
DDS
Subscriber
Subscriber
Subscriber
Publisher
Publisher
Publisher
Brokers
![Page 30: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/30.jpg)
© 2009, PrismTech. All Rights Reserved
As Simple as it Gets‣ DDS is based around the concept
of a fully distributed Global Data Space (GDS)
‣ Publishers and Subscribers can join and leave the GDS at any time
‣ Publishers and Subscribers express their intent to produce/consume specific type of data, e.g., Topics
‣ Data flows from Publisher to Subscribers
Global Data Space
DDS
Subscriber
Subscriber
Subscriber
Publisher
Publisher
Publisher
Brokers
![Page 31: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/31.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Defining Data
![Page 32: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/32.jpg)
© 2009, PrismTech. All Rights Reserved
DDS Topics
Topic
‣ Unit of information exchanged between Publisher and Subscribers.
‣ An association between a unique name, a type and a QoS setting
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
{ShapeType}
{Circle, Square, Triangle}
{...}
Topic Type.
‣ Type describing the data associated with one or more Topics
‣ A Topic type can have a key represented by an arbitrary number of attributes
‣ Expressed in IDL (or XML)
![Page 33: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/33.jpg)
© 2009, PrismTech. All Rights Reserved
DDS Topics
Circle
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
Topic QoS
![Page 34: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/34.jpg)
© 2009, PrismTech. All Rights Reserved
DDS Topics
Triangle
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
Topic QoS
![Page 35: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/35.jpg)
© 2009, PrismTech. All Rights Reserved
DDS Topic Instances and Samples
Topic Instances
‣ Each key value identifies a unique Topic Instance,
‣ Topic’s instance lifetime can be explicitly managed in DDS
Topic Samples
‣ The values assumed by a Topic Instance over time are referred as Instance Sample
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
![Page 36: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/36.jpg)
© 2009, PrismTech. All Rights Reserved
Topic/Instances/Samples Recap.Topics Instances
Samples
timeti tj tnow
![Page 37: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/37.jpg)
© 2009, PrismTech. All Rights Reserved
Content Filtering
‣DDS allows to specify content-filtered Topics for which a subset of SQL92 is used to express the filter condition
‣Content filters can be applied on the entire content of the Topic Type
‣Content filters are applied by DDS each time a new sample is produced/delivered
X0 X1
Y0
Y1
X0 <= X <= X1
Y0 <= Y <= Y1
![Page 38: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/38.jpg)
© 2009, PrismTech. All Rights Reserved
Local Queries‣ Subscribed Topics can be seen locally as “Tables”
‣ A subset of SQL92 can be used for performing queries on multiple topics as well as natural joins
Circle Topic
SELECT * FROM ShapeType s WHERE s.x > 25 AND s.y < 55
color x y shapesizeyellow 30 25 50
color x y shapesizeredblue
yellow
57 62 5090 85 5030 25 50
‣ Queries are performed under user control and provide a result that depends on the current snapshot of the system, e.g., samples currently available
X0
Y0Y0
Y1
Y0 <= Y <= Y1
![Page 39: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/39.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Anatomy of a DDS App.
![Page 40: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/40.jpg)
© 2009, PrismTech. All Rights Reserved
Anatomy of a DDS Application
1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
Instances1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
InstancesSamples
Topic (e.g. Circle)
InstancesDataReaderDataReaderDataReaderDataReaderDataReaderDataReaderDataWriterDataWriterDataWriterDataWriter
blue 21 62 80
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
blue 23 66 80 blue 27 70 80
orange 21 62 80
green 97 75 50 green 94 71 50
orange 21 62 80
![Page 41: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/41.jpg)
© 2009, PrismTech. All Rights Reserved
Anatomy of a DDS Application
Arrows show structural relationships, not data-flows Partition
1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
Instances1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
InstancesSamples
Topic (e.g. Circle)
InstancesDataReaderDataReaderDataReaderDataReaderDataReaderDataReaderDataWriterDataWriterDataWriterDataWriter
blue 21 62 80
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
blue 23 66 80 blue 27 70 80
orange 21 62 80
green 97 75 50 green 94 71 50
orange 21 62 80
![Page 42: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/42.jpg)
© 2009, PrismTech. All Rights Reserved
Anatomy of a DDS Application
Partition
Domain
Domain Participant
Arrows show structural relationships, not data-flows Partition
1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
Instances1 21 62
2 20 61
3 25 70 25
1 22 62 1 23 63
2 19 60
3 25 71 3 25 74 3 26 77
Samples
Topic
InstancesSamples
Topic (e.g. Circle)
InstancesDataReaderDataReaderDataReaderDataReaderDataReaderDataReaderDataWriterDataWriterDataWriterDataWriter
blue 21 62 80
struct ShapeType { long x; long y; long shapesize; string color;};#pragma keylist ShapeType color
blue 23 66 80 blue 27 70 80
orange 21 62 80
green 97 75 50 green 94 71 50
orange 21 62 80
PublisherSubscriber
![Page 43: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/43.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
From Shapes to Market Data
![Page 44: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/44.jpg)
© 2009, PrismTech. All Rights Reserved
The Market Data TopicType Declaration
symbol name exchange quoteAAPLGOOGMSFT
Apple Inc. NASD 165.37Google Inc. NASD 663.97
Microsoft Corp. NASD 33.73
struct StockQuote { char<4> symbol;
char<4> exchange; char<64> name; float quote;
};#pragma keylist StockQuote symbol
Topic Instantiation
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create Content Filtered Topicdds::ContentFilteredTopic cfTopic = topic.create_content_filtered(expr, params);
![Page 45: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/45.jpg)
© 2009, PrismTech. All Rights Reserved
Filtering Market Data
‣ DDS continuous queries can be used for filtering the stream of data produced for a specific stock symbol.
0
10
20
30
40
Microsoft
MSFT
154.0
156.5
159.0
161.5
164.0
Untitled 1 Untitled 20Untitled 39Untitled 58Untitled 77Untitled 96
Apple
AAPL
MSFT > $35
AAPL < $157 | AAPL > $162
![Page 46: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/46.jpg)
© 2009, PrismTech. All Rights Reserved
Writing Market DataCreating a Writer
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qosdds::Writer<StockQuote> writer(topic);
![Page 47: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/47.jpg)
© 2009, PrismTech. All Rights Reserved
Writing Market DataCreating a Writer
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qosdds::Writer<StockQuote> writer(topic);
Writing Data
StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.”};
writer.write(quote);
![Page 48: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/48.jpg)
© 2009, PrismTech. All Rights Reserved
Writing Market DataCreating a Writer
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Writer with Default Qosdds::Writer<StockQuote> writer(topic);
Writing Data
StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.”};
writer.write(quote);
Fastest Way of Writing Data
StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.”};
dds::InstanceWriter iwriter = writer.register_instance(quote);iwriter.write(quote);
![Page 49: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/49.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Read vs. Takestruct Counter { int cID; int count;};#pragma keylist Counter cID
Topic Used in next section.
![Page 50: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/50.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading Samples
‣ Read iterates over the available sample instances
‣ Samples are not removed from the local cache as result of a read
‣ Read samples can be read again, by accessing the cache with the proper options (more later)
DataReader
1 1
2 1
3 1
2 2
Samples Read Samples not Read
1 2 1 3 1 4
2 3
3 2 3 3 3 4 3 5
![Page 51: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/51.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading Samples
‣ Read iterates over the available sample instances
‣ Samples are not removed from the local cache as result of a read
‣ Read samples can be read again, by accessing the cache with the proper options (more later)
Samples Read Samples not Read
DataReader
1 1
2 1
3 1
2 2
1 2 1 3 1 4
2 3
3 2 3 3 3 4 3 5
![Page 52: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/52.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading Samples
‣ Read iterates over the available sample instances
‣ Samples are not removed from the local cache as result of a read
‣ Read samples can be read again, by accessing the cache with the proper options (more later)
Samples Read Samples not Read
DataReader
1 1
2 1
3 1
2 2
1 2 1 3 1 4
2 3
3 2 3 3 3 4 3 5
![Page 53: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/53.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Taking Samples
‣ Take iterates over the available sample instances
‣ Taken Samples are removed from the local cache as result of a take
‣DataReader
1 1
2 1
3 1
2 2
Samples not Taken
1 2 1 3 1 4
2 3
3 2 3 3 3 4 3 5
![Page 54: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/54.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Taking Samples
‣ Take iterates over the available sample instances
‣ Taken Samples are removed from the local cache as result of a take
Samples not Taken
DataReader 2 2
1 2 1 3 1 4
2 3
3 2 3 3 3 4 3 5
![Page 55: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/55.jpg)
DataReader Cache
Topic
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Taking Samples
‣ Take iterates over the available sample instances
‣ Taken Samples are removed from the local cache as result of a take
Samples not Taken
DataReader
1 3 1 4
2 3
3 3 3 4 3 5
![Page 56: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/56.jpg)
© 2009, PrismTech. All Rights Reserved
Reading DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
![Page 57: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/57.jpg)
© 2009, PrismTech. All Rights Reserved
Reading DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
Reading Data
StockQuoteSeq data;dds::SampleInfoSeq info;
reader.read(data, info);
![Page 58: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/58.jpg)
© 2009, PrismTech. All Rights Reserved
Reading DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
Reading Data
StockQuoteSeq data;dds::SampleInfoSeq info;
reader.read(data, info);
Reading a Specific Instance
StockQuote quote;quote.symbol = “APPL”;StockQuoteSeq data;dds::SampleInfoSeq info;
dds::InstanceReader ireader = reader.instance(quote);ireader.read(data, info);
![Page 59: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/59.jpg)
© 2009, PrismTech. All Rights Reserved
Taking DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
![Page 60: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/60.jpg)
© 2009, PrismTech. All Rights Reserved
Taking DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
Taking Data
StockQuoteSeq data;dds::SampleInfoSeq info;
reader.take(data, info);
![Page 61: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/61.jpg)
© 2009, PrismTech. All Rights Reserved
Taking DataCreating a Reader
// Create Topic with Default QoSdds::Topic<StockQuote> topic(“stock_quote”);
// Create a Reader with Default Qosdds::Reader<StockQuote> reader(topic);
Taking Data
StockQuoteSeq data;dds::SampleInfoSeq info;
reader.take(data, info);
Taking a Specific Instance
StockQuote quote;quote.symbol = “APPL”;StockQuoteSeq data;dds::SampleInfoSeq info;
dds::InstanceReader ireader = reader.instance(quote);ireader.take(data, info);
![Page 62: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/62.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Sample, Instance and View States
‣ Along with data samples, DataReaders are provided with state information allowing to detect relevant transitions in the life-cycle of data as well as data writers
‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this DataReader or not.
‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). Determines wether (1) writer exist for the specific instance, or (2) no matched writers are currently available, or (3) the instance has been disposed
‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance
Dat
aRea
der
History Depth = 2
DataReader Cache
Samples
1 1
2 2
3 1
1 2
2 3
SampleInfo
1
2
3
1
2
Topic
![Page 63: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/63.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Would you let me know...
DDS provides three main mechanism for exchanging information with the application
‣Polling. The application polls from time to time for new data or status changes. The interval might depend on the kind of applications as well as data
‣WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened.
‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or
![Page 64: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/64.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Waitsets// Functor handling new data arrivalstruct MarketDataHandler { void operator()(MarketDataReader& reader);};
1
![Page 65: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/65.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Waitsets// Functor handling new data arrivalstruct MarketDataHandler { void operator()(MarketDataReader& reader);};
1
// Create & Register ConditionMarketDataHandler functor;dds::ReadCondition cond = reader.create_read_condition(functor);dds::WaitSet ws;ws.attach(cond);
2
![Page 66: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/66.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Waitsets// Functor handling new data arrivalstruct MarketDataHandler { void operator()(MarketDataReader& reader);};
1
// Create & Register ConditionMarketDataHandler functor;dds::ReadCondition cond = reader.create_read_condition(functor);dds::WaitSet ws;ws.attach(cond);
2
// Wait & Handledds::ConditionVector cvec = ws.wait();for (int i = 0; i < cvec.size(); ++i) cvec[i].execute();
3’
![Page 67: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/67.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Waitsets// Functor handling new data arrivalstruct MarketDataHandler { void operator()(MarketDataReader& reader);};
1
// Create & Register ConditionMarketDataHandler functor;dds::ReadCondition cond = reader.create_read_condition(functor);dds::WaitSet ws;ws.attach(cond);
2
// Wait & Handledds::ConditionVector cvec = ws.wait();for (int i = 0; i < cvec.size(); ++i) cvec[i].execute();
3’ // Dispatchws.dispatch();
3”
![Page 68: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/68.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Listeners
// Functor handling new data arrivalclass MarketDataListener {public: void handle_data(MarketDataReader& reader);};
1
![Page 69: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/69.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Reading with Listeners
// Functor handling new data arrivalclass MarketDataListener {public: void handle_data(MarketDataReader& reader);};
1
// Functor handling new data arrivalMarketDataListener mdl;dds::connection cnt = reader.on_data_available_signal_connect( std::bind(&MarketDataListener::handle_data, &mdl, _1));
2
![Page 70: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/70.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Quality of Service
![Page 71: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/71.jpg)
© 2009, PrismTech. All Rights Reserved
QoS Model
‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, such as:‣ Temporal Properties
‣ Priority
‣ Durability
‣ Availability
‣ ...
‣ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement
‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability
Publisher
DataWriter
Topic
Type
QoS
Name
writes
QoS
DataWriter
Topic
Typewrites
Subscriber
DataReaderreads
DataReaderreads
...
QoS
Name
QoS
QoS QoS
QoS matching
......
QoS QoS
Type Matching
DomainParticipant DomainParticipant
QoS QoS
![Page 72: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/72.jpg)
© 2009, PrismTech. All Rights Reserved
QoS Policies
‣ Rich set of QoS allow to configure several different aspects of data availability, delivery and timeliness
‣ QoS can be used to control and optimize network as well as computing resource
QoS Policy Applicability RxO Modifiable
DURABILITY
DURABILITY SERVICE
LIFESPAN
HISTORY
PRESENTATION
RELIABILITY
PARTITION
DESTINATION ORDER
OWNERSHIP
OWNERSHIP STRENGTH
DEADLINE
LATENCY BUDGET
TRANSPORT PRIORITY
TIME BASED FILTER
RESOURCE LIMITS
USER_DATA
TOPIC_DATA
GROUP_DATA
T, DR, DW Y N Data AvailabilityT, DW N N
Data Availability
T, DW - Y
Data Availability
T, DR, DW N N
Data Availability
P, S Y N Data DeliveryT, DR, DW Y N
Data Delivery
P, S N Y
Data Delivery
T, DR, DW Y N
Data Delivery
T, DR, DW Y N
Data Delivery
DW - Y
Data Delivery
T, DR, DW Y Y Data TimelinessT, DR, DW Y Y
Data Timeliness
T, DW - Y
Data Timeliness
DR - Y Resources
T, DR, DW N N
Resources
DP, DR, DW N Y ConfigurationT N Y
Configuration
P, S N Y
Configuration
Publisher
DataWriter
Topic
Type
QoS
Name
writes
QoS
DataWriter
Topic
Typewrites
Subscriber
DataReaderreads
DataReaderreads
...
QoS
Name
QoS
QoS QoS
QoS matching
......
QoS QoS
Type Matching
DomainParticipant DomainParticipant
QoS QoS
![Page 73: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/73.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Mapping QoS
Data Latency
LatencyBudget TransportPriority
Throughput
DeadlineTimeBasedFilter
Control over Latency/Throughput tradeoff Control over data latency Control over data priority
Which properties does QoS controls?
![Page 74: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/74.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Mapping QoS
Data Latency
LatencyBudget TransportPriority
Throughput
DeadlineTimeBasedFilter
Control over Latency/Throughput tradeoff Control over data latency Control over data priority
Data Availability
OwnershipOwnershipStrength
Durability
History
Lifespan
Control over data queueing Control over data persistency Control over data sources hot-swap
Which properties does QoS controls?
![Page 75: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/75.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Mapping QoS
Data Latency
LatencyBudget TransportPriority
Throughput
DeadlineTimeBasedFilter
Control over Latency/Throughput tradeoff Control over data latency Control over data priority
Data Availability
OwnershipOwnershipStrength
Durability
History
Lifespan
Control over data queueing Control over data persistency Control over data sources hot-swap
Data Delivery
Reliability
DestinationOrder
Presentation
Control over data distribution reliability Control over data ordering Control over presentation
OpenSplice DDS provides programmatic QoS-driven support for configuring the most important properties of data distribution!
Which properties does QoS controls?
![Page 76: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/76.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Playing with QoS
![Page 77: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/77.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
History QoS
‣ The History QoS Controls the number of samples-per-instance that will be stored by the middleware on behalf of a Reader
‣ Keep Last K. The History QoS can be set so to always have the latest K samples
‣ Keep All. The History QoS can be set so keep all samples produced by the writer and not yet taken, until resource limits are not reached
DataReader
History Depth = 5
Topic
1 1
2 1
3 1
1 2 1 3 1 4 1 5
2 2 2 3 2 4 2 5
3 2 3 3 3 4 3 5
// Create Reader QoSdds::DataReaderQos drqos;drqos.set_keep_last(N);// Create Readerdds::DataReader<MarketData> reader(topic, drqos);
// Create Reader QoSdds::DataReaderQos drqos;drqos.set_keep_all();// Create Readerdds::DataReader<MarketData> reader(topic, drqos);
1
2
![Page 78: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/78.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Transport Priority QoS
// Create Writer QoSdds::DataWriterQos dwqos;dwqos.set_priority(MID_PRIO);// Create Writerdds::DataWriter<MarketData> writer(topic, dwqos);// Write Datawriter.write(data);// Change Prioritydwqos.set_priority(HIGH_PRIO);writer.writer(data)
1
Make sure that important data gets the right priority
Traffic Shaping
OpenSplice DDSBinding
Networking
Shared
Memory
OpenSplice DDSBinding
Networking
Shared
Memory
Network Channels
Priority Bands
Pre-emptive Network Scheduler
Priority Scheduler
Data Urgency Traffic Pacing
Single Copy per Node
Pack Across Topics/Applications
Optimal Unmarshaling
![Page 79: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/79.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Ownership QoSAvailability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
R1
R2
R3
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
W1
W2’’
STRENGTH=3
STRENGTH=1
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuotesymbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
W1’
STRENGTH=2
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuotesymbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuotesymbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
![Page 80: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/80.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Ownership QoSAvailability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
R1
R2
R3
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
W1
W2’’
STRENGTH=3
STRENGTH=1
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
W1’
STRENGTH=2
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
![Page 81: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/81.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Ownership QoSAvailability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
R1
R2
R3
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
W1
W2’’
STRENGTH=3
STRENGTH=1
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
W1’
STRENGTH=2
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
![Page 82: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/82.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Ownership QoS Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
R1
R2
R3W2’’
STRENGTH=1
![Page 83: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/83.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Ownership QoS Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer
‣ Writer strength is used to coordinate replicated writers
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97
StockQuote
symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37
StockQuote
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuote
symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73
StockQuoteR1
R2
R3W2’’
STRENGTH=1
![Page 84: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/84.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Online References
![Page 86: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/86.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
SIMD Project
http://code.google.com/p/simd-cxx/
![Page 87: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/87.jpg)
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
Summing Up
![Page 88: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/88.jpg)
Pro
prie
tary
Info
rmat
ion
- D
istri
butio
n w
ithou
t Exp
ress
ed W
ritte
n P
erm
issi
on is
Pro
hibi
ted.
© 2009, PrismTech. All Rights Reserved
Concluding Remarks‣ DDS is the ideal fit for addressing several of the
data distribution and management challenges faced by financial firms, such as:‣ Market Data Distribution
‣ High Performance Caching
‣ Real-Time Business Intelligence
‣ OpenSplice DDS is an Open Source implementation of the DDS standard widely deployed and delivering great performance
‣ OpenSplice DDS is Commercially Supported Open Source Software
Delivering Performance, Openness, and Freedom
OpenSplice DDS
Enterprise Ed.
Community Ed.
Compact Ed.
Professional Ed.
Embrace Performance, Openness and Freedom. Join the OpenSplice DDS Community at www.opensplice.org
![Page 89: I need the data and I need it 1ms ago!](https://reader034.vdocuments.us/reader034/viewer/2022042815/557819cdd8b42ab40c8b4c47/html5/thumbnails/89.jpg)
Online Resources
http://www.opensplice.com/
emailto:[email protected]
http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com
http://bit.ly/1Sreg
© 2009, PrismTech. All Rights Reserved
http://www.slideshare.net/angelo.corsaro
http://twitter.com/acorsaro/