ertan deniz milsoft a.s, teknokent odtu,ankara/turkey edeniz @milsoft.tr

Copyright © MilSOFT,Turkey UNCLASSIFIED 1

Ertan DENIZMilSOFT A.S, Teknokent ODTU,Ankara/Turkey

[email protected]

Huseyin Kutluca, [email protected]

Video Over DDSDDS Information Day

Brussels, Belgium-June-26,2007

mailto:[email protected]




• Introduction• Image Exploitation System for UAVs• Why Use DDS for Video Streaming?• MilSOFT DDS • DDS Topics• QoS choices• Future Work• Conclusion

Outline


Introduction

• DDS middleware• Location transparency with pub/sub mechanism

• Redundancy and fault tolerance mechanisms

• Multiple readers/writers

• Transient and persistent services

• Asynchronous, time-independent data distribution

• Rich set of QoS, suitable for real-time systems

• Wide range of Application Areas• C4I

• Industrial automation

• Distributed control and simulation

• Telecom equipment control

• Sensor networks

• Use Case: Image Exploitation System for UAVs


Data Receiver

Pre-processor

DB Server

Screening Exploitation 1 Exploitation 2

Sensor Data(EO/IR, SAR, GMTI)

Telemetry

High CapacityStorage Unit

Raw Data Recording

DDS based Data

Streaming

DDS based Data

Streaming

Pre-processed & Georecorded Data

Artifacts

All Data in DB

Raw Data

DDS on UAV Image Exploitation System


DDS on UAV Image Exploitation System

• Data • Events

• User information

• Resource Management

• Radar data streaming• SAR streaming

• GMTI streaming

• Telemetry streaming

• Video streaming• Live data streaming

• Recorded data streaming

• Data Publisher: DataReceiver

• Data Subscribers: Consoles and Pre-Processor

• Content of data MPEG-2 TS


MPEG-2 TS (Transport Stream)

• Elementary Stream• Video, audio, telemetry data, etc...

• From each elementary stream Packetized Elementary Stream (PES) is formed

• And each PES is broken into fixed-size Transport Packets

• TS packets are 188 bytes of which 4 bytes are for the header and 184 bytes are for the payload

• Transport Stream can multiplex multiple elementary streams• The TS packet header contains a Packet Identifier (PID) to identify

different elementary streams

• 8 Mbit data rate for UAV Image Exploitation System


Why use DDS for video streaming?

• Publish-Subscribe simplifies the development process• The clients do not need to know where the server is, and the server

does not need to know the clients

• DDS specification implies mutlicast communication and the interoperability wire protocol RTPS supports and recommends multicasting• Dramatically decreased bandwith usage with DDS implementations

which support multicast

• Quality Of Service attributes like reliability, durability, transport priority directly can be used for video streaming channels

• Generally DDS implementations have very low latencies in data transmission compared to other middlewares, which makes it a good choice


Why MilSOFT DDS

• MilSOFT DDS is compliant with current DDS specification, version 1.2

• Provides Minimum Profile QoS attributes, which are neccessary for video streaming

• Completely decentralized design

• No dynamic memory allocation after initialization

• Zero-copy access to data

• Implements multicast communications


More information on http://dds.milsoft.com.tr

MilSOFT DDS Middleware


Video Over DDS Architecture


DDS Types and Topics Used For Video Streaming

• So how do we encapsulate MPEG2-TS packets into DDS types?

• There are two DDS types and two corresponding Topics.• VideoHeader VideoHeaderTopic

• VideoPacket VideoPacketType

• VideoHeader• Contains the header of the video, which is needed to initialize the

decoder. (FPS, height, width, etc... )

• VideoPacket• Contains MPEG2-TS packets


VideoHeader Topic

struct VideoHeader

{long headerType; // key

sequence<char, HEADER_MAX_DATA> headerData;

};

• Published once for each Transport Stream

• Contains the necessary information • initialize the video decoder

• information about the other multiplexed elementary streams

• Key to distinguish the video header from other header data

• Late joiners also get this topic


QOS for VideoHeader Topic

• VideoHeaderTopic is necessary to initialize decoding of the video

• Header information is critical for clients• Use Reliability QoS (RELIABLE)

• Both publishers and subscribers are Reliable

• Late joining viewers need this information• Use Durability QoS (TRANSIENT_LOCAL)

• There are different instances of the header according to it’s type. We need only the last update of each instance!• History QoS is default, kind = KEEP_LAST, depth = 1


VideoPacket Topic

struct VideoPacketType

{

long frameNumber;

sequence<char, PACKET_MAX_DATA_SIZE> packetData;

};

• MPEG2-TS packets are too small in size (188 bytes).

• VideoPacket type sends multiple TS packets of a video frame at each VideoPacket sample• Decreases the overhead and increases the bandwidth utilization

• A frame can span multiple VideoPacket samples (fragmented)

• Each sample contains TS packets of only one frame


QOS for VideoPacket Topic

• For VideoPacketTopic we may need both Reliable and Best Effort delivery, according to client needs• Use Offered >= Requested semantics

• Server publishes with a Reliable writer ( Offer Reliable communication )

• Clients subscribe with either Reliable or Best Effort readers

• Data is not needed by late joiners• Durability is Volatile

• VideoPacket Type has no key, but we can use History QoS for buffering• History QoS, kind = KEEP_LAST, depth value can be used as a buffer

(set it to more than 1)


Other Concerns And Solutions

• We don’t want out of sequence frames• DDS already satisfies this, no out of order packets!

• We don’t want frames which are too late• Use Lifespan QoS

• Decoding takes time• Either read data from a DataReaderListener and immediately put into

the queue of another thread

• Use WaitSets


Future Work

• Implement advanced services like rewind, replay, fast-forward etc...

• Network reservations• Use RSVP (IntServ) to reserve bandwith

• Analyze different methods to give DDS video streams a priority• Use Transport_Priority QoS (not completely standard)

• More standardization is needed in this area

• Standardize Flow Control mechanisms in DDS for better utilization of network bandwith


Conclusion

• Video over DDS is possible!

• DDS provides a good set of QoS attributes for video streaming

• Performance requirements• Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS• 20 Mbit video streaming is possible with DDS


Thank You!

ertan deniz milsoft a.s, teknokent odtu,ankara/turkey edeniz @milsoft.tr

Documents

dbraw data dds

telemetry data

data transmission

uavswhy use dds

current dds specification

video streamingso

elementary stream pes

payloadtransport stream