streaming video over the internet andreas panteli 03/05/2012 ece 654 advanced computer networks 03...
TRANSCRIPT
Streaming Video Over The Internet1
Streaming Video Over the Internet
Andreas Panteli
03/05/2012
ECE 654 Advanced Computer Networks
03 May 2012
Streaming Video Over The Internet2
Introduction
Video principles When an image appears on the retina it is retained for some
millisecond before decaying If a sequence of images is drawn line by line at 50 images/sec the
eye does not notice that it is looking at discrete images
Streaming Video Video content need not be downloaded in full It is being played out while parts of the content are being received
and decoded
Real-time multimedia have timing constraints Audio and video data must be played out continuously Bandwidth, delay and loss requirements
No QoS guarantees to streaming video, from the current best-effort Internet!
03 May 2012
Streaming Video Over The Internet3
An Architecture for Video Streaming
03 May 2012
Raw Video
CompressedAudio
Application-layerQoS ControlCompressed
Video
TransportProtocol
VideoDecoder
AudioDecoder
Raw Audio
VideoCompression
AudioCompression
Application-layerQoS Control
TransportProtocol
MediaSynchronization
Storage Device
Streaming Server Client/Receiver
Internet(Continuous media distribution services)
Streaming Video Over The Internet4
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet5
Key Areas
Video Compression JPEG standard MPEG standard
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet6
Digital Video
Digital video is represented as a sequent of frames, each consisting of a rectangular grid of picture elements (pixels)
For color video, 8 bits for each of the RGB colors are being used 24 bits/pixel => 16 million colors
To produce smooth motion, digital video, we must display at least 25 frames/sec, but to avoid flickering we need to repaint each frame on the screen twice
Smoothness and flickering affects the bandwidth requirements Consider the lower video configuration for computer monitor
1024x768 with 24 bits/pixel and 25 frames/sec Need to be fed at 472 Mbps
We need video compression!03 May 2012
Streaming Video Over The Internet7
Video Compression
Two algorithms are needed Encoding and Decoding
Asymmetries of the algorithms Acceptable for the encoding algorithm to be slow and require
expensive HW, if the decoding algorithm is fast Only for video on demand For real-time multimedia like video conferencing, slow encoding
is unacceptable Acceptable to have a slightly different video signal than the original,
after encoding and decoding (Lossy system) Lossy systems are important
Accepting a small amount of information loss can give a huge payoff in terms of the compression ratio possible
03 May 2012
Streaming Video Over The Internet8
The JPEG Standard
Compression algorithm for still images Could be applied to each image in succession to achieve video
compression
03 May 2012
Compute the luminance Y and the two chrominances I and Q Y = 0.30R + 0.59G + 0.11B I = 0.60R – 0.28G - 0.32B Q = 0.21R – 0.52G + 0.31B
Compress the two chrominances twice than the luminance Lossy reduction, but the eye barely notices it since it responds to
luminance more than to chrominance
JPEG often produces 20:1 compression or better Decoding a JPEG image requires running the algorithm backwards
Streaming Video Over The Internet9
The MPEG Standard (1/2)
Main algorithms for video compression and international standards since 1993
Can compress both audio and video
03 May 2012
Three parts in MPEG-1 Audio Video System (integrates the other two)
Streaming Video Over The Internet10
The MPEG Standard (2/2)
Two kinds of redundancies in videos Spatial redundancy
Coding each frame separately with JPEG Temporal redundancy
Consecutive frames are often almost identical
MPEG-1 output consists of four kinds of frames I (Intracoded) frames: Self-contained JPEG-encoded still pictures P (Predictive) frames: Block-by-Block difference with the last frame B (Bidirectional) frames: Differences between the last and the next
frame D (DC-coded) frames: Block averages used for fast-forward
MPEG-1 can achieve a compression factor of 40 Decoding has about the same complexity as encoding
03 May 2012
Streaming Video Over The Internet11
Key Areas
Video Compression
Application-Layer QoS Control Congestion control Error control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet12
Application-Layer QoS Control
Congestion Control
Rate Control Source-Based Rate
Control Receiver-Based Rate
Control Hybrid-Rate Control
Rate Shaping Frame-dropping filter Layer-dropping filter Frequency filter Re quantization filter
Error Control
FEC Delay-Constrained
Retransmission Error-Resilient Encoding Error-Concealment
03 May 2012
Avoids congestion and maximizes video quality in the presence to packet loss
Streaming Video Over The Internet13
Rate Control (1/2)
Technique to determine the sending rate of video traffic based on the estimated available bandwidth of the network
Source-based rate control The sender is responsible for adapting the video transmission rate
based on feedback from the receivers Probe-based approach
Sending rate adjustment to maintain the packet loss ratio below a certain threshold
Model-based approach
03 May 2012
Streaming Video Over The Internet14
Rate Control (2/2)
Receiver-based rate control The receivers regulate the receiving rate of video streams by
adding/dropping channels Probe-based approach
When no congestion is detected, join a channel => increase of it’s receiving rate
When congestion is detected, drop a channel => reduction of it’s receiving rate
Model-based approach Explicit estimation for the available network bandwidth Also based on Equation (1)
Hybrid rate Control Combination of the above mentioned techniques
03 May 2012
Streaming Video Over The Internet15
Rate Shaping
Matches the rate of a pre-compressed video bit stream to the target rate constraint
A rate shaper (or filter) is required for source based rate control
03 May 2012
Types of filters Codec filter
Performs transcoding between different compression schemes
Frame-dropping filter I, P ,B , D frames in MPEG
Frequency filter Operates in the frequency domain (DCT
coefficients) Re-quantization filter
Re-quantizes the DCT coefficients with a larger quantization step
Streaming Video Over The Internet16
Error Control (1/2)
Forward Error Correction (FEC) Add redundant information so that original message can be
reconstructed in the presence of packet loss Channel coding
Stream is chopped into segments Each segment is packetized into k packets A block code generates an n-packet block, where n>k User only needs to receive any k packets out of n of the block
Source coding N-th group of blocks contains redundant information of (n-1)th
group of blocks Joint source/channel coding
Optimal rate allocation between Channel and Source coding
03 May 2012
Streaming Video Over The Internet17
Error Control (2/2) Delay-constraint Retransmission
03 May 2012
Error-resilient Encoding Enhances robustness of compressed video to packet loss A video is compressed into multiple streams (descriptions) Each description provides acceptable visual quality Combined descriptions provide a better visual quality
Error Concealment Performed by the receiver when packet loss has already occurred Spatial interpolation
Missing pixel values are reconstructed using neighboring pixels Temporal interpolation
Loss data is reconstructed using data from the previous frame
When the receiver detects the loss of packet N
if (Tc + RTT + Ds < Td(N))send the request for packet N to
the sender
WhereTc current timeRTT estimated round-trip timeDs a slack term (e.g receiver’s decoding delay)Td(N) time when packet N is scheduled for display
Streaming Video Over The Internet18
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services Network Filtering Application-level Multicast Content Replication
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet19
Network Filtering
Adequate support from the network is critical in order to provide quality multimedia presentations
03 May 2012
Routers have no knowledge of the format of the media streams Randomly discard packets
Filters receive the client’s requests and adapt the stream sent by the server accordingly
Typically, frame-dropping filters are used Drop packets in a way that gracefully
degrades the stream’s quality Increase bandwidth efficiency by discarding
late frames
R: Router
Streaming Video Over The Internet20
Application-level Multicast
Internet’s original design fails to effectively support streaming-media multicast
IP multicast has barriers Scalability, network management, support for higher layer functionality (e.g. error, flow
and congestion control)
Application-level multicast to the rescue Builds a multicast service on top of the Internet
Media bridge Routing in the application layer Each Media Bridge is interconnected with one or more neighboring Media-Bridges
through explicit configuration Collectively employ a distributed application-level multicast routing algorthm
03 May 2012
Unicast Multicast
Streaming Video Over The Internet21
Content Replication
Mirroring
Scatter copies of the original multimedia files in different locations around the Internet
Clients retrieve data from nearest duplicate server
Caching Exploits temporal locality Cache retrieves data from
server Clients retrieve data from
cache if available Cache sharing and cache
hierarchies03 May 2012
Advantages Increased network’s bandwidth efficiency Reduced load on streaming servers Reduced latency for clients Increased availability
Streaming Video Over The Internet22
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms Intra-stream Synchronization Inter-stream Synchronization Inter-object Synchronization
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet23
Media Synchronization
Multimedia applications Various integrated media streams that must be presented in a
synchronized fashion
Media streams may lose synchronization after moving from the server to the client
Three layers of synchronization Intra-stream synchronization
Maintain continuity of logical data units Inter-stream synchronization
Maintain temporal relationships among different continuous media e.g. movements of the lips of a speaker do not correspond to the
presented audio Inter-object synchronization
Relation with time-independent data, such as text, still image and streams
e.g. during a presentation, audio is commenting one slide while another slide is presented
03 May 2012
Streaming Video Over The Internet24
Media Synchronization Mechanisms
Delay in the network is unpredictable due to the best-effort nature of the Internet
Axes-based specifications, or time-stamping At the source, a stream is time-stamped to keep temporal
information within the stream and with respect to other streams
Mechanisms implemented on the end systems Preventive mechanisms
Minimize latencies and jitters i.e. disk-reading scheduling algorithms, network transport
protocols, operating systems and synchronization schedulers Corrective mechanisms
Recover synchronization in the presence of synchronization errors
i.e. Stream synchronization protocol (SSP) Concept of ”intentional delay”
03 May 2012
Streaming Video Over The Internet25
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media Transport Protocols Session Control Protocols
03 May 2012
Streaming Video Over The Internet26
Protocols for Streaming Video
Data Plane Compressed video/audio is
packetized at the real-time transport protocol (RTP) layer
RTP packetized streams are then passed to the UDP/TCP layer and IP layer
IP packets are transported over the Internet
03 May 2012
Control Plane Real-time control protocol (RTCP) and real-time streaming protocol
(RTSP) packets are multiplexed at the UDP/TCP layer IP packets are transported over the Internet
Streaming Video Over The Internet27
Transport Protocols UDP and TCP provide basic transport functions
UDP and TCP Multiplexing, Error control (e.g. checksum)
TCP only: Retransmission, Congestion control, Flow control
UDP is typically employed as the transport protocol for video streams No packet delivery guarrantees Receiver needs to rely in upper layer to detect packet loss
RTP and RTCP run on top of UDP/TCP
03 May 2012
RTP : does not guarantee QoS or reliable delivery Time stamping Sequence numbering Payload type identification Source identification
RTCP : designed to work in conjunction with RTP QoS feedback Participant identification Control packets scaling Inter-media synchronization Minimal session control
information
Streaming Video Over The Internet28
Session Control Protocols
RTSP (Real-Time Streaming Protocol) Support VCR-like control operations
(stop, pause/resume, fast forward, fast backward) Choosing delivery channel (UDT, multicast UDT, TCP) Establish and control streams of continuous audio and video media
between the media servers and the clients Media retrieval: ask server to setup a session to send the
requested media data Adding media to an existing session: server or client can
notify each other about any additional media becoming available to the established session
SIP (Session Initiation Protocol) Similar to RTSP Support user mobility by proxying and redirecting request
03 May 2012
Streaming Video Over The Internet29
References
A. S. Tanenbaum, Computer Networks, 4th ed. New Jersey: Prentice Hall, 2002.
Dapeng Wu, Hou, Y.T., Wenwu Zhu, Ya-Qin Zhang, Peha, J.M., "Streaming video over the Internet: approaches and directions“, Circuits and Systems for Video Technology, IEEE Transactions on , vol.11, no.3, pp.282-300, Mar 2001
Dapeng Wu, Yiwei Thoms Hou, Ya-Qin Zhang, "Transporting real-time video over the Internet: challenges and approaches," Proceedings of the IEEE , vol.88, no.12, pp.1855-1877, Dec 2000
Dapeng Wu and Y. Thomas Hou and Jason Yao and Y. Thomas and Hou Jason Yao and H. Jonathan Chao, “Real-time Video over the Internet: A Big Picture”, IEEE NetWorld+Interop, 2000
03 May 2012
Streaming Video Over The Internet30
Questions?
Thank you!
03 May 2012