Low Latency MPEG-DASH SystemOver HTTP 2.0 and WebSocket

Xiaona Wu1,2(B), Cheng Zhao1, Rong Xie1,2, and Li Song1,2(B)

1 Institute of Image Communication and Network Engineering,Shanghai Jiao Tong University, Shanghai, China

{youli206,xierong,song li}@sjtu.edu.cn, zhaocheng100a@163.com2 Cooperative Medianet Innovation Center,

Shanghai Jiao Tong University, Shanghai, China

Abstract. Dynamic Adaptive Streaming over HTTP (MPEG-DASH)is an adaptive bitrate streaming technique that breaks the video con-tents into some sequences of small HTTP-based file segments in dif-ferent bitrates. With enough bandwidth now, live latency has becomethe most serious problem. MPEG has discussed two core experimentsServer and Network-assisted DASH (SAND) and Full Duplex HTTP-compatible Protocols (FDH) to improve performance of video streaming.In this paper, we refer the two ideas and complete a low delay streamingsystem over HTTP 2.0 and WebSocket. Based on our experiments, wecould adaptively choose which bitrate of segments to push according tothe network condition. With the smaller header size, utilization of band-width has been improved and there is 40.22% start-up time saved and57.96% transmission latency saved averagely in all situations.

Keywords: MPEG-DASH · FDH · SAND · HTTP 2.0 · WebSocketLive latency

1 Introduction

Live video streaming has become more and more popular recently. Along withthe increasing demand for better user experience, we need to solve problem of livelatency in live situations. Compared with traditional protocols and transmissionmethods [1], MPEG-DASH standard supposed two core experiments FDH andSAND using HTTP 2.0 and WebSocket.

MPEG-DASH was developed under MPEG (The Moving Picture ExpertsGroup) which contains the manifest file Media Presentation Description Fileand video segments with data in different bitrates. Studies on DASH startedin 2010 and DASH became a Draft International Standard in January 2011,and an International Standard in November 2011. The MPEG-DASH standard[2] was published in April, 2012. The media description file defines the mediainformation including adaptation sets, representations and so on. Segments aredefined based on two kinds of formats, ISO Base Media File Format [3] and

c© Springer Nature Singapore Pte Ltd. 2018G. Zhai et al. (Eds.): IFTC 2017, CCIS 815, pp. 357–367, 2018.https://doi.org/10.1007/978-981-10-8108-8_33

358 X. Wu et al.

MPEG-2 Transport Stream [4]. In 2014 DASH standard was updated to version2 and proposed FDH [5] and SAND [6] in 2015.

FDH defines full duplex HTTP-compatible protocols which both allowsserver-initiated and client-initiated transactions. The idea can be applied overHTTP 2.0 and WebSocket. In a DASH session the client requests the MPD filefirst and then requests video segments with corresponding segment URL and apush strategy. Then the server responses with the requested segments followingthe given push strategy. PushDirective is used to signal the push strategy thatthe client wants the server to use, while PushAck is used to inform the clientwhich strategy the server follows. The whole structure can reduce the transmis-sion delay by decreasing the number of requests in live streaming. However, allthe push strategies only support pushing segments from a single representation.

SAND enables a bidirectional messaging plane between the clients and otherDASH aware network elements (DANE) to carry any kind of operational informa-tion and assistance information. SAND messages includes Parameters Enhanc-ing Delivery (PED) messages exchanged between DANEs, Parameter EnhancingReception (PER) messages sent from DANEs to DASH clients and Metrics andStatus messages sent from clients to DANEs. These messages are delivered overHTTP via a URL in a HTTP header or a SAND channel (e.g. WebSocket).

In this work, we implement a low latency DASH system over HTTP 2.0 andWebSocket [7]. HTTP 2.0 supports function of server push. While using the HTTP2.0 protocol in adaptive streaming, another problem we need to solve is how toselect the quality of segments pushed. Of course, we can push definite amount ofsegments and change the selection after receiving a new request. In this article,we implemented a system using WebSocket to feedback SAND messages from theclient and make server determine which kind of content to be pushed.

The remainder of the paper is organized as follows. In Sect. 2, we summarizethe recent and related work on MPEG-DASH. Section 3 discusses our own systemincluding the push scheme and implementation of WebSocket to send SANDmessages to the server from the client. In Sect. 4, we describe the structure ofour system and its corresponding open source tools. The results of experimentsand comparison with traditional design are also shown. Section 5 concludes thepaper.

2 Related Work

MEPG-DASH is becoming the popular transmission protocol and many peopleresearch on it. Some use cases have been presented in [8] for hybrid broadcastingand it explained how the MPEG-DASH standard performed. Datasets composedof segments with different representations are provided in [9]. Comparisons ofHTTP 1.1 and HTTP 2.0 are provided both in terms of functionalities and pagedownload time in [10]. As to the researches on low latency, [11] conducted anumber of practical experiments on latency for protocols of HTTP 1.1, SPDYand HTTP 2.0. [12] explored WebRTC in a streaming transport process to carryvideo data and observed its contribution in low delay streaming with fast channelswitching.

Low Latency MPEG-DASH System Over HTTP 2.0 and WebSocket 359

Fig. 1. Traditional MPEG-DASH streaming system

One experiment demonstrated how a DASH client could use Quality of Server(QoS) information to achieve faster quality convergence at start up in [13]. Theoptimal quality is reached much faster when the DASH client is assisted inestimating the available bandwidth via SAND messages. In this paper, we notonly consider push technology from FDH idea, but also explore how to controlthe quality switch in server side. We finally establish our SAND system basedon HTTP 2.0 and WebSocket.

3 Low Latency MPEG-DASH System

In this section, we will introduce our system using push technology and Web-Socket in details.

Fig. 2. Low latency MPEG-DASH streaming system

360 X. Wu et al.

As shown in Fig. 1, media contents are composed of MPD files and DASHsegments in kinds of qualities, which are stored in the web server after generated.Then client requests for the MPD file first, parses it, gets the path of targetsegments according to the network conditions and requests them one by one intraditional streaming system.

In our system, we utilize the push idea in FDH over HTTP 2.0 and SANDmessages to improve the performance. The server will push next several segmentsand switch to different bitrates adaptively according to SAND messages sentfrom the client as shown in Fig. 2.

3.1 Push Strategy

In our system, we use the HTTP 2.0 protocol to implement push function. Theserver will push the corresponding resources to the client before it sends itsrequests for needed certain segments, which can reduce the live and start-updelay. This method can also decrease the number of connections to server andthe size of header sent by the client.

In this section, we mainly talk about two methods for media push in oursystem.

As we said before, push technology can reduce both live latency and start-uplatency. For the first purpose, we need to push several media segments to theplayer not wasting time for the connection establishment between them againand again as shown in Fig. 3(a). And for the start-up delay, we need to do somechanges in the beginning of playback for a video as shown in Fig. 3(b).

(a) Case I (b) Case II

Fig. 3. Two methods for reducing time delay. Case I in (a) shows that the serverpushes n segments for each request and Case II in (b) shows the server pushes MPDfile, initialization file and all the following segments to start playback more quickly.

Low Latency MPEG-DASH System Over HTTP 2.0 and WebSocket 361

After receiving the manifest file, the client parses the file and requests for thesegments in proper bitrates. The web server will push not only the requested onebut also a certain number of following segments in the same quality. After readingfrom the buffer the client will repeat the request process. This can obviouslyreduce the Round Trip Time (RTT) in live streaming. Considering that pushstrategy will mostly be used in live streaming, users watch videos continuously.So there will be no data redundancy because all resources will be provided tousers.

For the purpose of decreasing start-up delay, we can just push resources afterthe server receives the request for MPD. Once web server detects the request,it pushes the initialization mp4 file and several video segments and the clientcan start to play video immediately. Although we may not play the segments inmost appropriate quality to make full use of the bandwidth at first, we reducethe start-up latency to get better experience for users.

3.2 Messages Feedback Over WebSocket

Using pull technology the client would choose the required quality of segmentsaccording to the bandwidth calculated by the downloading bitrate. Then whenwe use the push technology, we need to consider the selection of the properquality. We attempted several ways to transfer the bandwidth message fromclient to server. WebSocket has a high real-time performance so we choose it totransfer the SAND messages.

Algorithm 1. Feedback SAND messages using WebSocketThe client:Input: current downloading bitrate A, previous bitrate B

while ‖A − B‖ is more than 500bps dosend SAND messages M (M=A) to serverlet B = Adetect downloading bitrate and update A

The server:Input: SAND messages M

Pushing repAif M accepted then

if server has repM thenPush repM in new quality

elsePush highest quality segments

elsePush repA

The basic idea is making the client to estimate the network conditions at alltime. Algorithm 1 gives the detailed description. When there is a large enough

362 X. Wu et al.

change the client will send the current downloading bitrate to server. Then theserver can judge the message and change to push segments in another bitrate toavoid the network choked or bandwidth wasted. Obviously this scheme proposeda more robust rate adaption method for live streaming. However, we did nottake the sharp changing of network into consideration and we will discuss howto improve robustness of the system in the future.

4 Experiments and Comparisons

We implement the low latency MPEG-DASH system based on HTTP 2.0 proto-col and WebSocket. The datasets Big Buck Bunny [14] we use are from ITEC-Dynamic Adaptive Streaming over HTTP [15]. The corresponding parts withdifferent segment durations we use are shown in Table 1. We choose Node.js[16] as the web server implementing our push strategy and open source projectopenssl to generate the key and certificate in the server to establish HTTPS con-nection for that is the need of HTTP 2.0 protocol. For the client side, dash.js [17],an opensource library, acts as the player developed by DASH Industry Forumand supported by many industry media companies. It also supports the pushtechnology and we only need to add some marks to do our experiments.

Table 1. Datasets with different durations

Index Duration FrameWidth FrameHeight Bitrate (kbps)

1 1 s 854 480 409

2 2 s 854 480 1236

3 6 s 854 480 3583

4 15 s 854 480 9517

4.1 Adaptive Switch Over WebSocket

First, we test adaptive switches according to SAND messages over WebSocket.The 1 s segments are used as MPEG-DASH video sources with bitrates of

226106bps, 270316bps, 352546bps, 424520bps, 537825bps, 620705bps, 808057bpsand 1071529bps shown in Table 2. Using SimpleHTTPServer module in Pythonto start a server for the dash.js player interface and node server for the videoresources, we can get all the information of network transmission in the Chromedevtools [18]. A WebSocket connection is established between the server andclient.

When network condition is changed large enough, the client will send thecurrent downloading bitrate message to media source server. Then the serverjudges the message and chooses segments to push. The results of adaptive switchare shown in Fig. 4.

Low Latency MPEG-DASH System Over HTTP 2.0 and WebSocket 363

Table 2. Datasets for adpative switch

Name FrameWidth FrameHeight Bitrate (kbps)

bunny 226106bps 480 360 187

bunny 270316bps 480 360 206

bunny 352546bps 480 360 241

bunny 424520bps 480 360 243

bunny 537825bps 854 480 409

bunny 620705bps 854 480 454

bunny 808057bps 1280 720 579

bunny 1071529bps 1280 720 751

From the above figure, we can see that the bandwidth of network is changingalong with time. For example, if the server finds out that the condition becomesworse, it will push lower bitrate segments to client to avoid the playback freezesdue to the slow downloading speed of high quality resources. On the contrary,when network gets better, the server will choose higher bitrate segments to pushin order to improve the bandwidth utilization.

Fig. 4. Adaptive switches over WebSocket with SAND messages

364 X. Wu et al.

4.2 Key Indicators

In this part we test the performances of our own system. For the experiments, wemainly talk about three main indicators: header sizes of requests and responses,start-up delay and transmission latency.

Header Size. First, we consider the difference of header sizes. Using the 1 ssegments we find that header size of requests is 514 and 446 bytes respectivelyand the response header size is 299 and 142 bytes for traditional and our systems.We get the new result of 259 and 80 bytes when we add the push technology inHTTP 2.0.

Fig. 5. Comparison of header sizes of request and response

The size is apparently reduced about 50% and it eventually saves the band-width resources. We get the same results when using 2 or 6 or 15 s segments.Figure 5 shows the comparison of request headers in the left and response headersanother side intuitively.

Start-Up Delay. The start-up delay is another important indicator for videostreaming. We test start-up delay using different segments and receive the follow-ing results in Fig. 6. The line above represents the start-up delay in our systemand the other is for the traditional transmission. As we can see, the player willplayback video contents more quickly. The time saved for segments’ duration of1, 2, 6 and 15 s is about 45.46%, 26.7%, 43.21% and 45.51%.

Transmission Latency. The last indicator we need to consider is the trans-mission latency. When we are stuck in a bad network, it is pretty significant tokeep downloading resources to avoid playback freezing. Under the considerationof the segments’ sizes, we compare the latency of same one in two systems eachtime. In the situation for 1 s segments, the results are shown in Fig. 7(a) andcompared for each one.

Low Latency MPEG-DASH System Over HTTP 2.0 and WebSocket 365

(a) 1s (b) 2s

(c) 6s (d) 15s

Fig. 6. Start-up delay for 1 s, 2 s, 6 s and 15 s segments.

(a) 1s (b) 2s

(c) 6s

Fig. 7. Transmission delay for 1 s, 2 s and 6 s segments.

366 X. Wu et al.

However, some sudden breaks may happen when downloading segments andwe have to request again or change to the next one. If the length of one segmentis too long, contents played will be dulplicate or missing. So we will ignore the15 s segment situation. The other results are shown in Fig. 7(b) and (c). It isobvious that the larger segments are, the more advantages it will have in savingtime. The time decreased are respectively 43.82%, 57.11% and 72.96%.

5 Conclusion

In this paper, we use HTTP 2.0 and WebSocket to develop a low latency MPEG-DASH streaming system. HTTP 2.0 protocol is utilized for push technology,while WebSocket is used to transfer the SAND messages in order to adaptivelychoose the resources pushed. Also, we take some experiments to prove its advan-tages of smaller header sizes to save network bandwidth, lower start-up delay togive better user experience and lower transmission latency to save time in livestreaming. To sum up, we observe that there is 40.22% start-up time saved and57.96% live delay saved averagely in all situations. Obviously the start-up timeand live delay are saved much and this scheme can be used in the live streaming.It can provide users a more comfortable experience for live broadcasting.

Acknowledgement. This work was supported by NSFC (61671296 and 61521062),the 111 Project (B07022 and Sheitc No. 150633) and the Shanghai Key Laboratory ofDigital Media Processing and Transmissions.

References

1. Stockhammer, T.: Dynamic adaptive streaming over http: standards and designprinciples. In: ACM Conference on Multimedia Systems, pp. 133-144 (2011)

2. ISO/IEC 23009–1:2012, Information technology - dynamic adaptive streaming overhttp(dash) - part 1: Media presentation description and segment formats (2012)

3. ISO/IEC 14496–12:2015, Information technology - Coding of audio-visual objects -Part 12: ISO base media file format (2015)

4. ISO/IEC 13818–1:2015, Information technology - generic coding of moving picturesand associated audio information - part 1: Systems (2015)

5. Information Technology Dynamic adaptive streaming over HTTP(DASH) - Part6: DASH over Full Duplex HTTP-compatible Protocols(FDH) (2015)

6. ISO/IEC 23009–5:2017, Information Technology Dynamic adaptive streaming overHTTP(DASH) - Part 5: Server and network assisted DASH (SAND) (2017)

7. RFC 6455, The WebSocket protocol (2011)8. Feuvre, J.L., Concolato, C.: Hybrid broadcast services using mpeg dash9. Lederer, S., Timmerer, C.: Dynamic adaptive streaming over http dataset. In: ACM

Sigmm Conference on Multimedia Systems, Mmsys 2012, pp. 89–94, Chapel Hill,Nc, Usa (2012)

10. Corbel, R., Stephan, E., Omnes, N.: Http/1.1 pipelining vs http2 in-the-clear:performance comparison. In: International Conference on New Technologies forDistributed Systems, pp. 1–6 (2016)

Low Latency MPEG-DASH System Over HTTP 2.0 and WebSocket 367

11. Naik, N., Jenkins, P.: Web protocols and challenges of web latency in the web ofthings. In: Eighth International Conference on Ubiquitous and Future Networks,pp. 845–850 (2016)

12. Zhao, S., Li, Z., Medhi, D.: Low delay mpeg dash streaming over the webrtc datachannel. In: IEEE International Conference on Multimedia Expo Workshops, pp.1–6 (2016)

13. Thomas, E., Deventer, M.O.V., Stockhammer, T., Begen, A.C., Famaey, J.:Enhancing mpeg dash performance via server and network assistance. In: Ibc(2015)

14. BigBuckBunny. http://www-itec.uni-klu.ac.at/ftp/datasets/DASHDataset2014/BigBuckBunny/

15. ITEC. http://www-itec.uni-klu.ac.at/dash/16. Node.js. https://nodejs.org/en/17. Dash.js. https://github.com/Dash-Industry-Forum/dash.js18. Chrome DevTools. https://developer.chrome.com/devtools