Subjective and Objective Evaluation and PacketLoss Modeling for 3D Video Transmission over

LTE NetworksMoustafa M. Nasralla, C.T.E.R. Hewage, and Maria G. Martini

Kingston University, London, UK{M.Nasralla, C.Hewage, M.Martini}@kingston.ac.uk

Abstract—The recent Long-Term Evolution (LTE) standard,thanks to the provision of high data rates, will enable bandwidthdemanding multimedia applications like Three Dimensional (3D)video streams over wireless. In this paper, we study the transmis-sion of 3D video sequences over LTE networks by modeling anLTE wireless network, which uses the best performing downlinkpacket scheduling strategy (Modified Largest Weighted DelayFirst (M-LWDF)) for delay sensitive applications, through thewell-known Gilbert-Elliot Channel Model (GE). Also, we performsubjective and objective quality evaluation of the received 3Dvideo sequences. The perceived video sequences are impairedwith six packet loss rates produced by the corresponding LTEnetwork. The three major contributions of the paper are: 1) GEparameters that represent real statistics of an LTE network 2) asubjective study on 3D video transmission over an LTE systemmodeled as a GE channel for different levels of impairmentsand 3) the provision of a publicly available database with 3Dvideo sequences affected by packet losses distributed accordingto a GE model and associated Mean Opinion Score (MOS)values, to enable researchers to test their video quality evaluationalgorithms.

Index Terms—LTE, 3D video, Gilbert-Elliot channel model,MOS and Subjective evaluation

I. INTRODUCTION

With the recent advent of multimedia applications suchas 3D video streaming, Voice over IP (VoIP) and Videoconferencing, the LTE wireless standard has emerged to copewith these services efficiently, thanks to the high data ratessupported. The Third Generation Partnership Project (3GPP)LTE standard enables high data rate content transfer bysupporting radio access with up to 100Mbps in full mobilitywide area deployments and 1Gbps in low mobility localarea deployments [1]. The progress in multimedia compres-sion/representation fosters two-dimensional (2D) and three-dimensional (3D) multimedia communications, utilizing theever-increasing available bandwidth for a variety of personal-ized services. Indeed, the projection in [2] reports that videowould exceed 90% of the global consumer traffic and 66% ofthe world’s mobile traffic by 2015.

Wireless channels are known by generating burst packet-losses/errors due to the nature of the connectivity. The Gilbert-Elliot Channel Model (GE), introduced in [3] and [4], is asimple channel model chosen to characterize/study the packetlosses produced from wireless fading channels. Packet lossesthat may occur during the transmission of data in a network

could be due to several factors: channel errors, networkcongestion or delay deadline violation.

In this paper, we mathematically model the packet lossbehavior of the LTE wireless network. Different 3D videostreams sent over different LTE network‘s scenarios are con-sidered in order to ensure different packet loss conditions toconduct quality evaluation studies on the impaired sequences.

The paper is organized as follows. Section II compares andcontrasts the related work and the approach adopted in thispaper. An overview of LTE systems and the GE model isprovided in Section III. The system model is discussed inSection IV, where we provide the scenario description andthe 3D video encoding strategy. The evaluation methodologyand results are presented in Section V, where the relevantsubjective and objective evaluation metrics and relevant resultsare discussed. Finally, Section VI concludes the paper.

II. RELATED WORK

The authors in [5] and [6] investigate a developed packetloss model for H.264 video transmission over IEEE 802.11gwireless networks, where loss patterns are generated frommeasured data. They considered the GE channel model tocharacterize packet loss behaviors and extract loss patternsfrom measurement data for different scenarios. Results showedthat the model enables the evaluation of H.264 video trans-mission strategies in terms of error resilience and sensitivityto packet losses for IEEE 802.11g systems. We discussed in[7] context-aware ultrasound video transmission over WiMAX(Worldwide Interoperability for Microwave Access). The GEchannel model is considered to represent the characteristicsof a real WiMAX network. GE parameters were obtainedaccording to WiMAX measurements and packet loss-patternswere generated accordingly. The proposed video transmissionstrategy over WiMAX systems was evaluated via the generatedloss patterns.

The effect of random packet losses on the overall 3Dperception was studied in [8]. Subjective and objective qualitymeasurements are provided using several quality metrics.In addition, a 3D video database was developed to helpresearchers to evaluate their 3D video assessment strategiesunder different packet loss conditions. In this work, the authorsassumed that packets have the same probability of being lost(memoryless erasure channel). This assumption is not always

valid, in particular in wireless systems, where bursts of packetlosses often occur, due to the variability and memory ofthe channel. The works carried out in [9] and [10] presentsubjective quality evaluation measurements for various videosequences sent over noisy channels. The evaluation is carriedout by impairing the video sequences according to differentpacket loss rates with the help of the Gilbert channel model.The model is used to represent an error-prone network usinga given loss-pattern from the Joint Video Team (JVT) in [9],and to represent the IEEE 802.11 wireless system in [10].

The authors in [11] investigate the performance of different3D video rates delivery over LTE networks through simula-tions. This work includes objective evaluation metrics such aspacket loss ratio, delay, goodput and Peak Signal to NoiseRatio (PSNR).

In this paper, we study the quality degradation of 3D videotransmitted over LTE, through subjective quality evaluationtests. Different packet loss rates are introduced due to severalnetwork conditions described later in this paper. This workextends the study on subjective quality evaluation for 3Dvideo presented in [8], by considering a packet loss modelderived from the simulation of LTE systems. This contributionis expected to support researchers and developers in testingand developing their video quality assessment strategies withrespect to the recent wireless transmission technologies likeLTE and LTE-Advanced (LTE-A).

III. LTE WIRELESS SYSTEMS AND GILBERT-ELLIOTCHANNEL MODEL OVERVIEW

This section introduces a brief overview of LTE wirelessnetworks and the adopted GE channel model.

A. The LTE wireless system

LTE has been introduced by the Third Generation Partner-ship Project (3GPP) as the next technology after the 3.5G(HSPA+) cellular networks. The system architecture of the3GPP LTE system contains several base stations calledEvolved NodeB (eNodeB) where the packet scheduling pro-cess is performed along with other Radio Resource Manage-ment (RRM) tasks. LTE uses Orthogonal Frequency DivisionMultiple Access (OFDMA) in the downlink transmissionmode and Single Carrier Frequency Division Multiple Ac-cess (SC-FDMA) in the uplink transmission mode. OFDMAextends the multi-carrier technology Orthogonal FrequencyDivision Multiplexing (OFDM) to provide a better and moreflexible multiple access scheme. In other words, OFDMAsplits the frequency band into multiple orthogonal sub-carriers.This helps improving the system capability to support highdata rates, provide multi-user diversity, and compact the Inter-Symbol-Interference (ISI) [12] [13] [14].

Various packet scheduling algorithms have been developedto support Real Time (RT) and non-Real Time (NRT) flows,such as Proportional Fair (PF), M-LWDF, and ExponentialProportional Fair (EXP/PF) [15]. In the aforementioned sched-ulers, each flow is assigned a priority value by considering

specific metrics. The flow with the best metric is scheduledfirst at the next transmission time interval (TTI).

B. Gilbert-Elliot channel model

The GE model, introduced in [3] and [4], is a simple channelmodel. The GE model has been widely used in the literatureto characterize and represent the burstiness of packet loss-patterns produced by wireless fading channels, as discussed inSection II. The model is based on a two-state Markov chain.The two states represent the good (S0) and bad (S1) channelstates in terms of packet losses, as depicted in Figure 1.

Fig. 1. Gilbert-Elliot channel model.

The loss-pattern generated from a wireless fading channel,due to the transition from good to bad channel states, is com-posed of a consecutive sequence of correctly received packetsand a consecutive sequence of lost packets, as illustrated inFigure 2. This figure represents the received packets as (0)sand the lost packets as (1)s, in a loss-pattern sequence. Also,two bursts of different length are illustrated in this example,as one burst is composed of three lost packets and the otherone is composed of two lost packets.

Fig. 2. Burst representation in a loss-pattern sequence.

The GE model, depicted in Figure 1, is completely deter-mined by two parameters: the probability of packet loss PE ,and the mean burst length LB . These parameters are obtainedfrom the channel state transition probability matrix P. Thismatrix is composed of conditional probabilities according tothe following structure:

Pi,j = P [S(t) = Sj |S(t− 1) = Si]; i, j ∈ {0, 1} (1)

where the transition probability from state Si at time t − 1to state Sj at time t is represented. An example is given asfollows: {

q = P [S(t) = S0|S(t− 1) = S1],

p = P [S(t) = S1|S(t− 1) = S0](2)

The transition probability matrix is given by

P =

(1− p pq 1− q

). (3)

The stationary probabilities are also defined here to rep-resent stationary states: the mean arrival probability π0 =

P1,0

P1,0+P0,1and the mean loss probability π1 =

P0,1

P1,0+P0,1.

The mathematical expressions used to obtain the transitionprobabilities, i.e., P0,1 and P1,1, are derived in [16], wherethey are used to determine the GE parameters, i.e., PE andLB given as follows:

PE = π1 (4)

LB =1

q(5)

IV. SYSTEM MODEL

We discuss in the following the proposed transmissionmodel and the 3D video encoding strategy and traffic model.

A. The proposed transmission model

The transmission scenario considered in this work is basedon modeling the transmission of 3D video sequences over LTEwireless networks. We consider the same conditions of theLTE wireless network used in our previous study in [17] andreported in Table I. Briefly, we consider a single LTE cell withmultiple users, in which the users are uniformly distributed.The eNodeB is positioned in the centre of the cell. The users’mobility is pedestrian with constant speed of 3 km/h.

The serving eNodeB’s Medium Access Control (MAC)layer controls the available physical resource blocks (PRB)sby assigning them to the active flows which are competingfor resources. At the MAC layer, the M-LWDF scheduler isimplemented for serving the video flows. The reason whywe choose the M-LWDF scheduling scheme in this study isbecause the simulation results presented in [15] and [17] showthat the M-LWDF algorithm outperforms other packet schedul-ing algorithms in supporting delay sensitive applications suchas video transmission.

In order to consider a wide range of video qualities, weassume the LTE cell comprises maximum six 3D users,although the LTE system with 10 MHz can only supportfour 3D users with acceptable quality level as investigatedalready in [17]. Simulations are averaged over 10 simulationrepetitions in order to obtain accurate and reliable results.The simulation parameters adopted using the LTE-Sim [18]simulator are reported in Table I.

Different average packet loss ratios are produced accordingto the channel conditions and network parameters reportedin Table I, as in [17]. The packet loss ratio and the averageburst length parameters obtained from the simulation resultsare averaged over 10 simulation runs in order to show reliableresults. Figures 3(a) and 3(b) show a comparison betweenthe parameters produced from the simulation results and theones obtained from the GE model (the red curve reportsthe GE parameters). We observe that the adopted GE modelsuccessfully represents the realistically simulated LTE wirelessnetwork as described above. In particular, packet loss-patternsof the sent 3D video are generated for different numbers ofusers served in the LTE cell, i.e., 1, 2, 3, 4, 5, 6 3D users

TABLE ISIMULATION PARAMETERS FOR 3D VIDEO TRANSMISSION OVER LTE.

PARAMETERS VALUEBandwidth 10 MHzCell radius 1 kmE-UTRAN frequency band 1 (2.1 GHz)Max delay 100 ms3D video bit-rate 2.2 MbpsScheduler type M-LWDFVideo flow duration 20 secSimulation time 30 secPath loss/channel model Typical Urban (Pedestrian-A

propagation model)Simulation repetitions 10

(a) Simulation and model average packet loss ratios.

(b) Simulation and model average burst length.

Fig. 3. Simulation and GE model average packet loss ratios and average burstlengths for different numbers of users.

in the cell. The average packet loss ratio and average burstlength increase as the number of users increases in the cell, asseen in Figure 3. From the generated packet loss patterns, weare able to obtain the GE parameters, i.e., LB and PE . Thereported parameters in Table II are the GE parameters derivedfrom ten simulations and they represent the transmission ofthe 3D video for different numbers of users. The obtained GEparameters can be used by researchers to generate packet loss-patterns and then test their developed video quality evaluationmetrics on the impaired video sequences.

TABLE IIGE PARAMETERS DERIVED FROM 10 SIMULATIONS.

Numberof 3Dusers

MeanPacket

loss ratio(PE )

Meanburstlength(LB)

p =P0,1

q =P1,0

1 0.0248 3.094 0.0082 0.3232 0.0304 3.216 0.0097 0.3103 0.0437 4.195 0.0109 0.2384 0.0835 6.324 0.0144 0.1585 0.1627 8.156 0.0238 0.1226 0.2216 8.989 0.0317 0.111

B. 3D video encoding strategy and traffic model

The barrier, news-readers, football and umbrella 3D videosequences, snapshot in Figure 4 respectively, are importedfrom the NAMA3DS1-COSPAD1 database [19]. They arecompressed using the H.264/AVC compression standard witha fixed bit rate = 2.2 Mbps to obtain similar bit rates which canbe used over the modeled channel and maintain the quality. Aten-seconds sequence (250 frames at 25 fps) is considered forthe experiment. Simulcast encoding approach (i.e., two parallelencoders) is employed to encode both left and right image se-quences. They were encoded using the IPPP. . . IPPP. . . framesequence format to provide a high quality 3D video stream. AnI frame is encoded by every 1 second interval. The barrier andnews-readers sequences have 1920x1080 image resolution,whereas the football and umbrella sequences have 832x480image resolution. The maximum transmission unit is set to500 bytes as in [20].

(a) barrier. (b) news-readers.

(c) football. (d) umbrella.

Fig. 4. Snapshots of the 3D test sequences considered in the test.

The encoded 3D bit-streams are corrupted based on theerror traces generated by the GE model described in theprevious section. The corrupted H.264/AVC bit-streams arethen decoded after enabling error concealment (i.e., usingframe copying strategy). Error concealment allowed us toobtain a corrupted sequence with the same frame length asthe original video.

The 3D video flow used in the simulator in Table I is a trace-based application that sends packets based on realistic videotrace files. The left and right image sequences are merged intoa single trace file in a sequence order.

V. EVALUATION METHODOLOGY AND RESULTS

This section discusses the quality evaluation methods usedfor the proposed 3D video database and analyzes the obtainedresults.

In order to evaluate the true user perception, subjectivequality tests are performed using 15 naive subjects (as rec-ommended by ITU in [21]). Subjective quality experimentsare carried out in WMN Research Group LAB at KingstonUniversity-London, UK. The LG 47" HD-3D display withpolarized glasses is used for subjective quality experiments.The Absolute Category Rating with Hidden Reference (ACR-HR) method is employed to record subject’s opinions. Theimpaired sequences for subjective quality testing are chosento represent the average quality of several simulation runs.For instance, in case of six users in the cell, we choosethe sequence which has the closest quality to the averagequality of all the users. All subjects’ results are analyzedaccording to the Annex 2 (Analysis and presentation of results)of ITU-R BT.500-13 Recommendation [21]. Mean opinionscores and 95% confidence intervals (CIs) are calculated foreach of the presentation. Observer screening is carried outafter analyzing the reliability of individual opinion scores. Nooutliers were found for the tests considered. Figure 5 showsthe subjective results (i.e., MOS) distribution for all the testsequences considered in the test. This shows the distributionof subject scores over all the considered quality levels for theselected sequences for the database.

Fig. 5. MOS for all 3D test sequences.

Table III shows the MOS, the standard deviation and the95% confidence interval for the selected 3D test video se-quences and for all the sequences in average. Figure 6 showsthe calculated MOS scores for the perceived overall 3D imagequality of the news-readers sequence with different numbers ofusers in the cell. The 95% Confidence Intervals (CI) for MOSscores are also presented. The x-axis and y-axis respectivelyrepresent the packet loss rates for increasing number of usersin the cell, as reported in Table II and MOS scores from 1(bad) to 5 (excellent). It can be observed that, as expected, theperception of 3D video quality degrades as the number of usersin the cell increases for the news-readers sequence. Figure 7shows the MOS scores for the perceived overall 3D image

TABLE IIISTATISTICS RELATED TO SUBJECTIVE TEST SCORES.

Sequence umbrella barrierNumberof users

Original Oneuser

Twousers

Threeusers

Fourusers

Fiveusers

Sixusers

Original Oneuser

Twousers

Threeusers

Fourusers

Fiveusers

Sixusers

Mean 4.26 2.8 2.53 2.46 2.06 1.33 1.06 4.93 3.6 3.53 2.73 2.53 2.46 1.33STD 0.59 0.77 0.51 0.83 0.70 0.61 0.25 0.25 0.50 0.51 0.88 0.83 0.51 0.48

95% CI 0.30 0.39 0.26 0.42 0.35 0.31 0.13 0.13 0.25 0.26 0.44 0.42 0.26 0.24Sequence football news-readersNumberof users

Original Oneuser

Twousers

Threeusers

Fourusers

Fiveusers

Sixusers

Original Oneuser

Twousers

Threeusers

Fourusers

Fiveusers

Sixusers

Mean 4.53 3.33 3.26 2.2 2 1.86 1.06 4.8 3.53 3.4 2.26 1.53 1.4 1.06STD 0.51 0.97 0.70 0.67 0.75 0.63 0.25 0.41 0.51 0.63 0.70 0.63 0.63 0.25

95% CI 0.26 0.49 0.35 0.34 0.38 0.32 0.13 0.20 0.26 0.32 0.35 0.32 0.32 0.13

quality with different packet loss rates for all the sequencesconsidered. The 95% CIs for MOS scores are also presented.This figure also shows the negative trend in 3D perceptionwhen the number of users in the cell is increasing.

Fig. 6. MOS vs. PLR, for news-readers sequence.

Fig. 7. MOS vs. PLR, for all the sequences.

The quality degradations due to the wireless channels modelare also evaluated objectively using the PSNR and theStructural Similarity (SSIM) quality metrics. This is carriedout to see the lation between objective and subjective qualityresults. Table IV reports the average 2D PSNR and average

2D SSIM measurements across the six packet loss impairmentsrepresented as how many users are in the cell, including theaverage PSNR and average SSIM values for every originalencoded test sequence.

TABLE IVAVERAGE PSNR AND AVERAGE SSIM MEASUREMENTS FOR 3D TEST

SEQUENCES.

Average PSNR (dB)3D sequence Origi-

nalOneuser

Twousers

Threeusers

Fourusers

Fiveusers

Sixusers

barrier 34.39 32.74 32.42 30.53 25.20 25.10 22.05news-readers 38.00 36.82 35.08 31.04 26.19 24.90 22.98

football 35.02 32.46 30.73 26.74 24.08 23.00 20.00umbrella 31.15 29.48 28.41 26.76 24.34 23.37 20.00

3D sequence Average SSIMbarrier 0.89 0.88 0.88 0.86 0.83 0.82 0.78

news-readers 0.95 0.94 0.94 0.93 0.90 0.88 0.87football 0.93 0.92 0.90 0.86 0.78 0.79 0.66umbrella 0.84 0.83 0.82 0.81 0.77 0.76 0.61

In order to understand the relationship between subjectivescores for overall 3D image quality and objective qualitymeasures (PSNR), a four parameters logistic curve fittingmodel is used to show the objective quality ratings and theircorrelation to subjective results. The relationship between theperceived 3D video quality (i.e., MOS) and the objectivemeasures of distortion (i.e., average PSNR) is approximatedby the function, as illustrated in Figure 8. Extended resultswhich are not reported in this paper are available with thedatabase.

Figure 9 shows the surface plot of MOS score versusaverage burst length and PLR, for the umbrella, car, football,news-readers sequences.

VI. CONCLUSION

Video quality evaluation and transmission for bandwidthdemanding traffic like 3D video over LTE networks are vital.This paper presents a packet loss model for representingthe impact of transmitting video over LTE, and a 3D videodatabase designed to analyze the effect of wireless channelerrors on the perceived overall 3D video quality w.r.t differentpacket loss rates. MOS results are obtained using a seriesof subjective tests for four 3D test sequences and six packet

Fig. 8. MOS vs. average PSNR.

Fig. 9. Surface plot between MOS and the mean PLR and the mean Burstlength of the model for the umbrella, car, football, news-readers sequencesand the overall 3D image quality.

loss rates. The corresponding objective quality results are alsoreported in the paper and also available to download fromthe database. We aimed at deriving GE parameters which rep-resent real LTE network statistics and providing original andcorrupted 3D video sequences with their Quality of Experience(QoE) measurements as a benchmark for researchers to enablethem to test their 3D video quality evaluation algorithms underdifferent packet loss rates.

ACKNOWLEDGMENTS

The research leading to these results has received fundingfrom the European Union’s Seventh Framework Programme(FP7) under grant agreement No 288502 (CONCERTO).Moustafa Nasralla would like to thank Dr Sergio Pezzulli formentoring him with statistical analysis. We also would like tothank the European 3DConTourNet for the active support andcooperation.

REFERENCES

[1] 3GPP TR 21.902: Technical Specification Group Services and SystemAspects; Evolution of 3GPP system, 3GPP Std. TR 21.902, 2007.

[2] “Cisco visual networking index: Global mobile data traffic forecastupdate 2011-2016,” White Paper, Cisco, February 2012.

[3] E. N. Gilbert, “Capacity of a burst-noise channel,” Bell system technicaljournal, vol. 39, no. 5, pp. 1253–1265, 1960.

[4] E. Elliott, “Estimates of error rates for codes on burst-noise channels,”Bell system technical journal, vol. 42, no. 5, pp. 1977–1997, 1963.

[5] P. Ferré, D. Agrafiotis, T. K. Chiew, A. Doufexi, A. Nix, and D. Bull,“Packet loss modelling for H.264 video transmission over IEEE 802.11g wireless LANs,” in IEEE 13th International Workshop on ImageAnalysis for Multimedia Interactive Services, WIAMIS, 2005.

[6] L. Carvalho, J. Angeja, and A. Navarro, “A new packet loss model ofthe IEEE 802.11 g wireless network for multimedia communications,”IEEE Transactions on Consumer Electronics, vol. 51, no. 3, pp. 809–814, 2005.

[7] M. G. Martini and C. T. Hewage, “Flexible macroblock ordering forcontext-aware ultrasound video transmission over mobile WiMAX,”International journal of telemedicine and applications, pp. 1–14, 2010.

[8] C. T. Hewage, M. G. Martini, M. Brandas, and D. De Silva, “A studyon the perceived quality of 3D video subject to packet losses,” inIEEE International Conference on Communications Workshops (ICC),Budapest, 2013, pp. 662–666.

[9] F. De Simone, M. Naccari, M. Tagliasacchi, F. Dufaux, S. Tubaro, andT. Ebrahimi, “Subjective assessment of H.264/AVC video sequencestransmitted over a noisy channel,” in IEEE International Workshop onQuality of Multimedia Experience (QoMEx), San Diego, CA, 2009, pp.204–209.

[10] K. Piamrat, C. Viho, J. Bonnin, and A. Ksentini, “Quality of expe-rience measurements for video streaming over wireless networks,” inSixth IEEE International Conference on Information Technology: NewGenerations, Las Vegas, NV, 2009, pp. 1184–1189.

[11] G. Piro, C. Ceglie, D. Striccoli, and P. Camarda, “3D video transmissionsover LTE: A performance evaluation,” in IEEE EUROCON, Zagreb,2013, pp. 177–185.

[12] S. Sesia, I. Toufic, and M. Baker, “LTE - the UMTS long term evolution,”Wiley, UK, 2009.

[13] W. Tong, E. Sich, Z. Peiying, and J. Costa, “True broadband multimediaexperience,” IEEE Micro-wave Mag., vol. 9, no. 4, pp. 64–73, Aug.2008.

[14] S. Shakkottai, T. S. Rappaport, and P. C. Karlsson, “Cross-layer designfor wireless networks,” IEEE Commun. Mag., vol. 41, no. 10, pp. 74–80,2003.

[15] H. Ramli, R. Basukala, K. Sandrasegaran, and R. Patachaianand, “Per-formance of well known packet scheduling algorithms in the downlink3GPP LTE system,” in Proc. of IEEE Malaysia International Conf. onComm., (MICC), Kuala Lumpur, Malaysia, 2009, pp. 815–820.

[16] H. A. Sanneck and G. Carle, “Framework model for packet loss metricsbased on loss runlengths,” in Electronic Imaging. International Societyfor Optics and Photonics, 1999, pp. 177–187.

[17] M. G. Martini, C. T. Hewage, M. M. Nasralla, R. Smith, I. Jourdan, andT. Rockall, “3-D robotic tele-surgery and training over next generationwireless networks,” in 35th Annual International Conference of the IEEEEngineering in Medicine and Biology Society (EMBC), Osaka, 2013, pp.6244–6247.

[18] G. Piro, L. A. Grieco, G. Boggia, F. Capozzi, and P. Camarda, “Sim-ulating LTE cellular systems: an open source framework,” in IEEETransactions on Vehicular Technology, vol. 60, no. 2, Feb, 2011, pp.498–513.

[19] M. Urvoy, M. Barkowsky et al., “NAMA3DS1-COSPAD1: Subjectivevideo quality assessment database on coding conditions introducingfreely available high quality 3D stereoscopic sequences,” in 4th Interna-tional Workshop on Quality of Multimedia Experience (QoMEX), YarraValley, 2012, pp. 109 – 114.

[20] C. Hewage, M. Martini, and N. Khan, “3D medical video transmissionover 4G networks,” in 4th International Symposium on Applied Sciencesin Biomedical and Communication Technologies, Barcelona, Spain,October, 2011, pp. 26 – 29.

[21] International Telecommunication Union/ITU-Radio communication sec-tor, “Methodology for the subjective assessment of the quality oftelevision pictures,” in ITU-R BT.500-13, 2012.