“evalvid-ra” simulation of rate adaptive video

“Evalvid-RA”Simulation of rate adaptive video

TTM4142 Networked Multimedia Systems

Arne Lie, SINTEF ICT

November 6, 2008

6 Nov. 2008TTM4142:: Arne Lie, Evalvid-RA2

Overview

• Why Evalvid-RA• How to compress video• How to simulate video transmission• How to simulate rate adaptive video• Evalvid-RA architecture• How to use Evalvid-RA

Objectives

• Congestion control for media:: highest possible perceived quality!– Avoid persistent long queues

• low latency (media sender, network queues, media receiver)• low drop probability

– Bandwidth:• Fair bandwidth• Avoid unnecessary large rate reduction• Grab available excess bandwidth

• Network simulation of media:: requirements– Run traffic with the right characteristics

• Use source models, or• Use trace driven simulation (i.e. genuine video traffic)

– Perceived quality: need real media!• Evalvid tool-set• But we need “online” rate adaptive trace simulations

Example with congestion

Link capacity 15Mbps

Link utilization

MPEG-4 “Foreman” ~700kbps

~6Mbps

2s 4s 6s 8s 10s

The throughput using best effort Internet

Link capacity 15Mbps

Link utilization

MPEG-4 “Foreman” still ~700kbps

~4.8Mbps

2s 4s 6s 8s 10s

~6Mbps

Only 15/18.7=80.2% of the packets can survive after congestion takes place: 20% packet loss for all flows!

Or adapt the rate with 20%

Comparison of TailDrop, P-AQM and P-AQM with “ECF CC”

0 50 100 150 200 250 3005

frame number

TailDrop

P-AQM with ECF CC

0 5 10 15 200

7Bandwidth share tailDrop example

time [s]

0 5 10 15 200

time [s]

Bandwidth share P-AQM

0 5 10 15 200

time [s]

Bandwidth share P-AQM w/ECF CC

Main challenges

• Encoding/decoding of video is CPU demanding• We want to be able to simulate multiple video traffic

flows in mixed traffic scenarios on a single computer!– How to keep complexity low?

• We want to be able to play resulting video so that perceptual quality can be determined– How to avoid “online” encoding/decoding?

2005: What was available

• Trace driven simulation need trace files from real sources, e.g.– http://www-tkn.ee.tu-berlin.de/research/trace/ltvt.html

– Only the frame SIZES and timing is used, not the content

• or synthetic traffic that models real traffic very closely– e.g. GenSyn http://www.item.ntnu.no/~poulh/GenSyn/gensyn.html

• Evalvid tools from http://www.tkn.tu-berlin.de/research/evalvid/ – real traces, and media is re-assembled after network simulation for

visual inspection and PSNR calculation (Jirka Klaue)

• Evalvid interface to ns-2 (Ke Chih-Heng)– http://hpds.ee.ncku.edu.tw/~smallko/ns2/Evalvid_in_NS2.htm

• but rate adaptive media will change depending on network state…

Video coding

• Intra-frames (key-frames)– still images self-contained– used at scene changes

• Predicted-frames (P-frames)– uses motion-estimation

• Bidirectional frames (B-frames)– uses motion-estimation both forward and backward

in time– must be relative to an anchor picture (I- or P-frame)

Hybrid encoding: transform (spatial) + prediction (time)

Quantization of 8x8 pixel block

quantization steps (SQ)

after quantization performed

Motion vector, prediction error

• Source: Eckehard Steinbach: Internet Media Streaming

Scalable media

• change either (video / audio)– frame rate / sample rate (temporal)– frame size / sample size (spatial)– compression quantization Q (quality)

• =quantiser_scale in MPEG-4• or a combination• Most players/decoders don’t respond

(correctly) to changes in frame size and frame rate change of the Q-value

(=quantiser_scale) is easiest– the Q-value actually normally change

each frame, or even each macro block (video)

– but how to avoid doing this “live” in the network simulation?

Differences between live adaptation and pre-stored media with adaptation possibilities (scalable video coding)

Rate controllers varies Q

• Adjust output rate according to a bit rate budget on time average and variability constraints– leaky bucket

• CBR: constant bit rate– each GOP has the same number of bits (or bit/s)

– Q changes from macro block to macro block

– Cost: algorithmic delay, variable quality

• VBR: variable bit rate– allows for more variability

– Q changes less: more stable quality

• Quality based (“VBR open loop”, constant Q)– rate totally dependent on content

VBR open loop @ Q=2

0 200 400 600 800 1000 1200 1400 1600 1800 20000

frame number

s)I & P-frame sizes

I-frame

0 10 20 30 40 50 60 700

time (s)

Objective quality at Q=2: PSNR

• Constant Q gives ~ constant quality

200 400 600 800 1000 1200 1400 1600 1800 20000

frame number

Rate dependability on Q

0 10 20 30 40 50 60 700

time (s)

(bit/s

Q=3Q=4

2 4 6 8 10

Q-value

Aha_you_are.mov ffmpeg

Approximation f(q)

Rate controller objectives

• Limits the rate fluctuations & have an average rate constraint, by varying the quantization value Q– at each macro block– at each frame,– or at each GOP

• If Congestion Control is applied– the rate controller must have adaptable average rate

constraint!– Problem: the rate controller must run at simulation time!

No rate controller (VBR open loop)

• Bit rate too variable to control• Has Long Range Dependence (LRD)

Football

Stefan

Rate controller (VBR constrained)

0 200 400 600 800 1000 1200 1400 1600 1800 20000

frame No.

Concat MPEG, r=600kbit/s

0 20 40 60 80 100 120 140 1600

GOP No.

0 200 400 600 800 1000 1200 1400 1600 1800 20000

frame No.

Adaptive rate controller (VBR constrained)

• Red line: no adaptive rate control• Blue line: adaptive rate reduces the bit rate at ~40 second

0 50 100 150 200 250 300 350 4000

Inconvenient truth example

0 50 100 150 200 250 300 350 4000

10x 10

0 50 100 150 200 250 300 350 4000

time (s)

Quality of received video• PSNR of video flows examined (with delay constraints)

– P-AQM with highest score, and with Statistical Multiplexing Gain

– TFRC gains on running over networks with AQM

150ms delay constraint

TFRC 1: RED w/ ECN

TFRC 2: RED w/ dropping

TFRC 3: FIFO

32 videos @ 150 ms e2e delay constraint, 32 FTP, Web traffic

Original quality

TFRC supported adaptation P-AQM supported adaptation

(600 kbit/s)

400 kbit/s600 kbit/s400 kbit/s600 kbit/s

Adapt to 400 kbit/sAdapt to 400 kbit/s

32 video flows

How to avoid having “online” encoder to follow the adaptive feedback

• CBR changes Q at macro block granularity– too detailed for frame size trace files!

• VBR changes Q at frame or GOP granularity– Yes!

• “SVBR” (shaped VBR) by Hamdi/Roberts/Rolin ’97– change Q at GOP scale to constrain video to LB(r,b)

constraint• r: average video rate (=leaky bucket rate)• b: bucket size (to allow variability)

– very simple, no extra delay– my modification: variable r

Hamdi’s SVBR leaky bucket controller

encoder packetizer

calc. Q next GOP

Qto network

How to map r to Q

• rate x Q product almost independent on Q– dependent only on content complexity

at start of any new GOP, assuming complexity change smoothly from GOP to GOP

– r (bits/s) R(k+1) (bits/GOP) using a formula (PhD Thesis)– Q(k+1) = R(k)*Q(k)/R(k+1) (if video complexity does not change)– for stored media, next GOP complexity is known a priori

Pseudo code from rate adapt algorithm

Evalvid-RA solutions

• Multiple trace files– one per Q-value– Q=[2,3,4,…,31] (ffmpeg)

• make SVBR calculate Q(k+1): select GOP(k+1) trace • This requires fixed GOP sizes!• LB(r,b) parameters change at feedback event

– but the new Q-value is not used before start of next GOP

• Received video file must be assembled – using trace of actual Q(i)-values used, and – multiple *.m4v files

Long Range Dependence (LRD)

• Garrett & Willinger ’94: VBR video traffic is self-similar– the autocorrelation (k) function decays slowly at increasing

lag– makes buffer dimensioning & high link utilization very difficult– The cause of LRD: scene complexity changes!

• many papers on video characterization (GOP scale, frame scale)– very little related to what kind of rate controller in use!

• Hamdi showed in his thesis that– a stream satisfying a LB(r,b) constraint, where r equals the

traffic average rate, is not self similar

rate controllers limits the rate variance…

0 10 20 30 40 50 60 700

time (s)

Q=3Q=4

0 10 20 30 40 50 60 700

2.5x 10

time (s)

No apriori information

Apriori information

(k) of “concatenated” video (GOP)

• Positive correlations at lag k poses long bursts of time duration k

-20 0 20 40 60 80 100 120 140 160-0.4

Autocorrelation GOP scale

Apriori

Non-aprioriQ=8 open loop

-20 0 20 40 60 80 100 120 140 160-1

3.5x 10

10 Autocovariance GOP scale

Apriori

Non-aprioriQ=8 open loop

Evalvid-RA: overview

The tool-set overviewPre-process

• ffmpeg -s cif -r 30 -i video.yuv -vcodec mpeg4 -4mv -g 12 –flags sgop -sc_threshold 20000 -qscale 8 -s cif -r 30 -y video_Q12.m4v

• mp4.exe -send <IP address> <port #> <MTU> <fps> video_Q12.m4v > st_video_Q12.txt

ffmpeg.exe

Pre-process once (shell script)

Evalvidmp4.exe

video_Q*.m4v

Q=[2..31]

2, 3, …, 31

st_*.txt2, 3, …

30 MPEG-4 compressed video rate

variants

30 different possible frame traces:

[No. frame_size type]

*.yuv, *.mov, *.mp4, ...

Original video source e.g. video_orig.yuv

The tool-set overview (cont.)Network Simulation

• ns-2: evalvid_rateadapt.tcl – modified ns-2 interface to

“Evalvid-RA” & adaptive SVBR responding to P-AQM feedback

– Tcl init-function makes• video2.dat (frames all Q)

– read into memory

– used by all nodes sending the same media (different timing)

• gop_size.dat (GOP size all Q)– used by et_ra.exe

– sd_be_* stores e.g. actual Q used

ns-2 simulation

sd_be_5

sd_be_7 rd_be_8

rd_be_6

Actual frame traces used at packet level:

[Time packet_size type Q]

Actual packets received:[Time packet_size type or Loss]

· Tcl pre-process:· run through all st_*.txt input files· generate media matrix at frame

and GOP scale

video2.dat

gop_size.dat

The multi-rate trace file (multi Q)

• video2.dat:

The tool-set overview (cont.)Post-process

• et_ra.exe – modified Evalvid original et.exe

– Reads packet Tx and Rx trace files

– Finds used Q

– Reads video2.dat for frame sizes and types

– Reads gop_size.dat to assist assembling the resulting MPEG-4 file

• ffmpeg to decode to YUV• fixyuv_ra.exe: takes e2e delivery

time constraints into account resulting video file (*.yuv)

• psnr.exe: compare decoded YUV to original

Post-processes

et_ra sd_be_5 rd_be_6 video2.dat video_Q 2 31 gop_size.dat video_received.m4v

ffmpeg -i video_received.m4v -vcodec rawvideo video_received.yuv

fixyuv sd_be_5 rd_be_6 new_st.txt video_received.yuv fixed_packetloss.yuv

psnr 352 288 420 video_orig.yuv fixed_packetloss.yuv

et_ra.exe (Evaluate Trace, rate Adaptive)

• original et.exe:

• et_ra.exe:

frame 1frame 2frame 3frame 4frame 5

packet 1

packet 2

packet 3

frame 1:

Limitations of this implementation

• GOP time scale rate adaptation– Hamdi confirms that SVBR could be modified to frame scale

• Fixed GOP size– live encoders could start a new GOP (i.e. next frame

being I-frame) at a feedback event!– relaxation will make distortions?

• error concealment (packet loss)– FRAME mode vs. PACKET mode considerations– ffmpeg drops first frame after frame marked with “loss”

• No audio yet– limitation in mp4.exe tool

Research usage of Evalvid-RA

• Simulate many flows, coming from many sources, all of them rate adaptive

• Have different media sources, not only one • Wireless rate adaptive multimedia• Different congestion control algorithms

– Self-limited sources and their actual bandwidth– friendliness (towards TCP, UDP, DCCP, etc.)

• Different queuing systems (FIFO, AQMs, QoS/DiffServ e.g.)• Investigate the removal of LRD, or not?• trade latency for loss (short queues)• how to inject new flows• new initiatives for rate adaptation incentives• vary the sources rates• …

Evalvid-RA lab

• Software– Windows/Cygwin (Linux on Win32)– ns-2 w/ Evalvid-RA– ffmpeg & video inspection programs

• Script files• The pre-process is already performed• To do:

– Modify TCL script / run ns-2 simulation with selected parameters

– Run post-process and inspect video quality / statistics

“evalvid-ra” simulation of rate adaptive video

Documents

evalvid - unimore · 2010-12-13 · laboratorio di...

8,10 ra 9710 & ra 9262

adaptive skills adaptive partitions...

ra r ra r ra r r raa ra a r ra ra r ra rta a r r ra a r …

the determination of 224 ra, 226 ra, & 228 ra in water...

adaptive adaptive adaptive indexing

- ra - x pdf · 2008. 10. 20. · ra 011 ra 014ra 012 ra...

lecture 15: nonlinear adaptive systems - umu.se · lecture...

why adaptive (smart) antennas? - guc -...

ra international sustainability report 2019 · 6 ra...

ra 7164 vs ra 9173 - comparison

old.kgainfo.spb.ruold.kgainfo.spb.ru/img/flash/ppt/moskva/882_...

model thermostat dial ra-08mf / ra-08mef …model ra-08mf /...

1 “evalvid-ra” simulation of rate adaptive video ttm4142...

city of madera zoning map · 2018. 6. 6. · pz-ra pz-ra...

securities and exchange commission · social work (ra 4373)...

ra 7164 vs. ra 9173

ekladata.comekladata.com/jaimelespartitions.eklablog.com/perso/partitions/book... ·...

ra zeitschriften geographie - rvk.uni- · pdf fileafrica...

badminton ra ra ra