![Page 1: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/1.jpg)
MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding
Authors from: University of GeorgiaSpeaker: Chang-Kuan Lin
![Page 2: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/2.jpg)
2
Reference
S. Chattopadhyay, S. M. Bhandarkar, K. Li, “FGS-MR: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding,” ACM NOSSDAV 2006.
W. Li, “Overview of Fine Granularity Scalability in MPEG-4 Video Standard,” IEEE Trans. on Circuits and Systems for Video Technology, Vol. 11, No. 3, pp. 301-317, Mar. 2001.
H. Radha, M. van der Schaar, and Y. Chen, “The MPEG-4 fine-grained scalable video coding method for multimedia streaming over IP,” IEEE Trans. on Multimedia, vol.3, pp. 53–68, Mar. 2001.
![Page 3: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/3.jpg)
3
Outline
Introduction MPEG-4 Fine Grained Scalability Motivation
FGS-AQ vs. FGS-MR Experimental Results Conclusion
![Page 4: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/4.jpg)
4
Introduction MPEG4 Fine Grained Scalability (FGS) profile for
streaming video Base Layer Bit Stream
must exist at the decoder has coarsely quantized DCT coefficients provides the minimum video quality
Enhancement Layer Bit Stream can be absent at the decoder contains encoded DCT coefficient differences provides higher quality can be truncated to fit the target bit rate
![Page 5: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/5.jpg)
5
FGS Encoding Block Diagram
![Page 6: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/6.jpg)
6
Motivation Base Layer video quality is usually not
satisfactory in order to provide a wide range of bit rate adaptation
MPEG4 FGS Adaptive Quantization (FGS-AQ) for Base Layer video does not provide good rate-distortion (R-D) performance parameter overhead at the decoder
Proposed FGS-MR no parameter overhead to transmit transparent the codec better rate-distortion performance
![Page 7: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/7.jpg)
7
Outline Introduction
MPEG-4 Fine Grained Scalability Motivation
FGS-AQ vs. FGS-MR FGS-AQ FGS-MR
MR-Mask Creation MR-Frame
Experimental Results Conclusion
![Page 8: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/8.jpg)
8
FGS Adaptive Quantization (AQ) Goals
To improve visual quality To better utilize the available bandwidth
Method Define different quantization step sizes for differen
t transform coefficients within a macro-block (low freq. DCT coeff. => small step
size) for different macro-blocks (different quantization factors)
Disadvantages R-D performance degrades due to FGS-AQ param
eter overhead
![Page 9: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/9.jpg)
9
Proposed Multi-Resolution FGS (FGS-MR)
Goal To improve the visual quality To better utilize the available bandwidth No transmission overhead and hence maintaining
the R-D performance Method
Apply a low-pass filter on “visually unimportant” portion of the original video frame before encoding.
![Page 10: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/10.jpg)
10
Two Equivalent Operations
Apply a low-pass filter on the spatial domain of an image
Truncate DCT coefficients in the corresponding transform domain of an image
![Page 11: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/11.jpg)
11
FGS-MR Process (Step 1)
MR-Mask creation Use Canny edge detector to detect edges Weight Mask
an weight parameter wi, j for each pixel p(i, j) of an image, 0 ≦ wi, j 1≦
wi, j = 1, if p(i, j) is on the edge 0 < wi, j 1, if ≦ p(i, j) is near edge wi, j = 0, if p(i, j) is in non-edge region
![Page 12: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/12.jpg)
12
Original (5.12Mbps)
![Page 13: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/13.jpg)
13
MR-Mask
![Page 14: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/14.jpg)
14
FGS-MR Process (Step 2)
MR-Frame Creation VI = (I-W) VL +W VH
VF = Iteration( VI, G(σI))
Note VI contains abrupt changes i
n resolution VF is a smooth version of VI
Parameters Vo: original video frame VL : low resolution frame from the co
nvolution of Vo and G(σL) VH : high resolution frame from the c
onvolution of Vo and G(σH) VI : intermediate video frame VF : final multi-resolution frame I: matrix with all entries as 1 W: MR-mask weight matrix G(σ): Gaussian filter with standard d
eviation of σas LPF σL >σH
![Page 15: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/15.jpg)
15
Original (5.12Mbps)
![Page 16: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/16.jpg)
16
FGS-AQ (0.17Mbps, PSNR = 22.77dB)
![Page 17: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/17.jpg)
17
FGS-MR (0.17Mbps, PSNR = 26.5dB)
![Page 18: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/18.jpg)
18
Determine Parameters σL, σH, and σI
to control the bit rate W (weight matrix)
to control the quality of the encoded video frame Figure of merit function: δ=Q/C
Q = 2^( PSNR(σL, σH, σI)/10 ) or PSNR = 10log(Q) C: compression ratio
The authors empirically determine the parameters σL = 15, σL = 3, and varying σI
![Page 19: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/19.jpg)
19
Outline Introduction
MPEG-4 Fine Grained Scalability Motivation
FGS-AQ vs. FGS-MR FGS-AQ FGS-MR
Experimental Results Rate Distortion Resource Consumption
Conclusion
![Page 20: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/20.jpg)
20
Experiments
Video 1 320x240, fps = 30 A single person walking in a well lighted room
Video 2 176x144, fps = 30 A panning view across a poorly lighted room. No moving object
![Page 21: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/21.jpg)
21
Rate Distortion Performance
Vary σI from 3 to 25 to adjust the target bit rate
![Page 22: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/22.jpg)
22
Power Consumption
Energy used and hence power consumed by wireless network interface card (WNIC):
T: time durationS: data sizeb: the bit rate of streaming videoB: available BWER: energy used by WNIC during data reception
Es: energy used by WNIC when sleeping
![Page 23: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/23.jpg)
23
Power Consumption Comparison
![Page 24: MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding Authors from: University of Georgia Speaker: Chang-Kuan Lin](https://reader035.vdocuments.us/reader035/viewer/2022062523/5a4d1ad07f8b9ab0599710ac/html5/thumbnails/24.jpg)
24
Conclusion
The rate distortion performance of FGS-MR is better than FGS-AQ.
FGS-MR can be seamlessly integrated into existing MPEG4 codec.
My comment Processing time issue of FGS-MR Empirical determined filter parameters