a hardware-friendly wavelet entropy codec for scalable video hendrik eeckhaut elis-paris ghent...
TRANSCRIPT
![Page 1: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/1.jpg)
A hardware-Friendly Wavelet Entropy Codec for Scalable video
Hendrik EeckhautELIS-PARIS
Ghent UniversityBelgium
![Page 2: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/2.jpg)
2
Scalable video
Encodeonce
Decode according torequired QoS or
available hardware resources.
Quality ~ deployed hardware resources
![Page 3: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/3.jpg)
3
Overview• Scalable video codec
• Scalable Wavelet Entropy Encoding :• New algorithm
• Memory footprint
• Compression results
• Conclusions
![Page 4: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/4.jpg)
4
Overview video codec
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
originalframes
Exploittemporal
redundancy
Exploitspatial
redundancy
Exploitstatisticalproperties
![Page 5: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/5.jpg)
5
Motion estimation
vectors + error frame
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
![Page 6: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/6.jpg)
6
Motion compensationReference framecurrent GOP
Reference framenext GOP
timeScalability in time
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
![Page 7: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/7.jpg)
7
Wavelet transform
Scalability in resolution
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
![Page 8: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/8.jpg)
8
Wavelet Entropy Coder
• Scalability:• Quality
• Resolution
• Good compression
• Economical with memory
• High degree of parallelism
• Sweet and simple
Wavelet Entropy Codec
• Actual compression of frames• Tight coupling of pixel characterization and
Arithmetic coding (modified arithmetic encoder of CABAC)
ModelSelector
ArithmeticEncoder
Entropy Encoding
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
![Page 9: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/9.jpg)
9
Scalability in quality
Scalability in quality
![Page 10: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/10.jpg)
10
Parallelism
In parallelinParallel
Easy to omit subbands → resolution scalability
![Page 11: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/11.jpg)
11
Algorithm
Bitplane
Significance bitmap
Sign bitmap
Contextcharacterization Model Selection
Aritmetic Encoder
0.3 0.7
0.4 0.6
0.45 0.55
0.5 0.5
0.6 0.4
0.7 0.3
Significance bitmap
Sign bitmap
Current pixel
0.3 0.70.80.2
![Page 12: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/12.jpg)
12
Algorithm
Bitplane
Significance bitmap
Sign bitmap
![Page 13: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/13.jpg)
13
Compression gain
Context/Arithmetic models64 different models: each model needs only a 9 bit state
• 1 data model
• 27 sign models
• 27 significance models
• 8 special highest bitlayer significance models
• 1 refinement model
64 for each type of subband:480 subband models
[LL,HL,LH,HH] [resolution layer][color channel] [temporal level]
~> 30.720 different models in total
0
1
1/2
real probability
time
unitialised
trained probability
• Models are initialised (warmed up)with values obtained from training a number of representative sequences
![Page 14: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/14.jpg)
14
Memory requirements
• Arithmetic Decoder:• State Lookup table: 256B
• Range Lookup table: 2048B
• Model state table: 64B
• Buffer
• Model Selector• Significance and sign bitmap: 2 x 3168B
• A few buffers• (1 MRAM block is large enough to contain multiple
significance and sign bitmaps)
Arithmetic DecoderModel selector
Name Number Size (byte + parity)MRAM 2 64kM4K 138 512M512 224 64
Registers 25660 1/8214kiB
Altera Stratix S25
~7 M4K block
~13 M4K blocks
![Page 15: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/15.jpg)
15
Results
0 5000 10000 15000 20000 250000
5
10
15
20
25
30
35
40
45
50
55
60
Average PSNRMobile CIF, 289 frames, 30Hz
WEEQTL
bitrate (kibps)
PS
NR
(d
B)
QTL: QuadTree Limited (comparable compression as EBCOT of jpeg2000)
![Page 16: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/16.jpg)
16
Conclusion• Algorithm is simple
• No recursive behaviour
• Small memory footprint[ 2 bitmaps of size(wavelet subband) ]
• Memory access is very regular
• Arithmetic coder must process a lot symbols, but is simple and fast
• Compression is better than QTL-algorithm for all bitrates
TODO: We are working hard on aneffective FPGA-implementation of the decoder
![Page 17: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/17.jpg)
17
![Page 18: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/18.jpg)
18
Arithmetic Coding
Contextcharacterization Model Selection
Aritmetic Encoder
0.3 0.7
0.4 0.6
0.45 0.55
0.5 0.5
0.6 0.4
0.7 0.3
Significance bitmap
Sign bitmap
Current pixel
0.3 0.70.80.2
![Page 19: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/19.jpg)
19
Scalable video (animated)
Encodeonce
Decode according torequired QoS or
available hardware resources.
Quality ~ deployed hardware resources
![Page 20: A hardware-Friendly Wavelet Entropy Codec for Scalable video Hendrik Eeckhaut ELIS-PARIS Ghent University Belgium](https://reader033.vdocuments.us/reader033/viewer/2022051401/56649ef65503460f94c098f4/html5/thumbnails/20.jpg)
20
Overview video codec
MotionEstimation P
ack
EntropyEncoding
WaveletTransform
Motion VectorEncoding
MotionComp.
EntropyDecoding
Motion VectorDecoding
Unpack
Pullbitstream
InverseWavelet T.
originalframes
decodedframes