wavelet based image coding -...
TRANSCRIPT
ENEE631 Digital Image Processing (Spring'04)
Wavelet Based Image CodingWavelet Based Image Coding
Spring ’04 Instructor: Min Wu
ECE Department, Univ. of Maryland, College Park
� www.ajconline.umd.edu (select ENEE631 S’04)� [email protected]
Based on ENEE631 Spring’04Based on ENEE631 Spring’04
Section 11Section 11
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [2]
Overview and LogisticsOverview and Logistics
� Last Time:
– Transform coding
� Today:
– JPEG compression standard: Baseline block-DCT based algorithm
� lossy part: quantization with different step size for each coeff. Band
� lossless part: differential coding, run-length coding, Huffman
=> Continued with the class notes handed out in last lecture
– Subband and Wavelet based compression� Exploiting the structures between coefficients for removing redundancy
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [3]
Wavelet Transform for Image CompressionWavelet Transform for Image Compression
� ENEE631 emphasis
– Focus on conceptual aspects related to image compression
– Wavelet is also useful for denoising, enhancement, and image
analysis
– Build upon filterbank and subband coding from ENEE630
(For more in-depth info. on wavelet: wavelet course offered in Math Dept.)
� K-level 1-D wavelet/subband decomposition
– Successive lowpass/highpass filtering and downsampling� on different level: capture transitions of different frequency bands
� on the same level: capture transitions at different locations
UMCP ENEE631 Slides (created by M.W
u ©
2001/2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [4]
Successive Wavelet/Subband DecompositionSuccessive Wavelet/Subband Decomposition
Successive lowpass/highpass filtering and downsampling
� on different level: capture transitions of different frequency bands
� on the same level: capture transitions at different locations
Figure from Matlab Wavelet Toolbox Documentation
UMCP ENEE631 Slides (created by M.W
u ©
2001/2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [5]
Examples of 1Examples of 1--D Wavelet TransformD Wavelet Transform
From Matlab
Wavelet Toolbox
Documentation
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [6]
22--D Wavelet Transform via Separable FiltersD Wavelet Transform via Separable Filters
From Matlab Wavelet
Toolbox
Documentation
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [7]
UMCP ENEE631 Slides (created by M.W
u ©
2004)
22--D ExampleD Example
From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [8]
Subband Coding TechniquesSubband Coding Techniques
� General coding approach
– Allocate different bits for coeff. in different frequency bands
– Encode different bands separately
– Example: DCT-based JPEG and early wavelet coding
� Some difference between subband coding and early wavelet coding ~ Choices of filters
– Subband filters aims at (approx.) non-overlapping freq. response
– Wavelet filters has interpretations in terms of basis and typically designed for certain smoothness constraints
(=> will discuss more )
� Shortcomings of subband coding
– Difficult to determine optimal bit allocation for low bit rate applications
– Not easy to accommodate different bit rates with a single code stream
– Difficult to encode at an exact target rate
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [9]
Review: Filterbank & Multiresolution AnalysisReview: Filterbank & Multiresolution Analysis
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [10]
UMCP ENEE631 Slides (created by M.W
u ©
2004)
Smoothness Conditions on Wavelet FilterSmoothness Conditions on Wavelet Filter
– Ensure the low band coefficients obtained by recursive filtering
can provide a smooth approximation of the original signal
From M. Vetterli’s wavelet/filter-bank paper
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [11]
Embedded ZeroEmbedded Zero--Tree Wavelet Coding (EZW)Tree Wavelet Coding (EZW)
� “Modern” lossy wavelet coding exploits multi-resolution and self-similar nature of wavelet decomposition
– Energy is compacted into a small number of coeff.
– Significant coeff. tend to cluster at the same spatial location in
each frequency subband
� Two set of info. to code:
– Where are the significant
coefficients?
– What values are the significant
coefficients?
From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [12]
Key Concepts in EZW Key Concepts in EZW
� Parent-children relation among coeff.
– Each parent coeff at level k spatially correlates
with 4 coeff at level (k-1) of same orientation
– A coeff at lowest band correlates with 3 coeff.
� Coding significance map via zero-tree
– Encode only high energy coefficients� Need to send location info. � large overhead
– Encode “insignificance map” w/ zero-trees
� Successive approximation quantization
– Send most-significant-bits first and
gradually refine coeff. value
– “Embedded” nature of coded bit-stream� get higher fidelity image by adding extra refining bits
From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [13]
EZW Algorithm and ExampleEZW Algorithm and Example
� Initial threshold ~ 2 ^ floor(log2 xmax)
– Put all coeff. in dominant list
� Dominant Pass (“zig-zag” across bands)
– Assign symbol to each coeff. and entropy encode symbols� ps – positive significance� ns – negative significance� iz – isolated zero� ztr – zero-tree root
– Significant coeff.� move to subordinate list� put zero in dominant list
� Subordinate Pass
– Output one bit for subordinate list� According to position in up/low half of quantization interval
� Repeat with half threshold
– Until bit budget achieved
From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [14]
After 1After 1stst PassPassFrom Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [15]
After 2After 2ndnd PassPass
From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [16]
Beyond EZWBeyond EZW
� Cons of EZW
– Poor error resilience
– Difficult for selective spatial decoding
� SPIHT (Set Partitioning in Hierarchal Trees)
– Further improvement over EZW to remove redundancy
� EBCOT (Embedded Block Coding with Optimal Truncation)
– Used in JPEG 2000
– Address the shortcomings of EZW (random access, error resilience, …)
– Embedded wavelet coding in each block + bit-allocations among blocks
UMCP ENEE631 Slides (created by M.W
u ©
2001/2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [17]
JPEG 2000 Image Compression StandardJPEG 2000 Image Compression Standard
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [18]
From Christopoulos
(IEEE Trans. on CE
11/00)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [19]
JPEG 2000: A WaveletJPEG 2000: A Wavelet--Based New StandardBased New Standard
� Targets and features
– Excellent low bit rate performance without sacrifice performance at
higher bit rate
– Progressive decoding to allow from lossy to lossless
– Region-of-interest (ROI) coding
– Error resilience
� For details
– David Taubman: “High Performance Scalable Image Compression
with EBCOT”, IEEE Trans. On Image Proc, vol.9(7), 7/2000.
– JPEG2000 Tutorial by Skrodras @ IEEE Sig. Proc Magazine 9/2001
– Taubman’s book on JPEG 2000 (on library reserve)
– Links and tutorials @ http://www.jpeg.org/JPEG2000.htm
UMCP ENEE631 Slides (created by M.W
u ©
2001/2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [20]
ExamplesExamples
JPEG2KJPEG2Kvs. vs. JPEGJPEG
From Christopoulos
(IEEE Trans. on CE
11/00)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [21]
DCT vs. Wavelet: Which is Better?DCT vs. Wavelet: Which is Better?
� 3dB improvement?
– Wavelet compression was claimed to have 3dB improvement over DCT-based compression
– Comparison is done on JPEG Baseline
� Improvement not all due to transforms
– Main contribution from better rate allocation, advanced entropy coding, & smarter redundancy reduction via zero-tree
– DCT coder can be improved to decrease the gap
[Ref] "A comparative study of DCT- and wavelet-based image coding",Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang,IEEE Trans. on Circuits and Systems for Video Tech.,v.9, no.5, 8/99, pp692-695.
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [22]
A Close Look at Wavelet TransformA Close Look at Wavelet Transform
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [23]
Construction of Haar functionsConstruction of Haar functions
� Unique decomposition of integer k� (p, q)
– k = 0, …, N-1 with N = 2n, 0 <= p <= n-1
– q = 0, 1 (for p=0); 1 <= q <= 2p (for p>0)
e.g., k=0 k=1 k=2 k=3 k=4 …
(0,0) (0,1) (1,1) (1,2) (2,1) …
� hk(x) = h p,q(x) for x ∈ [0,1]
k = 2p + q – 1“reminder”
power of 2
∈
<≤−
<≤
==
∈==
]1,0[other for 0
22for 2
1
22
1for 2
1
)()(
]1,0[for 1
)()(
21
2/
21
2/
,
0,00
x
qx
q-
N
q-x
q-
N
xhxh
xN
xhxh
pp
p
pp
p
qpk
1
x
UMCP ENEE631 Slides (created by M.W
u ©
2001/2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [24]
Haar TransformHaar Transform
� Haar transform H
– Sample hk(x) at {m/N}� m = 0, …, N-1
– Real and orthogonal
– Transition at each scale p is
localized according to q
� Basis images of 2-D (separable) Haar transform
– Outer product of two basis vectors
−
−
−
−−−
−−−−
−−
2200
0022
0000
0000
0000
0000
2 2 00
0 0 2 2
2222
0000
1111
1111
0000
2222
1111
1111
8
1
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [25]
Compare Basis Images of DCT and HaarCompare Basis Images of DCT and Haar
See also: Jain’s Fig.5.2 pp136UMCP ENEE631 Slides (created by M.Wu © 2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [26]
Summary on Haar TransformSummary on Haar Transform
� Two major sub-operations
– Scaling captures info. at different frequencies
– Translation captures info. at different locations
� Can be represented by filtering and downsampling
� Relatively poor energy compaction
1
x
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [27]
Orthonormal FiltersOrthonormal Filters
� Equiv. to projecting input signal to orthonormal basis
� Energy preservation property
– Convenient for quantizer design� MSE by transform domain quantizer is same as reconstruction MSE
� Shortcomings: “coefficient expansion”
– Linear filtering with N-element input & M-element filter
� (N+M-1)-element output � (N+M)/2 after downsample
– Length of output per stage grows ~ undesirable for compression
� Solutions to coefficient expansion
– Symmetrically extended input (circular convolution) &Symmetric filter
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [28]
Solutions to Coefficient ExpansionSolutions to Coefficient Expansion
� Circular convolution in place of linear convolution
– Periodic extension of input signal
– Problem: artifacts by large discontinuity at borders
� Symmetric extension of input
– Reduce border artifacts (note the signal length doubled with symmetry)
– Problem: output at each stage may not be symmetric From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [29]
Solutions to Coefficient Expansion (cont’d)Solutions to Coefficient Expansion (cont’d)
� Symmetric extension + symmetric filters
– No coefficient expansion and little artifacts
– Symmetric filter (or asymmetric filter) => “linear phase filters”
(no phase distortion except by delays)
� Problem
– Only one set of linear phase filters for real FIR orthogonal wavelets
�Haar filters: (1, 1) & (1,-1)
do not give good energy compaction
UMCP ENEE631 Slides (created by M.W
u ©
2001)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [30]
Biorthogonal WaveletsBiorthogonal Wavelets
� “Biorthogonal”
– Basis in forward and inverse transf.are not the same but give overall perfect reconstruction (PR)� recall EE624 PR filterbank
– No strict orthogonality for transf. filters so energy is not preserved� But could be close to orthogonal filters’ performance
� Advantage
– Covers a much broader class of filters� including symmetric filters that eliminate coefficient expansion
� Commonly used filters for compression
– 9/7 biorthogonal symmetric filter– Efficient implementation: Lifting (see Swelden’s tuotial)
UMCP ENEE631 Slides (created by M.W
u ©
2001)
From Usevitch (IEEE Sig.Proc. Mag. 9/01)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [31]
Review: Perfect Reconstruction Filter Bank Review: Perfect Reconstruction Filter Bank
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [32]
More on Wavelets (1)More on Wavelets (1)
UMCP ENEE631 Slides (created by M.W
u ©
2004) � Linear expansion of a
function via an expansion set
– Basis function – if the
expansion is unique
� Orthogonal basis
� Non-orthogonal basis
– Coefficients are computed with
a set of dual-basis
� Discrete Wavelet Transform
– Wavelet expansion gives a set
of 2-parameter basis functions
and expansion coefficients:
scale and translation
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [33]
More on Wavelets (2)More on Wavelets (2)
UMCP ENEE631 Slides (created by M.W
u ©
2004)
� 1st-generation wavelet systems:
– Scaling and translation of a generating wavelet (“mother wavelet”)
� Multiresolution conditions:
– Use a set of basic expansion signals with half width and translated in half step size to represent a larger class of signals than the original expansion set (the “scaling function”)
� Represent a signal by combining scaling functions and wavelets
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [34]
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [35]
Summary of Compression ToolsSummary of Compression Tools
� Lossless compression tools
– Entropy coding� Huffman, Arithmetic, Lempel-Ziv
� run-length
– Predictive coding� reduce the dynamic range to code
– Transform� enhance energy compaction
� Lossy compression tools
– Discarding and thresholding
– Quantization� ( previous lectures ) Scalar quantizer
� Quantize more than one component => vector quantization
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [36]
Summary of Today’s LectureSummary of Today’s Lecture
� DCT-based coding and JPEG
� Wavelet-based coding– Exploring tree-based structure in Wavelet coefficients: EZW– JPEG 2000
� Next lecture (after spring break):– More on wavelet transform/filters and JPEG 2000– Bit allocation techniques in image compression
� Readings on Wavelet
– Gonzalez book: 8.5.3, 8.6.2
– “A Tutorial on Modern Lossy Wavelet Image Compression” by Usevitch @ IEEE Sig. Proc Magazine 9/2001
– See also reference list provided with project#1
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [37]
AssignmentAssignment
� Required readings
– [Wavelet coding] � Usevitch’s tutorial in IEEE Sig. Proc. Magazine 9/01 � Xiong’s paper on DCT vs. Wavelet on IEEE Trans. CSVT (Univ. Library’s E-journal)� MATLAB 2-D Wavelet Demos
– [JPEG2000] Christopoluos’ tutorial in IEEE Trans. CE 11/00� http://etro.vub.ac.be/~chchrist/paper_ieee_ce_jpeg2000_Nov2000.pdf
– [VQ] Bovik’s Handbook Sec.5.3
� Recommended readings
– www.jpeg.org
– Gray’s tutorial paper on VQ
– Bovik’s Handbook Sec.5.4
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [38]
Summary of Today’s LectureSummary of Today’s Lecture
� Wavelet-based coding
– Exploring tree-based structure in Wavelet coefficients: EZW
– JPEG 2000
– Wavelet transform/filters
� Next lecture :
– More on wavelet filters
– Bit allocation techniques in image compression
� Readings on Wavelet
– Gonzalez book: 7.1.1-7.1.2, 8.5.3, 8.6.2
– “A Tutorial on Modern Lossy Wavelet Image Compression” by Usevitch @ IEEE Sig. Proc Magazine 9/2001
– Taubman: “JPEG 2000”, Proc. of IEEE,
– See also reference list provided with project#1
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [39]
Summary of Today’s LectureSummary of Today’s Lecture
� Wavelet filters
� Bit allocation techniques in image compression
– Rate-distortion functions and reverse water-filling solution
� Next Lecture:
– Optimal transform
� Readings
– Wavelets and JPEG2000: reference list provided in project#1� See also Gonzalez’s book Section 7.1, (7.2-7.3), 8.5.3, 8.6.2
– “Wavelets and the lifting scheme: A 5 minute tour” by Wim Sweldens
[Zeitschrift für Angewandte Mathematik und Mechanik, 76 (2), pp. 41-44, 1996]
– Results on rate-distortion theory: Jain’s book Section 2.13
� Further exploration – Tutorial on rate-distortion by Ortega-Ramchandran in IEEE Sig. Proc. Magazine, Nov. 98
UMCP ENEE631 Slides (created by M.W
u ©
2004)
ENEE631 Digital Image Processing (Spring'04) Lec14 –Wavelet Coding [40]
Question for Today: Bring in Motion Question for Today: Bring in Motion �������� VideoVideo
� Capturing video
– Frame by frame => image sequence
– Image sequence: A 3-D signal
� 2 spatial dimensions & time dimension
� continuous I( x, y, t ) => discrete I( m, n, tk )
� Encode digital video
– Simplest way ~ compress each frame image individually
� e.g., “motion-JPEG”
� only spatial redundancy is explored and reduced
– How about temporal redundancy?� Will differential coding work well? U
MCP ENEE408G Slides (created by M.W
u &
R.Liu © 2002)