![Page 1: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/1.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20121
Post-processing: Fighting Against Coding Artifacts
• Deblocking of DCT coded images– Image smoothing based approach– Wavelet-thresholding based approach
• Deringing of wavelet coded images– Re-compression approach– PDE-based approach
• Post-processing by alternating projections: a unified approach
![Page 2: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/2.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20122
Block Artifacts
Smooth areas become blocky JPEG decoded image at 0.23bpp
zoom
![Page 3: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/3.jpg)
Why block artifacts occur?
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20123
x
f(x)
original
x
f(x)^
B 2B 3B
JPEGdecodedat lowbit rate
Only DC component is preserved
![Page 4: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/4.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20124
Deblocking as Denoising
Standard image
denoisingalgorithm
JPEG compressed
imagepostprocessed
image
th 1000
Manually tune the threshold parameter!
![Page 5: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/5.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20125
Experiment Results
before deblocking after deblocking
![Page 6: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/6.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20126
Wavelet-based Deblocking
Behavior of block artifacts in wavelet domain
![Page 7: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/7.jpg)
Fundamental Issues behind Deblocking
• Motivation - modeling uncertainty– Location of block artifacts is known (block
boundaries)– How to distinguish significant coefficients
generated by artifacts from those associated with true edges
• Strategy– Recall how JPEG2000 is free from block artifacts
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20127
![Page 8: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/8.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20128
Deblocking via Wavelet Thresholding
WaveletTransform
BlockWavelet
Transform reorder
X
X
Y1
Y2
X: JPEG decoded image
![Page 9: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/9.jpg)
• Apply both WT and block-based WT to X to get Y1,Y2;
• Locate the coefficients at block boundaries;• If |Y1(i,j)|>T and |Y2(i,j)|<T, apply soft
thresholding to Y1(i,j);• Apply IWT to processed Y1 to obtain
deblocked image
EE5965 Advanced Image Processing Copyright Xin Li 2009-
20129
Deblocking Algorithm
![Page 10: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/10.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201210
Example
Before deblocking (PSNR=27.39dB)After deblocking (PSNR=28.07dB)
![Page 11: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/11.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201211
Ringing Artifacts
Sharp edges become unnatural JPEG2000 decoded image at 0.125bpp
zoom
![Page 12: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/12.jpg)
Why ringing artifacts occur?
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201212
x(n)
H1
Key observation: wavelet transform lacks translation invariance
2
x(n-1)
H1
2
origin origin
![Page 13: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/13.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201213
Deringing by Re-compression
JPEG: JPEG2000 encoder JPEG-1: JPEG2000 decoder
![Page 14: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/14.jpg)
Example
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201214
before processing after processing
![Page 15: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/15.jpg)
PDE-based Deringing
• The power of anisotropic diffusion– Nonlinear diffusion can handle a variety of
noise– Which PDE is suitable for deringing?– Implication into wavelet coding
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201215
![Page 16: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/16.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201216
Perona-Malik Filtering
PSNR=30.86dBPSNR=31.09dB
![Page 17: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/17.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201217
Mean Curvature Filtering
PSNR=31.09dB PSNR=30.27dB
![Page 18: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/18.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201218
Post-processing: Fighting Against Coding Artifacts
• Deblocking of DCT coded images– Image smoothing based approach– Wavelet-thresholding based approach
• Deringing of wavelet coded images– Re-compression approach– PDE-based approach
• Post-processing by alternating projections: a unified approach
![Page 19: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/19.jpg)
Recall: Alternating Projection
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201219
X0
X1
X2
X∞
Projection-Onto-Convex-Set (POCS) Theorem: If C1,…,Ck areconvex sets, then alternating projection P1,…,Pk will convergeto the intersection of C1,…,Ck if it is not empty Alternating projection does not always converge in the caseof non-convex set. Can you think of any counter-example?
C1
C2
![Page 20: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/20.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201220
Projection Operators
● Constraint set QTC
otherwisexTyxTyTyxTy
xPTfPTfP QQQT 2/2/2/2/
)()},({1
y y+T/2y-T/2
● Constraint set sC fffPs )(
xB xB
f Ps(f)
![Page 21: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/21.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201221
Projection-based Deblocking● DCT quantization set
}][|{},|{, gfQfCgTffCCCC QTQTQT
DCT Quantization● Smoothness constraint set
Cs={f|f is smooth in the block boundaries}
}||||{ 2 DffCs at block boundaries
Linear edge detection operatorYongyi Yang; Galatsanos, N.P.; Katsaggelos, A.K.; , "Projection-based spatially adaptive reconstruction of block-transform compressed images,“ IEEE Trans. on Image Proc., vol.4, no.7, pp.896-908, Jul 1995
![Page 22: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/22.jpg)
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201222
Projection-based Deringing
● WT quantization set
}][|{},|{, gfQfCgTffCCCC QTQTQT
WT Quantization● Smoothness constraint set
][,1, IcIcIcIcII WWEESSNN
tji
tji
Perona-Malik diffusion as a nonlinear projection operator
Xin Li; , "Improved wavelet decoding via set theoretic estimation," IEEE Trans. on CSVT, vol.15, no.1, pp. 108- 112, Jan. 2005
![Page 23: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/23.jpg)
Algorithm Flowchart
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201223
C1 : observation constraint set
C2 : regularization constraint set
![Page 24: EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bcb7f8b9ab0599d6f2d/html5/thumbnails/24.jpg)
Summary
• Connection with models (PDE-based, wavelet-based, patch-based)– They serve as image prior/regularization
constraint set– Jointly work with quantization (observation data)
constraint set• Convergence is NOT always guaranteed but
can be terminated strategically.
EE5965 Advanced Image Processing Copyright Xin Li 2009-
201224