block loss recovery techniques for image communications jiho park, d-c park, robert j. marks, m....
TRANSCRIPT
Block Loss Recovery Techniques for Image Communications
Jiho Park, D-C Park, Robert J. Marks, M. El-Sharkawi
The Computational Intelligence Applications (CIA) Lab.
Department of Electrical Engineering
University of Washington
May 29, 2002
2
Projections based Block Recovery – Motivation
Conventional Algorithms use information of all surrounding area. Using only highly correlated area
3
Alternating Projections is projecting between two or more convex sets iteratively.
Alternating Projections
Converging to a common point
4
Projections based Block Recovery – Algorithm
2 Steps Pre Process : 1) Edge orientation detection
2) Surrounding vector extraction
3) Recovery vector extraction
Projections : 1) Projection operator P1
2) Projection operator P2
3) Projection operator P3
5
Edge orientation in the surrounding area(S) of a missing block(M). In order to extend the geometric structure to the missing block.
Simple line masks at every i, j coordinate in surrounding area(S) of the missing block(M) for edge detection.
Pre Process 1 –Edge Orientation Detection
121
121
121
vL
111
222
111
hL
Horizontal Line Mask Vertical Line Mask
6
Pre Process 1 – Edge Orientation Detection
Responses of the line masks at window W :
Total magnitude of responses :
Th > Tv ; Horizontal line dominating area
Th < Tv ; Vertical line dominating area
987
654
321
www
www
www
W987654321 w-w-w-w2w2w2w-w--w hR
987654321 w-w2w-w-w2w-w-w2-w vR
,||T S
hh R S
vv R ||T
7
Pre Process 2 – Surrounding Vectors
Surrounding Vectors, sk, are extracted from surrounding area of a missing block by N x N window.
Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, sk, is 8N.
}W),(),,(:{ jijixxks
8
Pre Process 3 – Recovery Vector Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the
edge orientation.
Recovery vectors consist of known pixels(white color) and missing pixels(gray color).
The number of recovery vectors, rk, is 2.
}W),(),,(:{ jijixxkr
Vertical line dominating area Horizontal line dominating area
9
Projections based Block Recovery –Projection operator P1
Recovery vectors, ri, for i = 1, 2
Surrounding vectors, sj , for j = 1 ~ 8N
Surrounding vectors, S, form a convex hull in N2-dimensional space
Recovery vectors, R, are orthogonally projected onto the line defined by the closest surrounding vector, si, j : Projection Operator P1.
10
Projections based Block Recovery –Projection operator P1
Projection operator P1 :
Convex hull (formed by surrounding vectors, containing information of local image structure)
11
Projections based Block Recovery –Projection operator P1
Surrounding vectors, sj , for j = 1 ~ 8N Recovery vectors, ri, for i = 1, 2
The closest vertex, sdi , from a recovery vector, ri.
or equivalently in DCT domain,
P1 :
Njiford jij
i 81,21||||minarg sr
Njiford jij
i 81,21||||minarg SR
21,||||
,)(
2
ii
i
d di
idiiP S
R
RSRS
12
Convex set C2 acts as an “identical middle”.
Projection operator P2 :
Projections based Block Recovery –Projection operator P2
otherwise
nforFFC
o
n
ff
ff
:
L: maxmin2
otherwise
nFforF
nFforF
P
n
n
n
n
f
f
f
f L
L
max,max
min,min
2
13
Convex set C3 acts as a convex constraint between missing pixels and adjacent known pixels, (fN-1 fN).
where,
and is a N x N recovery vector in
column vector form.
Projections based Block Recovery – Projection operator P3
fN-1 fN
}||:{3 EC n gg
)}(....,),{( ,,10,0,1 NNNNNN ffffg
}....,,,{ 21 Nffff
Projection operator P3 :
otherwise
nEforE
nEforE
P
mn
nmn
nmn
mn
,
,1
,1
,3 L,
L,
f
gf
gf
f
14
Projections based Block Recovery –Iterative Algorithm
Missing pixels in recovery vectors are restored by iterative algorithm of alternating projections :
N x N windows moving :
ii fPPPf 3211
Vertical line dominating area Horizontal line dominating area
15
Projections based Block Recovery - Summary
Edge Orientation Detection
Surrounding Vector Extraction
Recovery Vector Extraction
Projection Operator P1
Projection Operator P2
Projection Operator P3
Iteration=I?
All pixels?
16
Simulation Results –Lena, 8 x 8 block loss
Original Image Test Image
17
Simulation Results –Lena, 8 x 8 block loss
Ancis, PSNR = 28.68 dB Hemami, PSNR = 31.86 dB
18
Simulation Results –Lena, 8 x 8 block loss
Ziad, PSNR = 31.57 dB Proposed, PSNR = 34.65 dB
19
Simulation Results –Lena, 8 x 8 block loss
Ancis
PSNR = 28.68 dB
Hemami
PSNR = 31.86 dB
Ziad
PSNR = 31.57 dB
Proposed
PSNR = 34.65 dB
20
Simulation Results – Each StepLena 8 x 8 block loss
(a)
(b)
(c)
21
Simulation Results –Peppers, 8 x 8 block loss
Original Image Test Image
22
Simulation Results – Peppers, 8 x 8 block loss
Ancis, PSNR = 27.92 dB Hemami, PSNR = 31.83 dB
23
Simulation Results – Peppers, 8 x 8 block loss
Ziad, PSNR = 32.76 dB Proposed, PSNR = 34.20 dB
24
Simulation Results –Lena, 8 x one row block loss
Original Image Test Image
25
Simulation Results –Lena, 8 x one row block loss
Hemami, PSNR = 26.86 dB Proposed, PSNR = 30.18 dB
26
Simulation Results –Masquerade, 8 x one row block loss
Original Image Test Image
27
Simulation Results –Masquerade, 8 x one row block loss
Hemami, PSNR = 23.10 dB Proposed, PSNR = 25.09 dB
28
Simulation Results –Lena, 16 x 16 block loss
Original Image Test Image
29
Simulation Results –Lena, 16 x 16 block loss
Ziad, PSNR = 28.75 dB Proposed, PSNR = 32.70 dB
30
Simulation Results –Foreman, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 25.65 dB Proposed, PSNR = 30.34 dB
31
Simulation Results –Flower Garden, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 20.40 dB Proposed, PSNR = 22.62 dB
32
Simulation Results – Test Data and Error
512 x 512 “Lena”, “Masquerade”, “Peppers”, “Boat”, “Elaine”, “Couple”
176 x 144 “Foreman” 352 x 240 “Flower Garden”
8 x 8 pixel block loss 16 x 16 pixel block loss 8 x 8 consecutive block losses
Peak Signal – Noise – Ratio
)|),(ˆ),(|
255log(10
1 1
2
2
N
i
M
j
jixjix
MNPSNR
33
Simulation Results – PSNR (8 x 8)
Lena Masqrd Peppers Boat Elaine Couple
Ancis 28.68 25.47 27.92 26.33 29.84 28.24
Sun 29.99 27.25 29.97 27.36 30.95 28.45
Park 31.26 27.91 31.71 28.77 32.96 30.04
Hemami 31.86 27.65 31.83 29.36 32.07 30.31
Ziad 31.57 27.94 32.76 30.11 31.92 30.99
Proposed 34.65 29.87 34.20 30.78 34.63 31.49
34
Simulation Results – PSNR (Row, 16 x 16)
(16 x 16) Lena Foreman Garden
Ziad 28.75 25.65 20.40
Proposed 32.70 30.34 22.62
(8 x Row) Lena Maskrd Peppers Boat Elaine Couple
Hemami 26.86 23.10 25.41 24.54 26.87 24.30
Proposed 30.18 25.09 28.31 26.06 30.11 26.12