technical university of berlin communication systems group director: prof. thomas sikora carsten...
Post on 26-Mar-2015
217 Views
Preview:
TRANSCRIPT
Technical University of BerlinCommunication Systems Group
Director: Prof. Thomas Sikora
Carsten ClemensClemens@nue.tu-berlin.de
Error Concealment for Stereoscopic Sequences
ITG Fachausschusstagung 3.2, Juni 2006
Error Concealment for Stereoscopic Sequences 2Communication Systems GroupC. Clemens
Technische Universität Berlin
• Monoscopic Error Concealment strategies are not well suited for stereoscopic scenario
• Assumtions
– independently coded views of a stereo image pair
• remaining redundancies between the channels, which can be utilized for error concealment
– block based coding (16x16 blocks)
Introduction
Error Concealment for Stereoscopic Sequences 3Communication Systems GroupC. Clemens
Technische Universität Berlin
Algorithm Overview
Identification of corresponding region– feature extraction– feature matching along epipolar line– selection of matches (M-estimator/RANSAC)
Projective Transformation– initial parameter set from matches– optimization by Newton Method
Smoothing– only in case of discontinuities of depth
Error Concealment for Stereoscopic Sequences 4Communication Systems GroupC. Clemens
Technische Universität Berlin
Matching and Transformation
projective transformation:
Error Concealment for Stereoscopic Sequences 5Communication Systems GroupC. Clemens
Technische Universität Berlin
Selection of feature matches
• M-estimator– uses all matches with different weights– In some cases the transformation fails, because pixels from
outside the image were warped into the erroneous burst.
• RANSAC (random sample consensus)– uses a number of subsamples (four feature matches) – minimize the sum of squared residues of the boundary region:
RANSAC yields better results than M-estimator
Error Concealment for Stereoscopic Sequences 6Communication Systems GroupC. Clemens
Technische Universität Berlin
Adapted Newton Method
• Find the optimal transformation parameter by minimizing a cost function C(k):
• b is the Border Region of the erroneous block burst
T (k ;(xl, yl))
left view right view
b
Error Concealment for Stereoscopic Sequences 7Communication Systems GroupC. Clemens
Technische Universität Berlin
Adapted Newton Method
• Iteration step:
• Problem I: Local minimum solution
• Initial Parameter set is of prime importance
Cost function C(k) over horizontal and vertical translation parameter
Error Concealment for Stereoscopic Sequences 8Communication Systems GroupC. Clemens
Technische Universität Berlin
Adapted Newton Method
• Problem II: Convergence of Newton method
• Successivly decreasing of border pixel size L after every minimum search
Speed of convergence of the Newthon algorithm for different border sizes L
Error Concealment for Stereoscopic Sequences 9Communication Systems GroupC. Clemens
Technische Universität Berlin
3D Block Smoothing
• In case of great discontinuities in depth (variation of disparity) we perform a linear smoothing algorithm towards the surrounding pixel region (3D-BS)
• Minimization of the intersample variance between neighboring samples and to the block borders
Error Concealment for Stereoscopic Sequences 10Communication Systems GroupC. Clemens
Technische Universität Berlin
HQ EC: Results / Example
Error Concealment for Stereoscopic Sequences 11Communication Systems GroupC. Clemens
Technische Universität Berlin
Results / Subjective Evaluation
• Subjective Simulation Results:
– Double Stimulus Continuous Quality Scale Method (DSCQS) as phsychovisual test with 15 subjects
– Shutterglasses (StereoGraphics)
DSCQS method
-0,5
0,5
1,5
2,5
3,5
4,5
5,5
6,5
hall castle tower over all
monoscopic NEWTburst NEWT 3D-BS
DMOS
Error Concealment for Stereoscopic Sequences 12Communication Systems GroupC. Clemens
Technische Universität Berlin
Fast EC: Algorithm Overview
• Block Search– Directional Diamond Search– SAD
• Side Match Distortion ||1 out
jinjsm bb
Nd
Error Concealment for Stereoscopic Sequences 13Communication Systems GroupC. Clemens
Technische Universität Berlin
Fast EC: Matching Example „Hall“
• SMD determined for each possible position• Position with minimum SMD selected• Block used for reconstruction
Error Concealment for Stereoscopic Sequences 14Communication Systems GroupC. Clemens
Technische Universität Berlin
Fast EC: Subjective Evaluation
BALLOON DMOS S.D. conf. interval
Interpolation 5,69 1,19 ±0,65
SBS 4,39 1,12 ±0,62
TBS 2,52 2,64 ±1,45
TSBS 2,49 1,37 ±0,75
Error Concealment for Stereoscopic Sequences 15Communication Systems GroupC. Clemens
Technische Universität Berlin
Simple Matching - Example
• ‘Balloons’ 720x480, corrupted frame and features
Error Concealment for Stereoscopic Sequences 16Communication Systems GroupC. Clemens
Technische Universität Berlin
Simple Matching - Example
• Reference frame (temporal) with features
Error Concealment for Stereoscopic Sequences 17Communication Systems GroupC. Clemens
Technische Universität Berlin
Simple Matching - Example
• Reference blocks
Error Concealment for Stereoscopic Sequences 18Communication Systems GroupC. Clemens
Technische Universität Berlin
Simple Matching - Example
• Reconstructed frame
Error Concealment for Stereoscopic Sequences 19Communication Systems GroupC. Clemens
Technische Universität Berlin
Publications
• K. Günther, C. Clemens, and T. Sikora
A Fast Displacement-Estimation Based Approach For Stereoscopic Error Concealment
PCS 2004, San Francisco
• C. Clemens, M. Kunter, S. Knorr, and T. Sikora:
A hybrid approach for error concealment in stereoscopic images
WIAMIS '04, Lissabon
• M. Kunter, S. Knorr, C. Clemens, and T. Sikora:
A gradient based approach for stereoscopic error concealment
ICIP '04, Singapore
• S. Knorr, C. Clemens, M. Kunter, and T. SikoraRobust Concealment for Erroneous Block Bursts in Stereoscopic Images3D Data Processing, Visualization, and Transmission (3DPVT'04), Thessaloniki, Greece
• K. Günther, C. Clemens, and T. Sikora
A Fast Displacement-Estimation Based Approach For Stereoscopic Error Concealment
PCS 2004, San Francisco
• C. Clemens, M. Kunter, S. Knorr, and T. Sikora:
A hybrid approach for error concealment in stereoscopic images
WIAMIS '04, Lissabon
• M. Kunter, S. Knorr, C. Clemens, and T. Sikora:
A gradient based approach for stereoscopic error concealment
ICIP '04, Singapore
• S. Knorr, C. Clemens, M. Kunter, and T. SikoraRobust Concealment for Erroneous Block Bursts in Stereoscopic Images3D Data Processing, Visualization, and Transmission (3DPVT'04), Thessaloniki, Greece
top related