TTM4142 Networked Multimedia Systems

Stereo Analyses by Hybrid Recursive Matching (HRM) for Real-Time Immersive Video

Conferencing

Presentation by Leif Arne Rønningen, Item, NTNU, Autumn 2008

N Atzpadin, P Kauff, O SchreerIEEE Transactions on circuits and systems for video technology

Vol. 14, no 3, March 2004

Stereo correspondence search by HRM. The output disparity maps give the pixel correspondences of the left and right images

Two-camera model, rectification

The HRM algorithm - outline

HRM outline behaviour

• For the current image block posisiton do:• The selection-of-start-vector function evaluates three

candidate vectors for the current block posisiton, and outputs the vector with the smallest DBD (displaced block difference) as the start vector for pixel-recursion

• The pixel-recursion function outpus an update vector with the smallest DPD (displaced pixel difference)

• The selection-of-final-vector function selects as the final vector the one with the smallest DBD

Block Recursion

• The three candidate vectors are calculated using information of both the previous image and the spatial neighborhood

• Spatial candidate vectors are found using meander scan <link>

• Spatial candidates in the right image are equally distributed around the considered pixel in the left and right image

• A temporal candidate vector is taken from the same position in the previous frame <link>

Meander scan for arbitrarily shaped video objects

Even frames Odd frames

Shape driven displaced block difference - DBD

Pixel RecursionDisplaced pixel difference - DPD

Outline of pixel recursion

Mean absolute difference between candidateand final output vector

Frequency of identity between candidat andfinal output vector

Spatial distribution of squared difference betweenfinal output vector and candidate

Horizontal Vertical

Temporal Update

Spatial distribution of squared difference betweenfinal output vector and candidate

HRM applied to left and right images

Left-to-right Right-to-left disparities disparities

Result of the consistency check (unreliable disparities are marked as black).

Synthesized views (a) without postprocessing, (b) with consistency check and simple interpolation, and (c) with segmentation based postprocessing.

Zoom of the critical hand area in synthesized views from previous Figure(a) without postprocessing, (b) with consistency check and simple interpolation, and(c) with segmentation based postprocessing.