Compact Windows for Visual Correspondence via Minimum

Ratio Cycle Algorithm

Olga Veksler

NEC Labs America

Local Approach

• Look at one image patch at at time

• Solve many small problems independently

• Fast, sufficient for some problems

Global Approach

• Look at the whole image • Solve one large problem• Slower, more accurate

Local Approach

• Sufficient for some problems

• Central problem: window shape selection

• Efficiently solved using Minimum Ratio Cycle algorithm for graphs

stereo

left image right image disparity = x1-x2

Visual Correspondence

(x1,y) (x2,y)

motion

first image second image

vertical motion

horizontal motion(x1,y1) (x2,y2)

Local Approach [Levine’73]

left image right image

p

1C

1

2C

2

3C

3

++

+2 2

2 2=Common C

pd = i which gives best iC

Fixed Window Shape Problems

true disparities

fixed small window fixed large window

left image

nee

d d

iffe

ren

t w

ind

ow

sh

apes

• Two inefficient methods proposed previously 1. Local greedy search [Levine CGIP’73, Kanade’PAMI94]

2. Direct search [Intille ECCV94,Geiger IJCV95]

Variable Window: Previous Work

…….

• Need efficient optimization algorithm over sizes and shapes

Minimum Ratio Cycle

• G(V,E) and w(e), t(e): E R 0Ce

et

image pixels

• Find cycle C which minimizes:

WC

Ce

ew

Ce

et=

W

t

From Area to Cycle

blue edge

red edge+

5

-2sum up terms inside

using weights of edges

1

111

11

1

Window Cost

C(W) = =size of W+ … +

2 2

WC

Ce

ew

Ce

et

OK for any graphs

not OK for any graph 0Ce

et

positive negative

Compact Windows

p

simple graph cycles C

one-to-one

correspondencecompact windows W

we construct graph s.t. 0Ce

et

only clockwise cycles

Cycle which is not Simple

C Cin this case:

cycle C

cycle C

Solving MRC

• search for smallest s.t. there is negative cycle on graph with edge weights:

• negative cycle detection takes time due to the special structure of our graphs

find smallest s.t. for some cycle ew

et

ew et-

nnO

ew et -

• there are are compact windows, if the largest allowed window is n by n

• Contains all possible rectangles but much more general than just rectangles

• Find optimal window in in theory, linear ( ) in practice

• Search over in time

nO 2

4nO

3n 2n

perimeterarea

examples of compact windows

(small )

312O 231O

Sample Compact Windows

Speedup

for pixel p, the algorithm extends windows over pixels which are likely to have the same disparity as p

use the optimal window computed for p to approximate for pixels inside that window

Comparison to Fixed Window

Compact windows:16% errorstrue disparities

fixed small window: 33% errors fixed large window: 30% errors

motion

motion

Algorithm Tsukuba Venus Sawtooth Map

Layered 1.58 1.52 0.34 0.37Graph cuts 1.94 1.79 1.30 0.31Belief prop 1.15 1.00 0.98 0.84GC+occl. 1.27 2.79 0.36 1.79

Graph cuts 1.86 1.69 0.42 2.39Multiw. Cut 8.08 0.53 0.61 0.26

Comp. win. 3.36 1.67 1.61 0.33

Results

13 other algorithms, local and global

all

glo

bal

Running time: 8 to 22 seconds

Future Work

• Generalize the window class

• Generalize objective function – mean?– variance?