an experimental comparison of min-cut/max-flow algorithms for energy minimization in vision

1

AN EXPERIMENTAL COMPARISON OFMIN-CUT/MAX-FLOW ALGORITHMS

FORENERGY MINIMIZATION IN VISIONIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE

INTELLIGENCE, VOL. 26, NO. 9, SEPTEMBER 2004

Yuri Boykov, Member, IEEE Vladimir Kolmogorov, Member, IEEE

Guan-Yu Liu

2

Outline Introduction Overview Method

A : growth B : augmentation C : adoption

Experimental Results Part1 - introduce algorithm Part2 - results

Conclusion Reference Q & A

3

Introduction(1/3)

GREIG et al. [15] were the first to discover that powerful min-cut/max-flow algorithms from combinatorial optimization can be used to minimize certain important energy functions in vision.

123

where L = is a labeling of image , is a data penalty function, is an interaction potential, and is a set of all pairs of neighboring pixels.

[15] D. Greig, B. Porteous, and A. Seheult, “Exact MaximumA Posteriori Estimation for Binary Images,” J. Royal StatisticalSoc., Series B, vol. 51, no. 2, pp. 271-279, 1989.

4

Introduction(2/3)

Algorithm Dinic’s algorithm [14] Push-Relabel algorithm with the queue-based selection rule [12] Push-Relabel algorithm with the highest level selection rule [12]

[12] A.V. Goldberg and R.E. Tarjan, “A New Approach to theMaximum-Flow Problem,” J. ACM, vol. 35, no. 4, pp. 921-940,Oct. 1988.

[14] E.A. Dinic, “Algorithm for Solution of a Problem of Maximum Flow in Networks with Power Estimation,” Soviet Math. Dokl., vol. 11, pp. 1277-1280, 1970.

5

Introduction(3/3)

This paper use Min-Cut/Max-Flow to find minimum energy. Image Restoration

○ Potts model○ Linear interaction

Stereo○ Pixel-Labeling stereo with the Potts model○ Stereo with occlusion○ Multi-camera scene reconstruction

Segmentation

6

Overview(1/9)

A

B

C

D

9

92

2

7

Overview(2/9)

A

B

C

D

9

92

2

S T

Source Sink

8

Overview(3/9)

A

B

C

D

9

9

2

2

S T

2

2

9

Overview(4/9)

A

B

C

D

2/9

2/92/2

2/2

S T

10

Overview(5/9)

There are two basic way to search max-flow Breadth-first search(BFS) Push

A typical augmenting path algorithm stores information about the distribution of the current s→t flow f among the edges of G using a residual graph Gf. The topology of Gf is identical to G, but the capacity of an edge in Gf reflects the residual capacity of the same edge in G given the amount of flow already in the edge.

11

Overview(6/9)

A

B

C

D

9

92/2

2/2

S T2/2

12

Overview(7/9)

A

B

C

D

9

92/2

2/2

S T

2/22

22

13

Overview(8/9)

A

B

C

D

2/9

2/92/2

2/2

S T

2/22

2/22

14

Overview(9/9)

There are three steps on the proposed algorithm. A : growth B : augmentation C : adoption

15

Method A : growth

search trees S and T grow until they touch giving an s→t path

B : augmentation the found path is augmented, search tree(s) break into forest(s)

C : adoption trees S and T are restored.

16

Method.A(1/3)

At the growth stage, the search trees(s and t) expand. The active nodes explore adjacent nonsaturated edges and acquire new children from a set of free nodes. Active node Passive node Free node

17

Method.A(2/3)

The growth stage terminates if an active node encounters a neighboring node that belongs to the opposite tree.

18

Method.A(3/3)

19

Method A : growth

search trees S and T grow until they touch giving an s→t path

B : augmentation the found path is augmented, search tree(s) break into forest(s)

C : adoption trees S and T are restored.

20

Method.B(1/3)

The augmentation stage augments the path found at the growth stage. Since we push through the largest flow possible, some edge(s) in the path become saturated. Orphan node

21

Method.B(2/3)

22

Method.B(3/3)

The nodes that become orphan nodes keep their original own parents. Change another parent Become free nodes

23

Method A : growth

search trees S and T grow until they touch giving an s→t path

B : augmentation the found path is augmented, search tree(s) break into forest(s)

C : adoption trees S and T are restored.

24

Method.C(1/3)

The goal of the adoption stage is to restore the single-tree structure of sets S and T with roots in the source and the sink. Try to find a new valid parent for each orphan. A new parent should belong to the same set, S or T, as the orphan. If there is no qualifying parent, we remove the orphan from S or T and

make it a free node. Also declare all its former children orphans.

25

Method.C(2/3)

26

Method.C(3/3)

The stage terminates when no orphans are left and, thus, the search tree structures of S and T are restored. Since some orphan nodes in S and T may become free, the adoption stage results in contraction of these sets.

27

Method.conclusion

After the adoption stage is completed, the algorithm returns to the growth stage. The algorithm terminates when the search trees S and T cannot grow (no active nodes) and the trees are separated by saturated edges. This implies that a maximum flow is achieved.

28

Experimental Results.part1

A : Dinic’s algorithm

B : Push-Relabel algorithm with the queue-based

C : Push-Relabel algorithm with the highest level

29

30

Experimental Results.part1

A : Dinic’s algorithm

B : Push-Relabel algorithm with the queue-based

C : Push-Relabel algorithm with the highest level

31

Experimental Results.B(1/1)

To push water from source to sink. Every node can store water temporally. Terminate when bottleneck.

32

Experimental Results.part1

A : Dinic’s algorithm

B : Push-Relabel algorithm with the queue-based

C : Push-Relabel algorithm with the highest level

33

Experimental Results.C(1/1)

Improved queue-based Set height levels to avoid water flow back

34

Experimental Results.part2

A : Image Restoration Potts model Linear interaction energy

B : Stereo Pixel-Labeling stereo with the potts model Stereo with occlusion Multi-camera scene reconstruction

C : Segmentation

35

Experimental Results.A(1/4)

Potts model

123

Linear interaction

1

Reference [2] , [4] , [8] , [15] , [18] , [21] , [32] , [36] , [38]

36

Experimental Results.A(2/4)

37

Experimental Results.A(3/4)

38

Experimental Results.A(4/4)

39

Experimental Results.part2

A : Image Restoration Potts model Linear interaction energy

B : Stereo Pixel-Labeling stereo with the Potts model Stereo with occlusion Multi-camera scene reconstruction

C : Segmentation

40

Experimental Results.B(1/6)

Potts model 1

Stereo with occlusion 1

Reference [4] , [7] , [23] , [24]

41

Experimental Results.B(2/6)

42

Experimental Results.B(3/6)

43

Experimental Results.B(4/6)

Multi-camera scene reconstruction 1

○ 1

1○ The smoothness term is the sum of Potts energy terms over all cameras.

○ The visibility term is infinity if a configuration f violate the visibility constraint and zero otherwise.

44

Experimental Results.B(5/6)

45

Experimental Results.B(6/6)

46

Experimental Results.part2

A : Image Restoration Potts model Linear interaction energy

B : Stereo Pixel-Labeling stereo with the potts model Stereo with occlusion Multi-camera scene reconstruction

C : Segmentation

47

Experimental Results.C(1/4)

Potts model

123

Reference [3] , [5] , [28] , [29] , [33] , [35]

48

Experimental Results.C(2/4)

49

Experimental Results.C(3/4)

50

Experimental Results.C(4/4)

51

Conclusions In most examples, our new min-cut/max-flow algorithm

worked 2-5 times faster than any of the other methods, including the push-relabel and the Dinic algorithms (which are known to outperform other min-cut/max-flow techniques).

However, our algorithm is not a clear outperformer when the complexity of underlying graphs is increased.

52

Reference(1/4) [2] Y. Boykov and D. Huttenlocher, “A New Bayesian Framework for

Object Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 517-523, 1999.

[3] Y. Boykov and V. Kolmogorov, “Computing Geodesics and Minimal Surfaces via Graph Cuts,” Proc. Int’l Conf. Computer Vision, vol. I, pp. 26-33, 2003.

[4] Y. Boykov, O. Veksler, and R. Zabih, “Markov Random Fields with Efficient Approximations,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.

[5] Y. Boykov and G. Funka-Lea, “Optimal Object Extraction via Constrained Graph-Cuts,” Int’l J. Computer Vision, 2004. to appear.

[7] Y. Boykov and V. Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,” Proc. Int’l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 359-374, Sept. 2001.

53

Reference(2/4) [8] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy

Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.

[15] D. Greig, B. Porteous, and A. Seheult, “Exact Maximum A Posteriori Estimation for Binary Images,” J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp. 271-279, 1989.

[18] H. Ishikawa and D. Geiger, “Segmentation by Grouping Junctions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 125-131, 1998.

[21] J. Kim, J.W. Fisher III, A. Tsai, C. Wible, A.S. Willsky, and W.M. Wells III, “Incorporating Spatial Priors into an Information Theoretic Approach for f MRI Data Analysis,” Medical Image Computing and Computer-Assisted Intervention, pp. 62-71, 2000.

54

Reference(3/4) [23] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence

with Occlusions via Graph Cuts,” Proc. Int’l Conf. Computer Vision, July 2001.

[24] V. Kolmogorov and R. Zabih, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. Seventh European Conf. Computer Vision, vol. III, pp. 82-96, May 2002.

[28] S. Osher and N. Paragios, Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer Verlag 2003.

[29] S. J. Osher and R.P. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag, 2002.

[32] S. Roy and I. Cox, “A Maximum-Flow Formulation of the N-camera Stereo Correspondence Problem,” IEEE Proc. Int’l Conf. Computer Vision, pp. 492-499, 1998.

55

Reference(4/4) [33] G. Sapiro, Geometric Partial Differential Equations and Image

Analysis. Cambridge Univ. Press, 2001. [35] J.A. Sethian, Level Set Methods and Fast Marching Methods.

Cambridge Univ. Press, 1999. [36] D. Snow, P. Viola, and R. Zabih, “Exact Voxel Occupancy with

Graph Cuts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 345-352, 2000.

[38] B. Thirion, B. Bascle, V. Ramesh, and N. Navab, “Fusion of Color, Shading and Boundary Information for Factory Pipe Segmentation,” IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 349-356, 2000.

56

Q & A