s Maximum Flow byIncremental

Breadth First Search

Joint work with: Haim Kaplan Robert E. TarjanTel Aviv University Princeton University & HP LabsRenato F. Werneck Andrew V. GoldbergMicrosoft Research Microsoft Research

Sagi HedTel Aviv University

Maximum Flow• Input: graph G=(V,E), vertices s, t є V and capacity

assignment c(e) for e є E• Output: flow function f satisfying -

conservation: for every v≠s,t Σ(u,v)єE f(u,v) = Σ(v,u)єE f(v,u)capacity: for every e f(e) ≤ c(e)

with maximal |f|=sum of flow out of s (into t)• Well studied problem• Equivalent to the Minimum s-t Cut problem• Solution methods:

Augmenting Path (and blocking flow), Network Simplex, Push-Relabel

Push-Relabel

• A different approach –Push vs. augment path, pre-flow vs. flow[Goldberg, Tarjan 88]

• Some Push-Relabel bounds: O(mn2) Any active vertex selectionO(n3) FIFO active vertex selectionO(n2m½) Highest label active vertex

selectionO(mn log(n2/m)) with dynamic trees

• Push-Relabel considered the most efficient general-purpose solution in practice[Cherkassky, Goldberg 97]

Maximum Flowin Computer Vision

• Minimum s-t cut very useful in the field of computer vision

• Applications in image segmentation, stereo image processing, video transitioning…

• Typical Process –• 2D or 3D images are converted to input graphs,

where each vertex corresponds to a pixel• Minimum s-t cut on these graphs provides

information on the original image(s)

Maximum Flowin Computer Vision

• These graphs have specific structure

• Regular low degree grids• Arc capacities: different models for grid arcs and s-t arcs

BK

Boykov and Kolmogorov developed an algorithm (BK) which is the fastest in practice on the vision instances[Boykov, Kolmogorov 04]

• Used as the standard min-cut algorithm in computer vision • Usually outperforms Push-Relabel implementation by large

factors

Problem: BK has no known polynomial time guarantee…Best bound is O(mnF) for integral capacities (F is the maximal flow value)

• Indeed on some instances, BK performs poorly and is outperformed by Push-Relabel implementation

Our ContributionIBFS

We develop the IBFS algorithm –Incremental Breadth First Search

• Has many similarities to BK and to Dinitz• However, performs shortest path or nearly

shortest path augmentations• Competative in practice to BK

Usually outperforms BK by small factors• Has a polynomial worst case time guarantee

O(mn2)

BK Overview

• Grows trees S, T in the residual graph bi-directionally• We maintain a list of active nodes, from which the

trees can grow

s t

S T

active nodes

s

BK Overview

• When the trees meet, we augment flow• After an augmentation, we try to rebuild the trees

s t

S T

s

active nodes

BK Drill Down

• Initially: S={s}, T={t}, active node list = {s,t}• Iterates between 3 phases:

Growth, Augmentation, Adoption

s

s t

S T

BK Drill Down

Growth:Iterate through active node list and grow S,TAdd new vertices to the back of the active list (FIFO)

s t

S T

ts

s

BK Drill Down

Augment:• Discover other tree during growth => augment flow• Saturated tree arcs create orphan sub-trees

S T

s t

Orphan

Orphan

s

BK Drill Down

Adoption: (symmetric for S and T)• Iterate over orphans and check potential parents• If an orphan finds a parent its entire subtree is

reattached

t

v

s

BK Drill Down

Adoption:• Checking a potential parent u: traverse the path from

u to the root, no orphans on the path

t

v

s

BK Drill Down

Adoption: Orphan v does not reconnect -• Remove v from tree and make children orphans• Make potential parents active• Make v inactive

t

v

s

BK Drill DownContinue Growth, Augmentation, Adoption• The trees are no longer neccesarily BFS trees• Augmenting paths are no longer neccesarily shortest• Growth alternates between S and T

s t

S T

s

BK Drill Down

Termination:• No more active nodes (either in S or in T)• Maximum flow value is the total augmented flow

s t

S T

s

IBFS Overview• We maintain S, T as BFS trees with heights ≈ Ds , Dt

• Active nodes are on level Ds or Dt only

• Augment on shortest (Ds+Dt+1) paths only (later nearly shortest paths)

s t

S TDs Dt

s

IBFS Overview

• Vertex v has label ds(v) ≤ Ds+1 and a label dt(v) ≤ Dt+1

• ds(v) and dt(v), are the level of the tree v is in

• ds(v) is meaningful if v ϵ S, dt(v) is meaningful if v ϵ T

s t

S TDs Dt

s

IBFS Drill Down

• Initially: S={s}, T={t}, active node list = {s,t}• ds(s)=0, dt(t)=0, Ds=0, Dt=0• As in BK, iterates between 3 phases:

Growth, Augmentation, Adoption

s t

S T

s

IBFS Drill DownGrowth:• Grow one complete level at a time, Ds++ or Dt++• If u grows v, ds(v) = ds(u)+1 / dt(v) = dt(u)+1• Can alternate forward/backward passes arbitrarily

s t

S T

ts

ds=0 ds=1 ds=2 dt=2 dt=1 dt=0

s

IBFS Drill DownAugment:As in BK,• Discover other tree during growth => augment flow• Saturated tree arcs create orphan sub-trees

S T

s t

Orphan

Orphan

ds=0 ds=1 ds=2 dt=2 dt=1 dt=0

s

IBFS Drill DownAdoption: (symmetric for S and T)• Iterate over orphans and check potential parents• If orphan v finds a parent u with dt(u)=dt(v)-1

v’s subtree reconnects

t

v

dt=2 dt=1 dt=0

s

IBFS Drill DownAdoption: Orphan v does not reconnect at same level –• Relabel(v): dt(v)=min{dt(u)|uϵT, (u,v) residual}+1

parent(v) = argmin{...}• Make children orphans• Make v inactive

t

v

v

dt=2 dt=1 dt=0

s

IBFS Drill DownAdoption: (symmetric for S and T)Remove v from T, if Relabel(v) does not find a parent

or Relabel(v) finds a parent u such that –forward pass: dt(u) ≥ Dt

backward pass: dt(u) ≥ Dt+1

t

v

dt=1 dt=0dt=Dt=2

v

s

IBFS Drill DownAdoption: (symmetric for S and T)• Orphan v may reconnect to an orphan subtree

(its own or another)• If neccesary processed as an orphan again later

t

v

dt=2 dt=1 dt=0

v

s

IBFS Drill DownContinue Growth, Augmentation, Adoption• The trees are always maintained as BFS trees• Shortest augmenting paths = Ds+Dt+1 (proof soon)• Alternate between forward/backward passes

s t

S T

s

impossibleDs Dt

IBFS Drill Down

Termination:• Empty level (either in S or in T)• Maximum flow value is the total augmented flow

s t

S T

s

IBFS vs. Dinitz

• Basically a form of Dinitz• Bi-directional rather than uni-directional• Auxilary network for next passes prepared

while processing current pass.Network not rebuilt from scratch every pass!

s

IBFS Drill Down

Current Arc (time efficiency only)• Remeber where the last orphan parent scan stopped• When v is added to the tree, current arc = first arc• current_arc(v) ϵ {first_arc(v), (parent(v),v)}

=> can be implemented with a bit

s

s

S

Current Arc

IBFS CorrectnessLemma 1: (symmetric for forward / backward passes) If (u,v) is residual• During a forward pass –

u in S, v not in S, ds(u) ≤ Ds => u active (ds(u)=Ds)• After we increase Ds until the next forward pass –

u in S, v not in S => ds(u)=Ds

s

S

v

s

uu vu

Ds

IBFS Correctness

By Lemma 1, when the algorithm terminates there are no more residual augmenting paths=> the flow is maximal

s

s t

S T

IBFS Time BoundDefinition: u,v in S, (u,v) is admissible:

(u,v) is residual and ds(v) = ds(u)+1

Algorithm Invariants: (symmetric for ds and dt) 1. Tree arcs are admissible2. current arc of u precedes the first admissible arc to u3. ds is a valid labeling: (u,v) residual => ds(v) ≤ ds(u)+14. ds(v) never decreases

s

s

IBFS Time Bound

Algorithm Invariants Proof:• By induction on the algorithm steps.• Valid ds labeling (invariant 3):

Growth step:By Lemma 1, there are no connections from lower

levelsAugmentation:

New residual arcs do not violate,by inductive assumption of admissible tree arcs

Adoption:Orphan relabel does not break valid labeling(as in push-relabel)

s

IBFS Time Bound

Conclusions from Algorithm Invariants:(not directly needed for analysis)

• S and T are BFS trees:ds(v) = the distance from s to v in the residual graphdt(v) = the distance from v to t in the residual graph

• We always augment on shortest paths in the residual graph

s

IBFS Time BoundLemma 2: (symmetric for S and T)After an orphan relabel on v in S, ds(v) increases.If v is removed from S, then we consider the increased

label the next time v is added to S.

Lemma 2 Proof:• Easy, except to avoid

pathalogical case of adoption/growth

s

s

v

ds=2ds=1ds=0

IBFS Time BoundLemma 2 Proof:Let U ≡ {u | u in S and (u,v) is residual}Let ds’(v)=ds(v) at time of relabel(v)

• By valid labeling and current arc invariants –If new_parent(v) ϵ U=> ds(new_parent(v)) ≥ d’s(v)=> ds(v) = ds(new_parent(v))+1 ≥ d’s(v)+1

• By non-decreasing labels invariant, this is true at any future time

s

IBFS Time BoundLemma 3:IBFS runs in O(n2m) time

Lemma 3 Proof: (symmetric for S and T)Note there are ≤ n-1 different values for ds(v)• Growth

We charge the scan of arc (u,v) to ds(u)Each label charged by arc (u,v) at most once(u becomes inactive)Total: O(nm)

s

IBFS Time BoundLemma 3 Proof continued: (symmetric for S and T)• Adoption

We charge the scan of arc (u,v) (v orphan) to ds(v)Each label charged by arc (u,v) at most twice –• Once during scanning for a parent with label ds(v)-1

(due to remembering the current arc)• Once during orphan relabel (by Lemma 2)Total: O(nm)

s

IBFS Time BoundLemma 3 Proof continued: (symmetric for S and T)• Augmentation

Let (u,v) be an arc saturated in the augmentation.• If (u,v) is a tree arc, we create an orphan – no more than

charges made for adoption => O(nm)• If (u,v) is not a tree arc (u ϵ S, v ϵ T), we charge ds(u)

Each label is charged by arc (u,v) at most once(u becomes inactive) => O(nm)

Each augmentation takes O(n) timeTotal: O(n2m) □

s

IBFS VariantsNearly Shortest Path• Vertices at level Ds in S and Dt in T are activated together• We grow both trees at the same time, intermittently• Augmenting paths are shortest or “shortest+1”• More similar to BK

s

S T

s t

ds=0 ds=1 ds=Ds=2 dt=Dt=2 dt=1 dt=0

Ds+Dt+2

IBFS VariantsNearly Shortest Path• Passes are both forward and reverse at the same time• Correctness and running time are proved in the same

way after applying the above rule• Proves best in practice• Used in our implementation and experiments

s

IBFS Experiments

• Ran on 37 computer vision instances, different families27 public benchmark [http://vision.csd.uwo.ca/maxflow-data/]10 our own creation [http://www.cs.tau.ac.il/~sagihed/ibfs/]

• BK implementation available publicly [http://vision.csd.uwo.ca/code/]

• We compare to a modified version of BK, with the same low level optimizations as our own (≈ 20% faster)

• IBFS wins on 35 out of 372 different capacity versions of the instance “bone”

• Factors are mostly modest. For few they are large.

s

IBFS Experimentss

digged

hessi1a

house

anth

ra

bone_subx1

00

liver100

babyfa

ce100

bone100

bunny-med

camel-m

ed

gargo

yle-m

ed

kz2-ve

nus0.1

1

10

100

1000

IBFSBK

Running Time (seconds)

IBFS Experiments

• 4 operations counted:• Pushes – sum of augmentation lengths• Growth arc scans – number of arcs scanned during growth• Orphan arc scans – number of arcs scanned during adoption• Orphan traversal to root (BK) – number of arcs traverse to

check the root of the potential parent’s tree• Growth and Orphan arc scans access arcs sequentially in

memory• Pushes and Orphan traversal to root access arcs non-

sequentially in memory

s

Operation Counts

IBFS ExperimentsOT Orphans Growth Pushes

Speedup Instance BK BK IBFS BK IBFS BK IBFS

38.4 7.7 87.8 7.7 6.7 160.0 16.9 3 digged

126.5 43.9 601.7 25.4 7.3 353.2 108.4 1.11 hessi1a

43.7 13.3 129.6 10.2 6.3 122.2 33.0 1.24 house

83.3 27.3 348.3 17.3 6.8 153.0 53.5 1.07 anthra

23.0 6.8 30.1 8.8 6.8 10.9 2.8 1.17 bone_subx100

66.5 13.6 56.0 12.3 6.9 23.2 7.5 2.15 liver100

39.5 9.5 46.3 10.7 6.6 12.7 4.5 1.76 babyface100

7.0 5.1 35.6 8.1 6.9 2.0 0.5 0.79 bone100

0.6 0.4 0.6 6.2 6.2 0.5 0.3 1.23 bunny-med

61.2 13.0 92.4 9.4 6.8 74.0 20.4 1.54 camel-med

250.5 20.7 121.6 12.1 8.7 337.2 22.7 6.16 gargoyle-med

8.1 13.5 18.0 11.2 8.8 6.2 3.3 1.39 kz2-venus

s

Operation Counts (per vertex)

IBFS Experiments

• On non computer-vision graph families [http://www.avglab.com/andrew/CATS/maxflow_synthetic.htm]IBFS outperforms BK, some by large factors

• On these, standard Push-Relabel implementation outperforms both IBFS and BK.

s

Summary• BK works well on computer vision problems in

practice, but does not provide a polynomial run time guarantee

• IBFS works as well in practice and provides a polynomial run time guarantee

• IBFS operates similarly to BK, and can also be viewed as a bi-directional version of Dinitz where the auxiliary network is constantly recovered rather than rebuilt.

Open Question

• Can you find a maximum-flow algorithm with a polynomial time bound, which is competitive in practice with both Push-Relabel and BK on all graph families?

THANK YOU!

s