Expander Flows, Graph Spectra and Graph Separators

Umesh VaziraniU.C. Berkeley

Based on joint work with Khandekar and Rao and with Orrechia, Schulman and Vishnoi

Graph Separators

S T

Sparsest Cut/Edge Expansion: S

EG

TS

VS

,

2

||||

min)(

c-Balanced Separator: S

EG

TS

VSVc

,

2

||||||

min)(

Applications

• Clustering• Image segmentation• VLSI layout

• Underlie many divide-and-conquer graph algorithms

Interesting Techniques

• Spectral methods. Connection to differential

geometry, discrete isoperimetric inequalities.

• Linear/semidefinite programming

• Measure concentration

• Metric embeddings

Geometrical view

• Map vertices to points in some abstract space: - points well-spread - edges short

Geometrical view

• Map vertices to points in some abstract space: - points well-spread - edges short

• “Good bisection” of the space yields sparse cut in graph

Spectral Method

1)(

)(min

2

,

2

,

jji

i

jEji

i

xx

xx

Cut at random

Minimize sum of “edge lengths”:

Spread out vertices:

[Cheeger’70] [Alon, Milman ’85][Jerrum, Sinclair’89]

4

)()(

2

10

GG

Leighton-Rao ‘89

mw

w

jiij

Ejiij

,

,

min

22

22

2

5

4

5

5

4

11

1

1

1

1

Cut along ball of random radius

Distances form a metric: satisfy triangle inequality.wij + wjk >= wik

Minimize sum of “edge lengths”:

Spread out vertices:

O(log n) approximation: Approximate max-flow min-cut thm for multi-commodity flows.

ARV ‘04

Triangle inequality:222 )()()( kikjji vvvvvv

Unit sphere in Rd

Unit L22 embedding:

No anglesobtuse

Minimize sum of “edge lengths”

Spread out vertices

22

,

2

,

)1(4)(

)(min

nccvv

vvW

jji

i

jEji

i

)log)(( nGO Procedure to recover cut of size

ARV Procedure to recover cut

• Slice a randomly oriented “fat”-hyperplane of width

d

O1

Unit sphere in Rd

ARV Procedure to recover cut

• Slice a randomly oriented “fat”-hyperplane of width

• Discard pairs of points (u,v):

• Arrange points according to distance from S• Cut along ball of random radius r:

d

O1

Unit sphere in Rd

nvu

log

12

S

r0

Metric EmbeddingsFinite Metric Space (X, d)

x

y

Rk with L2 norm

f(x)

f(y)

Distortion of f is min c: ),()()(),(2

yxdcyfxfyxd

[Bourgain ’85] Every finite metric space can be embedded in L2 with distortion O(log n).

Longstanding open question: Better bound for L1?

[Enflo ’69] [Arora, Lee, Naor ’05] Any finite L1 metric can be embedded in

L2 with distortion nnO logloglog

f()

nlog

Today’s Talk

• Leighton-Rao: multi-commodity flow O(n2).• Arora, Hazan, Kale: O*(n2) ARV implementation

based on expander-flow formalism

• Much faster in practice. • [Khandekar-Rao-V] : O*(min{n1.5, n/α(G)}) single commodity

flow based algorithm. O(log2 n) approx. ratio.• [Arora, Kale]: matrix multiplicative weights algorithm based O(log n) approx• [Orrechia, Schulman, V, Vishnoi] O(log n) approx using

KRV style algorithm

Multi-commodity flow:

Single commodity flow:

Expander Flows

• Any algorithm for approximating sparse cuts must find a good cut, of expansion say β

• Must also certify no cut is much smaller.

• To give a k-approximation must certify that no cut has expansion less than β/k.

• Problem: there are exponentially many cuts.

ST

Expander Flows

G = H =

• For each edge of H, route one unit of flow through G

Expander Flows

G = H =

• For each edge of H, route one unit of flow through G

• Must route Ώ(|S|) units of flow from S to T.

• Therefore |ES,T| = Ώ(|S|/c) expansion = Ώ(1/c)

• Ideally c = O(1/α(G))

ST

max congestion = c expansion = Ώ(1/c)

Expander Flows

• max congestion = c. expansion = Ώ(1/c).

• ARV: max congestion =

• Leighton-Rao: H = complete graph. max cong = O(logn/α(G))tight example: G = expander graph.

• Motivating idea for ARV: write LP to find best embedding of H in G + exponentially many constraints saying H expander

eigenvalue bound gives efficient test for expansion!Therefore poly time using Ellipsoid algorithm.

• [Arora, Hazan, Kalle] O*(n2) implementation of ARV

)(

log

G

nO

• Know large number of vertices on each side of cut.

• A max-flow, min-cut computation should reveal sparse cut.

• But this is circular…

KRV

s t

• H Φ

• Embed candidate expander H in G with small congestion.

• Test whether H is expander (if so done!)

• Else non-expanding cut in H gives a bipartition of G; route a flow in G across this bipartition.

• Decompose flow into flow paths and add the resulting matching to H.

Outline of Algorithm

Cut-Matching Game

H Φ

Cut Player• Find bad 50-50 cut in H

• Goal: min # iterations until H is an expander

Matching Player• Pick a perfect matching

across cut

Goal: max # iterations untilH is an expander

Claim: There is a cut player strategy that succeeds in O(log 2 n) rounds.

Finding a cut: Spectral-like-method

= +1 charge

= –1 charge

Mix the charges alongthe matchings { M1, M2, …, Mt }

Random assignment of charge

V: Vertex setx

y

(x+y)/2

(x+y)/2

After t iterations, H = { M1, M2, …, Mt }.

Finding a cut: Spectral-like-method

Order the vertices according to the final charge presentand cut in half.

n/2 n/2

S S

But how to formalize intuition?

Lift to Rn

• Cannot directly formalize previous intuition Therefore lift random walk to Rn – walk embedding of H.

• n-dimensional vector associated with each vertex

• In each step, replace vectors at endpoints of matched edge by their average vector.

• Potential function to measure progress of this process.

• Potential function small implies H expander.

• Relate lifted process to original random walk: each successive matching decreases potential function.

Walk Embedding

H Rn,

Vertex i mapped to Pi = (pi1, …, pin)

pij = P[walk started at j ends at i]

Ht = { M1, M2, …, Mt }.

Small cut in graph shows up as clusters in walk embedding. (1/n, …, 1/n)

P1

P3P2

Pn

2

,

2)/1)((/)()( ntpntPt

jiij

ii 1Potential:

Claim: ψ(t) ≤1/4n2 implies α(Ht)≥ ½

Will show potential reduces by (1 – 1/log n) in each iteration.

Ψ(0) = n-1

(1/n, …, 1/n)

P1

P3P2

Pn

Main Question: How to augment Ht = { M1, M2, …, Mt }

by Mt+1 so H closer to expander?2

,

2)/1)((/)()( ntpntPt

jiij

ii 1Potential:

If Mt+1 matches vertex u to vertex v,

then potential reduction in t+1-st step

Since each of Pu and Pv replaced by

So potential reduction =

2vu PP

22

22

2

1

22 vu

vuvu PP

PPPP

The Lifted Walk

2

2

1vu PP

Potential Reduction

Pv

= v |Pv1/n|2

Reduction in = |green|2

1-d: reduction = ()n-d 1-d: log n stretch

Actual potential reduction = /log n

Original random walk = projection of lifted walk on random vector

Running time

• Number of iterations = O(log2 n)• Each iteration = 1 max-flow + O*(n) work

= O*(m3/2)

• [Benczur-Karger’96] In O*(m) time, we can transform any graph G on n vertices into G’ on same vertices:– G’ has O(n log (n)/ε2) edges– All cuts in G’ have size within (1 ± ε) of those in G

• Overall running time = O*(m + n3/2)

Improving to O(log n) approximation

• [Arora, Kale]: matrix multiplicative weights algorithm based combinatorial primal-dual schema for semidefinite progs

• [Orrechia, Schulman, V, Vishnoi]: simple KRV style algorithm

Idea: To find Mt+1 perform t steps of natural random walk

(instead of round-robin walk) on Ht = { M1, M2, …, Mt }

Brief Sketch

• Instead of showing that H has constant edge expansion after O(log2 n) steps, will show that the spectral gap of H is at least 1/log n, and therefore the conductance of H is at least 1/log n.

• Since degree of H is log2 n, this means its edge edge expansion is at least log n.

Why natural walk?

Suppose round robin walk on M1, … , Mk mixes perfectly

on each of S, T. Now a single averaging step on Mk+1

ensures perfect mixing on entire graph!

S T

Mk+1

Matrix inequality: tttt ABAABA )(

Question: Replace ½ self-loop with a ¾ self-loopin round-robin random walk!

x

y

(3x/4+y/4)

(3y/4 + x/4

Gives a way of relating round robin walk to time independent walk.

Conclusions and Open Questions

• Our algorithm is very similar to some heuristics.

• [Lang’04] similar to one iteration of our algorithm.

• METIS [Karypis-Kumar’99]– collapses random edges– finds a good partition in collapsed graph– induces it up to original graph, using local

search

• Connections with these heuristics? Rigorous analysis?

When the Expansion is large …

• Could have used [Spielman-Teng’04] “nibble” algorithm instead of walk-embedding. But:

Algorithm Output sparsity

Running Time

Spectral 1/2 n2/2

Spielman-Teng

1/3 log3 n n/3

KRV log2 n min {n3/2,n/}

• Conjecture: A single iteration of round-robin walk + max-flow should give a sparse cut.

• [Khot, Vishnoi] Ώ(loglog n) integrality gap

• [Orrechia, Schulman, V, Vishnoi] Ώ(√logn)bound on cut-matching game.

• Is it possible to obtain a O(√log n) approximation algorithm using single commodity flows via the cut-matching game?

Limits to these methods