institute for advanced study, april 16 2012 sushant sachdeva princeton university joint work with...

Institute for Advanced Study, April 16 2012

Sushant SachdevaPrinceton University

Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi

Linear Time Graph Partitioningvia Fast Simulation of Random walks

INTRODUCTIONTheory of near-linear time algorithms

Why linear-time algorithms?

Input graphs and data sets are massive.e.g. Web graph (1011 nodes), Social networks (109 nodes)

Super-linear algorithms are impractical.

© 2004–2012 Michael K. Bergman.

Need time algorithms for fundamental problems:e.g. MAX-MATCHING, s-t MINCUT/MAXFLOW, Balanced Graph Partitioning

Approximate solutions maybe?

(1-²) approx., time[Duan and Pettie]

(1-²) approx., time[Christiano et al.]

Spectral approx., time

[This talk]

GRAPH PARTITIONING

Conductance

Given a undirected, unweighted graph

S Conductance of

[Cheeger, Alon-Milman]

[Spielman-Teng] allows us to approximate ¸2(L) and find such a cut in

time

Assume G is d-regularSpectral approximation

Conductance of G

Fundamental quantity in Markov Chains, Riemannian

Manifolds

Balanced SeparatorGiven b, does G have a b-balanced cut of

conductance < °?S

G NP-Hard

Applications to clustering, image segmentation, community detection,

primitive for divide-and-conquerAlgorithm Techniqu

eDistinguishes ≤ ° and

Running Time

Recursive Eigenvector

Spectral

Use Cheeger to find a cut and

remove it. Rinse and repeat.

Can take time. Too slow.


conductance < °?S

G NP-Hard

Algorithm Technique

Distinguishes ≤ ° and

Running Time


Spectral

[Leighton-Rao] Flow [AK,OSVV,CKM+]

[Arora-Rao-Vazirani]

SDP [AK,She,CKM+]

[Madry]


conductance < °?S

G NP-Hard

Algorithm Technique Distinguishes ≤ ° and

Running Time


Spectral

[Spielman-Teng] Local Random walks

[Andersen-Peres] Evolving Sets

[Orecchia-Vishnoi] SDP


conductance < °?S

G NP-Hard

Algorithm Technique Distinguishes ≤ ° and

Running Time


Spectral

[Spielman-Teng] Local Random walks

[Andersen-Peres] Evolving Sets

[Orecchia-Vishnoi] SDP

[This talk] Simulating Random walks


conductance < °?S

G NP-Hard

Algorithm Technique

Distinguish ≤ ° and

Running Time


Spectral

[Leighton-Rao] Flow [AK,OSVV,CKM+]

[Arora-Rao-Vazirani]

SDP [AK,She,CKM+]

[Madry]

[This talk] Simulating random walks

Overview

1. Use continuous-time random walks to give an algorithm

2. Show how to simulate continuous-time random walks fast

Graph Cuts and Random Walks

Intimate connection between Graph cuts and Random walks.

[Alon-Milman][Mihail][Lovasz-Simonovits][Spielman-Teng][Anderson-Chung-Lang][Andersen-Peres][Orecchia-Vishnoi][this talk]…

[Mihail] Given the distribution of a random walk that has mixed, can find a cut S with

in time.

Usual Random Walks on a graph

Page Rank Random walksEvolving Sets Random WalksContinuous Time Random Walks

Bird’s eye view1. Simulate random walks (one or several)

3. Else, modify the graph

Regular walk / Page Rank walk /Evolving Sets /Continuous-time walks

2. Try to cut the graph using the random walksIf there’s no low conductance cut, we get a certificate.If you find a balanced low conductance cut, we’re done. To remove the unbalanced low conductance cuts

Remove the set

Add edges across the cut

Using matchings /flows /stars

Threshold cuts/SDP rounding

Continuous Time Random Walks

Thm. We can reduce Balanced Separator to simulating

heat-kernel random walks with

Given initial distribution ,

Random walk of length l ~ Poisson(t)

Heat-Kernel random walk

Why heat-kernel random walk?

e-x is a nice function. Track progress using Golden Thompson inequality [AK]Don’t know such a result for other random walks

Techniques from SDPs[OV], exponential updates aka Matrix Mult. Weights [AK]

Previous results had a poly(1/°) dependence

Algorithm

Wej

An embedding in Random walk from vertex i

Case 1: All vectors are small

Long vectors are random walks that

haven’t mixed

Wei

All random walks have mixed well.

No small cuts.

Ensures mixing across cuts with

conductance more than °

Algorithm

Case 2: Walks have not mixed

Wej

WeiRandom Projection

+Threshold Cut

“Cheeger Rounding”

If we find a balanced cut of small conductance , we’re

done.ELSE

Algorithm

Case 2b: Walks have not mixed (and we didn’t find a balanced cut)

Wej

WeiWe can find a ball cut S

S

S is the union of “all unbalanced low

conductance cuts”

Algorithm

Case 2b: Walks have not mixed (and we didn’t find a balanced cut)

Wej

WeiWe can find a ball cut S

SAdd edges from all vertices outside the ball to all inside

Soft removal of unbalanced cuts

O(log n) rounds

For efficiency, we use JL lemma and work with

O(log n) dimensional embeddings. [AK]Goal: Given L,¿,v. Compute Goal: Given L,¿ ,v. Approximate

SIMULATING RANDOM WALKS

Approximating Heat-Kernel

Goal: Given ¿, L, v, ±, find u s.t.

Taylor Series Approx.?

Need

Too Slow

Heat-Kernel random walk required by

algorithm

Small error suffices for the algorithm Used in

previous algorithms


Better polynomials?

YES!!

[Thm] There exists a polynomial p of degree such that

It suffices to approximate e-x on

[Thm] Given G that has a balanced cut of conductance , we can find one of conductance

in time Better guarantee, same

running time as [Andersen-Peres]

Still not near-linear!



[Thm] Any polynomial such that requires degree

Even better polynomials?

Not Really!

Simple proof using Markov’s theorem


Beyond PolynomialsRational Functions![Saff-Schönhage-Varga] There is a degree k polynomial pk such that

Infinite Interval!

Geometric Decay!Need

degree

Assume: 1. We knew pk explicitly 2. ST computation was exact

We’re DONE!

Two Issues: 1. [SSV] result is existential

2. ST computation is approximate

Approximating up to error ±

Computing vectors

Can use [Spielman-Teng]![Thm] Given ¿, L, v, ±, we can compute u such

that in

time

[Spielman-Teng] (Informal) We can approximate (cI+L)-1y in

time.

LANCZOS’ METHOD“But we don’t know the polynomial.”

Lanczos’ MethodLet

Define f so that

Goal: Approximate f(B)v as well as any degree k poly.

Lanczos Method

Existence of a good approximate poly suffices

Small degree implies efficiency

From real approximation to matrices

Lanczos’ Method

Goal: Approximate f(B)v as well as any degree k poly.

Observe

p – any degree k polynomial

Krylov Subspace · of

order k

Let be an orthonormal basis for ·

Let T be the operator B restricted to · (k+1) x (k+1) matrix

For any degree k polynomial p,

Let Projection on · Our guess:

T is a much smaller matrix!

Error Guarantee

Goal: Bound the norm of the error

Let the error be

Equal!

Since T is B restricted to ·,Spectrum(T) is bounded by Spectrum(B)

* error of best degree k polynomial approximation

Cost = k mult. by B + Construct basis + Diagonalize T

for our setting

Suppose p(x) is a good degree k approximation to f

Bounded by r on Spectrum(B)

Bounded by r on Spectrum(T)

Don’t even need to know the polynomial!

HANDLING APPROXIMATE COMPUTATION

What about the error?

We bound the error

Why Lanczos is insufficient

Multiplication withis only approximate

Subspace · is approximate.& Operator T is

approximate

T is not even symmetric!

We lose nice spectral

properties

[ST] computation is only approximate

We symmetrize.Compute approximation with

We needed to compute an orthonormal basis for

Sketch of Error AnalysisBound

Bound BoundBound

Polynomial part

Error part

Rational approx. result

BoundBound sum of coefficients of

pk

Bound spectrum

of

Error Analysis + Lanczos method = Fast random walks

Fast random walks + Algorithm = Fast Balanced Separator

Conclusions

• Reduced balanced separator to simulating heat-kernel random walks

• Approximated the heat kernel random walk in time.

• Can be used as primitives for designing near-linear time algorithms.

Open Questions

• Other applications of fast matrix exponentiation

• Linear-time Graph decomposition?• Linear time algorithms for small set

expansion?• Linear time algorithm for

approximation for Balanced Separator

Thank you!

An Important Tool[Christiano et al.]

s-t MAXFLOW s-t MINCUT

Electrical Flows

Solve Linear Systems

Lx=y

L is a graph Laplacian

Solving Laplacian Linear Systems

Direct Methods

Gaussian Elimination /

Cholesky Decomposition

Iterative methods

Too slow. O(n3) time

Hopeless

Conjugate Gradient

Approximate method.

time

[Spielman-Teng] (Informal) We can approximate L-1y in time.

Other Applications – Approximating second eigenvector [Spielman-Teng], Cover times [Ding-Lee-

Peres], Generating random spanning trees[Madry-Kelner], Graph Sparsification [Spielman-Srivastava]

Balanced graph partitioning,Simulating continuous-time random walks

[This talk]

Beyond Polynomials

Rational Functions!

[Saff-Schönhage-Varga] There is a degree k polynomial pk such that

Infinite Interval!

Geometric Decay!Small required

degree.

L1 bound on [0,1)

L1 bound on [-1,1]

L1 bound on [-1,1]

L2bound on [-1,1]

Change IntervalWrite function as an

integralCauchy Schwarz

Assume: 1. We knew pk explicitly 2. ST Solver was exact

We’re DONE!

Two Issues: 1. We don’t know pk explicitly 2. ST Solver

is approximate

Faster SimulationPolynomials don’t suffice, then, how do we simulate the random walk fast?

Lanczos Method

Goal: Given symmetric B, v, k, and function f;approximate f(B)v as well as any degree k poly

Observe

p – any degree k polynomial

Krylov Subspace · of

order kLet be an orthonormal basis for ·

k-Identity Projection on

·Let

For any degree k polynomial p,

Our guess:

T is a much smaller matrix!

Error Guarantee

Goal: Bound the norm of the error

Write Any degree

k polynomial

Equal!

Since and the columns of V are orthnormal,

The columns of V are orthnormal

True for any p!!

As good as the best degree k polynomial!

Cost = k mult. by B + Construct basis + Diagonalize T

if B is a Laplacian

Using Lanczos for our Algorithm

Goal: Given A, wanted to approximate as

Use Lanczos with

suffices.

Still need to address the issue of approximate multiplication with

institute for advanced study, april 16 2012 sushant sachdeva princeton university joint work with...

Documents