1 algebraic and combinatorial tools for optimal multilevel algorithms yiannis koutis carnegie mellon...
DESCRIPTION
3 May 3, 2007 Combinatorial linear algebra and scientific computing Start with a system Ax =bTRANSCRIPT
![Page 1: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/1.jpg)
1
Algebraic and combinatorial tools for optimal multilevel algorithms
Yiannis KoutisCarnegie Mellon University
![Page 2: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/2.jpg)
2 May 3, 2007
Spectral graph theory Combinatorial scientific computing Start with a system Ax =b
The problem of fill
![Page 3: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/3.jpg)
3 May 3, 2007
Combinatorial linear algebra and scientific computing Start with a system Ax =b
![Page 4: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/4.jpg)
4 May 3, 2007
Combinatorial linear algebra and scientific computing Start with a system Ax =b
Complete Mess!1
![Page 5: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/5.jpg)
5 May 3, 2007
Combinatorial linear algebra and scientific computing Start with a system Ax =b
1
![Page 6: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/6.jpg)
6 May 3, 2007
Combinatorial linear algebra and scientific computingMatrices viewed as graphs [direct methods]: Planar positive definite matrices in O(n1.5) time [George], [Lipton, Rose,Tarjan]
Graphs viewed as matrices [iterative methods]: Approximate sparsest cut in O(m polylog(n)) time [Spielman,Teng] Find eigenvector of the Laplacian of the graph
A wonderful theory of graph approximations where combinatorics and algebra work in synergy
![Page 7: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/7.jpg)
7 May 3, 2007
contributions
Linear work parallel algorithms for
Combinatorial problems: Multi-way planar edge partitioning Multi-way planar vertex partitioning
Algebraic problems: Solving systems with planar Laplacians Solving systems with a priori known structural properties
![Page 8: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/8.jpg)
8 May 3, 2007
contributions
Theory for Perturbations of graph eigenvectors Structure of eigenvectors with respect to edge cuts
Applications to Classical algebraic multigrid algorithms Graph-theoretic approach to design and analysis
![Page 9: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/9.jpg)
9 May 3, 2007
why planar systems?
•images are formulated as rectangular grids [up to 1 billion nodes]
•million of images must be processed every day (mammograms, OCT retinal)
•weights vary by a factor of 106
•PDEs discretized with finite elements [Boman,Hendrickson,Vavasis 05]
Leo Grady@ Siemens
![Page 10: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/10.jpg)
10 May 3, 2007
why laplacians? adjacency: A(i,j) = wi,j
degree sequence: D(i,i) = j wi,j
Laplacian: L = D-A Normalized: N = D-1/2 L D-1/2
Random walk matrix: I-0.5D-1L
cut structure of the graph [Cheeger], Euclidean commute time
spectral properties of the
Laplacian
capture
combinatorial properties of the
graph
![Page 11: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/11.jpg)
11 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
![Page 12: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/12.jpg)
12 May 3, 2007
planar multi-way edge partitioning
Partitioning the edges into disjoint clusters
with small boundaries
n/k1/2 edges delimiting
pieces of size O(k)
![Page 13: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/13.jpg)
13 May 3, 2007
planar multi-way edge partitioning is it possible for any planar graph?
planar separator theorem: every planar graph can be split roughly in half by removing n1/2 vertices.
+ recursive bisection+ a few bells and whistles
= O(n/k1/2) edges that delimit pieces of size O(k)
In O(nlog n) time: recursively apply planar separator [Fre87] In our work: O(kn) time using a localized approach
![Page 14: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/14.jpg)
14 May 3, 2007
a quick time outline of multi-way edge partitioning in linear time
triangulate graph form k-neighborhood of every face
2nd layer
partial layer
![Page 15: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/15.jpg)
15 May 3, 2007
multi-way edge partitioning in linear timethe set of independent k-neighborhoods
a set of independent k-neighborhoods
[no blue neighborhoods intersect]
the set is maximal [Every red neighborhood intersects a blue neighborhood]
![Page 16: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/16.jpg)
16 May 3, 2007
multi-way edge partitioning in linear timedecomposition into Voronoi regions
every exterior face is assigned to “closest” blue neighborhood
![Page 17: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/17.jpg)
17 May 3, 2007
multi-way vertex partitioning in linear timedecomposition into Voronoi regions
every exterior face is assigned to “closest” blue neighborhood
n/k connected Voronoi regions: faces in Voronoi graph
![Page 18: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/18.jpg)
18 May 3, 2007
multi-way edge partitioning in linear timedecomposition into Voronoi-Pair regions
paths from center faces of neighborhoods to surrounding Voronoi nodes
graph decomposed into constant size Voronoi-Pairs that are easy to deal with
how many paths did we add? O(n/k) still too many?
![Page 19: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/19.jpg)
19 May 3, 2007
multi-way edge partitioning in linear timecovering each long path with cores
total boundary = cores + exposed part = O(k1/2) n/k paths * k1/2 boundary = O(n/k1/2) edges
N v
we are done!!
![Page 20: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/20.jpg)
20 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
![Page 21: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/21.jpg)
21 May 3, 2007
multi-way vertex partitioning in linear timeinto expander graphs
there are planar expanders
A B
1 2 4 2n
![Page 22: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/22.jpg)
22 May 3, 2007
multi-way vertex partitioning in linear timeinto “isolated” expander graphs
Requirements:1. a set of m disjoint clusters of vertices Vi
2. each subgraph on Vi is an expander
3. each expander is “isolated” from its exterior
4. n/m is constant
![Page 23: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/23.jpg)
23 May 3, 2007
planar multi-way vertex partitioninglocal sparsification
Maximum Weight Spanning
Tree Factory
local sparse component: component size k
each vertex keeps 1/k of its incident weight
MST
![Page 24: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/24.jpg)
24 May 3, 2007
planar multi-way vertex partitioningglobal sparsification
Maximum Weight Spanning
Tree Factory
global sparse graph: each vertex keeps 1/k
of its incident weight
total number of edges n-1 + O(n/k1/2)
MST
![Page 25: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/25.jpg)
25 May 3, 2007
planar multi-way vertex partitioningthe numerical insight
Greedy contraction strategy for no fill:1. Greedily eliminate degree 1 vertices2. Greedily replace a vertex of degree 2
by an edge between its neighbors
How far do we get?If the graph has n-1+t edges greedy contraction gives a graph with 4t vertices
![Page 26: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/26.jpg)
26 May 3, 2007
planar multi-way vertex partitioningdecomposing the global sparse graph
n-1 + O(n/k1/2) edges greedy contraction stops in O(n/k1/2) block vertices
vertex disjoint trees: use parallel tree contraction [Miller, Reif]
lightest edge
we are done!!
![Page 27: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/27.jpg)
27 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
![Page 28: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/28.jpg)
28 May 3, 2007
solving Laplacian systemsmultilevel algorithms
Hard goals yield hard rulesHard goal: linear time algorithmHard rule: we cannot afford fill
![Page 29: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/29.jpg)
29 May 3, 2007
solving Laplacian systemshierarchies of graphs
A. not too many levelsB. good approximation
between levels
Solving requirement:reduction / approximation1/2 < ½
Solving complexity:O(reduction*graph size)
approximation measure
size reduction
![Page 30: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/30.jpg)
30 May 3, 2007
the approximation measurealgebraically natural
condition number
eigenvalue characterization
Rayleigh Quotient
![Page 31: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/31.jpg)
31 May 3, 2007
the approximation measure“naturally” natural
graph
xT A x =
electrical network
energy consumption with vector of voltages x
c r=1/c
A,B) compares the “energy” consumption of the two networks
![Page 32: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/32.jpg)
32 May 3, 2007
the approximation measurecombinatorially natural
Multicommodity flows: For every edge (u,v) of A: send w(u,v) units of flow between u and v in B
A solution is characterized by: congestion: the maximum congestion over edges in B dilation: “weighted” diameter of paths in solution
(A,B) < congestion*dilation
![Page 33: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/33.jpg)
33 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
![Page 34: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/34.jpg)
34 May 3, 2007
solving requirement and complexitythe guiding goal
Solving requirement:size reduction /condition number1/2 < ½
Solving complexity:O(size reduction*graph size)
![Page 35: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/35.jpg)
35 May 3, 2007
evolution of graph approximationsaka graph preconditioners [Vaidya] : MST with
[MMPRW 03]: tree T with: impractical algorithm for finding tree
[EEST 04-05]: tree T with:
T can be constructed in time
extra Steiner nodes logarithmic diameter
subtree, O(m) in general case
![Page 36: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/36.jpg)
36 May 3, 2007
evolution of preconditioningthe recent historyquestion: Can we augment T to get a smaller size reduction?
[ST 04] B = T + edges
approximation quality:
solving requirement with size reduction:
![Page 37: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/37.jpg)
37 May 3, 2007
the key in the analysis of preconditioners aka the Splitting Lemma a reduction to simpler graphs
assume and then
Ai: edges
Bi : pathsGoal : trees with low average stretch
![Page 38: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/38.jpg)
38 May 3, 2007
the key in the analysis of preconditioners aka the Splitting LemmaMonolithic preconditioners:
construct tree, add edges back (nlog n) no obvious way to parallelize motivated by the analysis “easiness”
![Page 39: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/39.jpg)
39 May 3, 2007
the key in the construction of preconditioners aka
the Splitting Lemma
Miniature
Preconditioners!
![Page 40: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/40.jpg)
40 May 3, 2007
optimal planar preconditioners
Spielman & Teng Preconditioner Factory
local mini preconditioner: component size k boundary size
approximate each Ai withBi = Ti + edges
approximation quality
S&T
![Page 41: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/41.jpg)
41 May 3, 2007
optimal planar preconditioners
Spielman & Teng Preconditioner Factory
global preconditioner: approximation quality
total number of edges
S&T
size reduction /condition number1/2 < 1/k1/2
we are done!!
![Page 42: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/42.jpg)
42 May 3, 2007
hey... great algorithm!(you ‘re just another theorist.... this will never be practical!)
Usually applications need to solve several systems with a given Laplacian. Hierarchies are constructed once.
Theorems need to be pessimistic because they have to deal with rare instances. Now we can measure the actual quality and optimize the solver.
Spend O(k2) time on each miniature preconditioner. Gremban & MMRPW factory is back in business.
The algorithm is parallel and work-efficient.
![Page 43: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/43.jpg)
43 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
![Page 44: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/44.jpg)
44 May 3, 2007
spectral perturbation theory for Laplacians
grid graph: A
split faces arbitrarily: Bwhat is the relationship
of the eigenvalues and eigenvectors of A and B?
embed B into A: (A,B) < congestion*dilation< 4
![Page 45: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/45.jpg)
45 May 3, 2007
spectral perturbation theory for Laplacians
eigenvalue decomposition of A and B
eigenvalue theorem eigenvector theorem
there are graphs with there are graphs with
can you always find a preconditioner B with
combinatorial approach for Algebraic MultiGrid Algorithms (AMG)
![Page 46: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/46.jpg)
46 May 3, 2007
sleek proofs via spectral graph theory
grid graph: A how many spanning trees ?
split faces arbitrarily: B how many more ?
By the eigenvalue perturbation:
![Page 47: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/47.jpg)
47 May 3, 2007
Outline
planar multi-way edge partitioning planar multi-way vertex partitioning
solving linear systems: introduction solving planar Laplacians
a bit of perturbation theory
we are done!!
![Page 48: 1 Algebraic and combinatorial tools for optimal multilevel algorithms Yiannis Koutis Carnegie Mellon University](https://reader035.vdocuments.us/reader035/viewer/2022062600/5a4d1bac7f8b9ab0599cb1d1/html5/thumbnails/48.jpg)
48 May 3, 2007
Thanks!