algebraic multigrid as solvers and as preconditionerta.twi.tudelft.nl/.../week1_part2_slides.pdf ·...
TRANSCRIPT
Algebraic Multigrid as Solvers and as Preconditioner
Domenico [email protected]
http://www.cs.kuleuven.ac.be/∼domenico/
Department of Computer ScienceKatholieke Universiteit Leuven
Celestijnenlaan 200 AB-3001 Heverlee
Belgium
n
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Preconditioner
M Structure of the Presentation Domenico Lahaye
Motivation for Iterative solvers
⊲⊳ 2D Eliptic partial differential equation − ∂
∂x
(ν
∂u
∂x
)− ∂
∂y
(ν
∂u
∂y
)= f
⊲⊳ Discretization by e.g. finite difference or finite elements
⊲⊳ Resulting system of linear equation A x = b
∗ A discrete differential operator∗ x and b unknown function u and f in discrete points respectively
⊲⊳ dim(A) large (acurate discretizations)
⊲⊳ A x = b has to be solved repeatedly (parameter studies)
⇒ Fast and memory efficient solution procedures required
M Motivation for Iterative solvers Domenico Lahaye
Motivation for Iterative solvers (2)
Example: Numerical models for electrical motors
⊲⊳ A = (0,0, u) B = ∇× A
H = ν B
⊲⊳ ⇒ computation of forces and torques
⊲⊳ A x = b dim(A) ∼ 105 − 106
⊲⊳ solved in time stepping oroptimization loop
M Motivation for Iterative solvers (2) Domenico Lahaye
Motivation for Iterative solvers (3)
Iterative methods
Given x0, compute xmm≥1 such that‖ em ‖=‖ x − xm ‖→ 0 if m → ∞
⊲⊳ stationary methods ek = Mke0∗ multigrid methods∗ domain decomposition methods
⊲⊳ non-stationary methods∗ Krylov subspace methods
Matrix-vector multiply as computational kernel⇒ sparsity exploited⇒ A v in O(N) flops, N = dim(A)
Direct methods
Rely on a factorization of A
M Motivation for Iterative solvers (3) Domenico Lahaye
Motivation for Iterative solvers (4)
Representative sparsity structure of A
0 20 40 60 80 100 120 140 160 180
0
20
40
60
80
100
120
140
160
180
1203
M Motivation for Iterative solvers (4) Domenico Lahaye
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Precondi-tioner
⊲⊳ Conclusions
M Structure of the Presentation Domenico Lahaye
Basic Iterative Schemes for A x = b
⊲⊳ Matrix splitting A = M − N with M−1 v “easy” to compute
⊲⊳ Iterative scheme xm+1 = M−1 N xm + M−1b (iterand)em+1 = (I − M−1 A) em (error)
⊲⊳ A = D − L − U ∗ Jacobi method MJAC = D (diagonal)∗ Gauss-Seidel method MGS = D − L (triangular)∗ Successive relaxation MSOR = 1
ωD − L ω > 0
⊲⊳ Asymptotic rate of convergence given by ρ(I − M−1 A) or ‖ I − M−1 A ‖
M Basic Iterative Schemes for A x = b Domenico Lahaye
Basic Iterative Schemes for A x = b (2)
Model Problem Analysis
⊲⊳ Continuous problem: − u = f on Ω = [0,1]× [0,1] + boundary conditions
⊲⊳ Discretization on uniform mesh, mesh width h, using central finite differences
⊲⊳ Linear system A x = b with [A] = 1h2
0 −1 0−1 4 −10 −1 0
Ωh =
M Basic Iterative Schemes for A x = b (2) Domenico Lahaye
Basic Iterative Schemes for A x = b (3)
Convergence history of lexicografic Gauss-Seidel (fixed mesh)
∗ for a fixed mesh (fixed h )initial stage: fast convergencelater stage: stalling convergence
0 10 20 30 40 50 6010
1
102
103
104
Number of iterations
Res
idua
l
M Basic Iterative Schemes for A x = b (3) Domenico Lahaye
Basic Iterative Schemes for A x = b (4)
Convergence history of lexicografic Gauss-Seidel (varrying mesh size)
∗ for finer meshes ( h → 0 )‖ I−M−1 A ‖= 1−O(h2)
⇒ increasing number ofiterations
0 10 20 30 40 50 6010
−2
10−1
100
101
102
103
104
105
h = 1/8
h = 1/16
Number of iterations
Res
idua
l h = 1/32
M Basic Iterative Schemes for A x = b (4) Domenico Lahaye
Basic Iterative Schemes for A x = b (5)
Slow convergence is caused by smooth error components
Initial error After 5 iterations After 10 iterations
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.05
0
0.05
0.1
0.15
0.2
0.25
Smooth error components can be represented on coarser grids
M Basic Iterative Schemes for A x = b (5) Domenico Lahaye
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Precondi-tioner
⊲⊳ Conclusions
M Structure of the Presentation Domenico Lahaye
Multigrid Methods
⊲⊳ exploit PDE background of the linear problem⊲⊳ multigrid methods = smoother (basic iterative scheme) + coarse grid correction⊲⊳ two-grid scheme
applypre
smoothing
projecton coarser
grid
correction
addcorrection
apply
smoothingpost
coarse gridcompute
fine gridto
interpolate
xhk xh
k+1
⊲⊳ multigrid scheme by recursive application to solve coarse grid problem
M Multigrid Methods Domenico Lahaye
Multigrid Methods (2)
Geometric multigrid methods
⊲⊳ Ah xh = bh on Ωh = ⇒ AH xH = bH on ΩH =
⊲⊳ construct restriction IHh : Ωh → ΩH interpolation Ih
H : ΩH → Ωh
⊲⊳ coarse grid correction Kh,H = Ih − IhH (AH)−1 IH
h Ah
smoother Sh = (I − (Qh)−1 Ah) Qh splitting of Ah
multigrid iteration matrix Mh,H(ν1, ν2) = (Sh2)
ν2 Kh,H (Sh1)
ν1
M Multigrid Methods (2) Domenico Lahaye
Multigrid Methods (3)
Geometric multigrid methods (2)
⊲⊳ Different orders in visiting coarser grids ⇒ different cycles (V-, W-, F-cycle)
⊲⊳ Mh,H multigrid iteration matrix‖ I − (Mh,H)−1 A ‖= C C small indepent of h⇒ mesh-width independent (h-independent) convergence
M Multigrid Methods (3) Domenico Lahaye
Multigrid Methods (4)
⊲⊳ multigrid methods require ahierarchy of grids
⊲⊳ this hierarchy can be cumber-some to construct
⊲⊳ example left: ΩH =???
M Multigrid Methods (4) Domenico Lahaye
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Precondi-tioner
⊲⊳ Conclusions
M Structure of the Presentation Domenico Lahaye
Algebraic Multigrid Methods
⊲⊳ Automatic construction of ΩH and AH xH = bH
⊲⊳ Using information contained in Ah only and no information on∗ differential equation∗ geometry of Ωh
⊲⊳ notion of strength of coupling between grid points exploited
⊲⊳ Strength of coupling influences the smoother
⊲⊳ Illustration for ∗ anisotropic diffusion equation∗ diffusion equation with discontinuous diffusion coefficient
M Algebraic Multigrid Methods Domenico Lahaye
Algebraic Multigrid Methods (2)
Anisotropic Diffusion Equation
⊲⊳ Equation −ǫ∂2u
∂x2− ∂2u
∂y2= f with ǫ ≪ 1 on [0,1] × [0,1]
⊲⊳ Finite difference discretization on Ωh Ah xh = bh with
[A] = 1h2
0 −1 0−ǫ 4 −ǫ0 −1 0
⊲⊳ weak coupling in x-direction, strong coupling in y-direction
⊲⊳ after point Gauss-Seidel smoothing, error is ∗ smooth in y-direction∗ oscilatory in x-direction
⊲⊳ coarsening feasable in y-direction only !
M Algebraic Multigrid Methods (2) Domenico Lahaye
Algebraic Multigrid Methods (3)
Initial error After 5 iterations After 10 iterations
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.3
−0.2
−0.1
0
0.1
0.2
0.3
Ωh = ⇒ ΩH =
M Algebraic Multigrid Methods (3) Domenico Lahaye
Algebraic Multigrid Methods (4)
Problem with discontinuous diffusion coefficient
⊲⊳ − ∂
∂x
(ν
∂u
∂x
)− ∂
∂y
(ν
∂u
∂y
)= f with ν =
0.0
0.375
0.625
1.0
ν = 1
ν = 106
⊲⊳ after point Gauss-Seidel smoothing, error oscilatory accros interface of discon-tinuity
⊲⊳ coarsening not feasable “close to” the interface
M Algebraic Multigrid Methods (4) Domenico Lahaye
Algebraic Multigrid Methods (5)
Initial error After 5 iterations After 10 iterations
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
1−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
10
0.05
0.1
0.15
0.2
0.25
0.3
0.35
00.2
0.40.6
0.81
0
0.2
0.4
0.6
0.8
10
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Ωh = ⇒ ΩH =
M Algebraic Multigrid Methods (5) Domenico Lahaye
Algebraic Multigrid Methods (6)
⊲⊳ geometric multigrid ∗ construct hierarchy of coarser grids∗ smoother adapted to constructed hierarchy
⇒ work invested in the smoother⇒ e.g. using line smoothers
⊲⊳ algebraic multigrid ∗ simple, fixed smoother: point Gauss-Seidel∗ coarse grid correction adapted to the local
properties of the smoother⇒ work invested in the coarse grid correction
M Algebraic Multigrid Methods (6) Domenico Lahaye
Algebraic Multigrid Methods (7)
⊲⊳ Setup phase ∗ construction of splitting Ωh = Ch ∪ Fh and set ΩH = Ch
∗ construction of interpolation IhH
∗ strength of coupling between nodes coded in Ah exploited
∗ matrix dependent interpolation: (IHh )ij ∼ Ah
ij/Ahii
∗ Galerkin coarsening: AH = IHh Ah Ih
H
∗ apply recursively using AH as input
⊲⊳ Solve phase ∗ multigrid cycling
M Algebraic Multigrid Methods (7) Domenico Lahaye
Algebraic Multigrid Methods (8)
Permanent Magnet Machine Test Case
M Algebraic Multigrid Methods (8) Domenico Lahaye
Algebraic Multigrid Methods (9)
⊲⊳ Adaptive construction of triangulation in 13 refinement steps
0 2 4 6 8 10 12
104
105
Adaptive refinement step
Nod
es in
tria
ngul
atio
n
M Algebraic Multigrid Methods (9) Domenico Lahaye
Algebraic Multigrid Methods (10)
⊲⊳ Initial and intermediate triangulation
M Algebraic Multigrid Methods (10) Domenico Lahaye
Algebraic Multigrid Methods (11)
⊲⊳ Number of iterations for AMG
0 2 4 6 8 10 120
20
40
60
80
100
120
140
160
Adaptive refinement step
Num
ber
of it
erat
ions
V(1,1)V(2,2)W(1,1)
∗ multigrid behavior
M Algebraic Multigrid Methods (11) Domenico Lahaye
Algebraic Multigrid Methods (12)
⊲⊳ CPU-time measurements for AMG
50000 100000 150000 2000000
120
240
360
480
600
720
Number of grid points
CP
U ti
me
(in s
ecs.
)
One−level solverAMG
∗ RAMG speedup byfactor 7.5
M Algebraic Multigrid Methods (12) Domenico Lahaye
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Precondi-tioner
⊲⊳ Conclusions
M Structure of the Presentation Domenico Lahaye
Krylov Subspace Methods
⊲⊳ Non-stationary iterative methods for solving A x = b
⊲⊳ Given x0 and r0 = b − A x0
find xm ∈ x0 + Km(A, r0) = spanr0, A r0, . . . , Am−1 r0
⊲⊳ Given Vm = [v1, . . . , vm] a basis for Km(A, r0) ⇒ xm = x0 + Vm ym
ym a vector of m coefficients
⊲⊳ basis Vm ∗ Arnoldi method vector ym ∗ residual projection methods∗ bi-Lanczos method ∗ residual norm minimization∗ Lanczos method
⊲⊳ No Krylov method optimal for large class of problems
M Krylov Subspace Methods Domenico Lahaye
Krylov Subspace Methods (2)
The Arnoldi Method for constructing Vm
⊲⊳ Initial vector v1 = r0/ ‖ r0 ‖
⊲⊳ Given Vm = [v1, . . . , vm] a basis for Km(A, r0)
compute A vm and orthogonalize with respect to Vm
normalize and set resulting vector equal to new basis vector vm+1
⊲⊳ recurrence relation A Vm = Vm+1 Hm+1,m Hm+1,m =
(Hm,m
0 . . .0hm+1,m
)
Hm,m upper Hessenberg
⊲⊳ long term recurrences: expensive in memory and CPU-time
M Krylov Subspace Methods (2) Domenico Lahaye
Krylov Subspace Methods (3)
The Bi-Orthogonal Lanczos Method for constructing Vm
⊲⊳ Avoid long term recurrences using bi-orthogonality
V Tm Wm = Dm = diag[d1, . . . , dm] Wm basis for Km(AT , r∗0)
⊲⊳ recurrence relation A Vm = Vm+1 Tm+1
Tm+1 tridiagonal ⇒ short term recurrences
⊲⊳ (near) breakdown if vTi wi = 0 ( vT
i wi ∼ 0 )
M Krylov Subspace Methods (3) Domenico Lahaye
Krylov Subspace Methods (4)
Determing the vector ym
⊲⊳ xm ∈ x0 + Km(A, r0) ⇒ rm = b − A xm = r0 − A Vm ym
⊲⊳ residual projection methods ⇔ V Tm rm = 0
⇔ Hm,m ym = β1e1m × m linear system: solve by LU-factorization
⊲⊳ residual minimization methods ⇔ minimize ‖ rm ‖⇔ Hm+1,m ym = β1e1
(m + 1) × m linear system: solve inleast-squares sense
M Krylov Subspace Methods (4) Domenico Lahaye
Krylov Subspace Methods (5)
A x = b with A symmetric, positive definite
⊲⊳ A = AT ⇒ Hm,m = (Hm,m)T ⇒ Hm,m tridiagonal
⊲⊳ A pos. def. ⇒ Hm,m = V Tm A Vm pos. def. ⇒ LU-fact. without pivoting
⊲⊳ update LU-fact. from step m to m + 1
⊲⊳ coupled 2-term recurrences rm−1, pm−1, rm, pm ⇒rm+1, pm+1
⊲⊳ pTi A pj = 0 for i 6= j ⇒ conjugate gradient method (CG)
M Krylov Subspace Methods (5) Domenico Lahaye
Krylov Subspace Methods (6)
Convergence of Krylov Subspace Methods
⊲⊳ Krylov subspace methods converge superlinearly
⊲⊳ Bound for CG ‖ x − xm ‖A ≤ 2
[√cond2(A)−1√cond2(A)+1
]m‖ x − x0 ‖A
⊲⊳ Preconditioning: A x = b ⇒ M−1 A x = M−1 b
∗ M−1 A ≈ I ∗ M z = r “easy” to solve
⊲⊳ Preconditioned CG: ∀m replace rm by zm = M−1 rm
⊲⊳ Possible preconditioners for CG: MJAC , MSSOR, . . .
M Krylov Subspace Methods (6) Domenico Lahaye
Krylov Subspace Methods (7)
The Convergence of CG: A closer look
⊲⊳ CG: ∀m Tm ym = β1e1 Tm = V Tm A Vm
⊲⊳ eigenvalues of Tm are called Ritz-values
⊲⊳ Ritz-values converge to spec(Am) for increasing m
fast converges towards “outliers” in spec(Am), if anycfr. Krylov subspace methods for eigenvalue computations
⊲⊳ convergence of Ritz values governs convergence of CG
M Krylov Subspace Methods (7) Domenico Lahaye
Structure of the Presentation
⊲⊳ Motivation for Iterative solvers
⊲⊳ Stationary Iterative Solvers
⊲⊳ Multigrid Methods
⊲⊳ Algebraic Multigrid Methods
⊲⊳ Krylov Subspace Methods
⊲⊳ Algebraic Multigrid Methods as Preconditioner
⊲⊳ Conclusions
M Structure of the Presentation Domenico Lahaye
AMG as Preconditioner
Problem with anti-periodic boundary conditions
Az|Γ1= −Az|Γ2
Number of iterationsAdaptive classical classical
refinement step AMG AMG/CG0 148 151 div 162 div 173 div 174 div 185 div 176 div 17
M AMG as Preconditioner Domenico Lahaye
AMG as Preconditioner (2)
convergence of the Ritz values towardsspec(I − M−1A ) during CG-iteration
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
AMG/CG Iteration step
Ritz
val
ues
of (
I − in
v(M
)*A
)
⊲⊳ spec(I − M−1A ) hasa few outliers
⊲⊳ outliers are captured fastby CG-iteration
M AMG as Preconditioner (2) Domenico Lahaye
AMG as Preconditioner (3)
Permanent Magnet Machine Test Case
M AMG as Preconditioner (3) Domenico Lahaye
AMG as Preconditioner (4)
⊲⊳ Number of iterations for AMG
0 2 4 6 8 10 120
20
40
60
80
100
120
140
160
Adaptive refinement step
Num
ber
of it
erat
ions
V(1,1)V(2,2)W(1,1)V(1,1)/CG
∗ CG accelerates andstabilizes convergence
M AMG as Preconditioner (4) Domenico Lahaye
AMG as Preconditioner (5)
⊲⊳ CPU-time measurements for AMG
50000 100000 150000 2000000
120
240
360
480
600
720
Number of grid points
CP
U ti
me
(in s
ecs.
)
SSOR/CGRAMGRAMG/CG
∗ AMG speedup byfactor 7.5
∗ AMG/CG speedup byfactor 15
M AMG as Preconditioner (5) Domenico Lahaye
Conclusions
⊲⊳ We presented multigrid and algebraic multigrid for two-dimensional diffusionproblems.
⊲⊳ Algebraic multigrid methods allow to solve problems on complicated geomet-ries.
⊲⊳ Algebraic multigrid methods are more efficient than single-level iterative solvers.
⊲⊳ Using algebraic multigrid as a precondioner improves its stability and speed.
M Conclusions Domenico Lahaye