cadd: component-averaged domain decomposition dan gordon computer science university of haifa rachel...

CADDCADD::Component-AveragedComponent-Averaged

Domain DecompositionDomain Decomposition

Dan GordonComputer ScienceUniversity of Haifa

Rachel GordonAerospace Engg.

Technion

January 13, 2008January 13, 2008 11Component-Averaged Domain DecompositionComponent-Averaged Domain Decomposition

January 13, 2008January 13, 2008 Component-Averaged Domain DecompositionComponent-Averaged Domain Decomposition 22

Outline of TalkOutline of Talk

CADD – Algebraic explanationCADD – DD explanation The Kaczmarz algorithm (KACZ) KACZ CARP (a CADD method) Applications of CARPCARP-CG: CG acceleration of CARP Sample results


CADD – Algebraic Explanation – 1CADD – Algebraic Explanation – 1 The equations are divided into blocks

(not necessarily disjoint) Initial estimate: vector x=(x1,…,xn) Suppose x1 is a variable (component

of x) that appears in 3 blocks x1 is “cloned” as y1 , z1 , t1 in the

different blocks. Perform one (or more) iterative

operation(s) on each block (independently, in parallel)


CADD – Algebraic Explanation – 2CADD – Algebraic Explanation – 2

The internal iterations in each block produce 3 new values for the clones of x1 : y1’ , z1’ , t1’

The next iterative value of x1 is

x1’ = (y1’ + z1’ + t1’)/3 The next iterate is

x’ = (x1’ , ... , xn’) Repeat iterations as needed for

convergence


CADD – non-overlapping domainsCADD – non-overlapping domains

xx xxyy

00 1111

domain Adomain A domain Bdomain B

external gridexternal gridpoint of Apoint of A

clone of clone of xx11


CADD – overlapping domainsCADD – overlapping domains

xx xxyy

00 1111

domain Adomain A

domain Bdomain B

external gridexternal gridpoint of Apoint of A

clone of clone of xx11

AA∩∩BB


CADD – Parallel ImplementationCADD – Parallel Implementation

Every processor is in charge of one domain

Mode: BLOCK-PARALLEL– All processors operate in parallel (each

on its domain)– Processors exchange clone values– All values are updated– New values = average of clones


CADDCADD Vs. Standard DD Methods Vs. Standard DD Methods

Difference is in handling external grid points:

CADD: A clone of the external grid point is modified by the domain's equations and contributes to new value of grid point.

Standard DD: Value of external grid point is fixed (contributes to RHS)


KACZKACZ: The Kaczmarz algorithm: The Kaczmarz algorithm

Iterative method, due to Kaczmarz (1937). Rediscovered for CT as ART

Basic idea: Consider the hyperplane defined by each equation

Start from an arbitrary initial point Successively project current point

onto the next hyperplane, in cyclic order


KACZ: Geometric DescriptionKACZ: Geometric Description

eq. 1

eq. 2eq. 3

initialpoint


KACZKACZ with Relaxation Parameter with Relaxation ParameterKACZ can be used with a relaxation

parameter λ

λ=1: project exactly on the

hyperplaneλ<1: project in front of hyperplane

λ>1: project beyond the hyperplane

Cyclic relaxation: Eq. i is assigned a

relaxation parameter λi


Convergence Properties of Convergence Properties of KACZKACZ

KACZ with relaxation (0< λ <2) converges for consistent systems:– Herman, Lent & Lutz, 1978

– Trummer, 1981

For inconsistent systems, KACZ converges cyclically:– Tanabe, 1971

– Eggermont, Herman & Lent, 1981 (for cyclic relaxation parameters).


Another view of Another view of KACZKACZ

Given the system Ax = b

Consider the "normal equations" system AATy = b, x=ATy

Well-known: KACZ is simply SOR applied to the normal equations

The relaxation parameter of KACZ is the usual relax. par. of SOR


CARPCARP: Component-Averaged Row Projections: Component-Averaged Row Projections

CARP is a CADD method with KACZ iterations in each domain:

The system Ax=b is divided into blocks B1,…,Bn (need not be disjoint)

Each processor is assigned one (or more) block

All the blocks are processed in parallel Results from blocks are “merged” to form

the next iterate Merging is done by CADD averaging


Overview of Overview of CARPCARP

domain Adomain A domain Bdomain B

KACZKACZiterationsiterations

KACZKACZiterationsiterations

averagingaveraging

cloningcloning

KACZKACZ in superspace in superspace(with cyclic relaxation)(with cyclic relaxation)


Convergence of CARPConvergence of CARP

Averaging Lemma: the component-

averaging and cloning operations of

CARP are equivalent to KACZ row-

projections in a certain superspace

(with λ = 1) CARP is equivalent to KACZ in the

superspace, with cyclic relaxation parameters – known to converge


"Historical" Note"Historical" Note

The term "component averaging" was first used by Censor, Gordon & Gordon (2001) for CAV and BICAV, used for image reconstruction in CT

These are Cimmino-type algorithms, with weights related to the sparsity of the system matrix

CADD, CARP use this concept in a different sense


CARPCARP Application: Solution of stiff Application: Solution of stiff linear systems from PDEslinear systems from PDEs

Elliptic PDEs w/large convection term result in stiff linear systems (large off-diagonal elements)

CARP is very robust on these systems, as compared to leading solver/preconditioner combinations

Downside: Not always efficient


CARPCARP Application: Electron Tomography Application: Electron Tomography(joint work with J.-J. Fernández)(joint work with J.-J. Fernández)

3D reconstructions: Each processor is assigned a block of consecutive slices. Data is in overlapping blobs.

The blocks are processed in parallel. The values of shared variables are

transmitted between the processors which share them, averaged, and redestributed.


CARP-CGCARP-CG: CG acceleration of : CG acceleration of CARPCARP

CARP is KACZ in some superspace (with cyclic relaxation parameter)

Björck & Elfving (BIT 79): developed CGMN, which is a (sequential) CG-acceleration of KACZ (double sweep, fixed relax. parameter)

We extended this result to allow cyclic relaxation parameters

Result: CARP-CG


CARP-CGCARP-CG: Properties: Properties

Same robustness as CARP Very significant improvement in

performance on stiff linear systems derived from elliptic PDEs

Very competitive runtime compared to leading solver/preconditioner combinations on systems derived from convection-dominated PDEs

Improved performance in ET

CARP-CGCARP-CG on PDEs on PDEs CG acceleration of projection

methods was done before, but with block projections, requiring multi-coloring. CARP-CG avoids this.

Tests were run on 9 convection-dominated PDEs, comparing CARP-CG with restarted GMRES, CGS and Bi-CGSTAB, with and without various preconditioners. Also tested: CGNR.

Domain: unit cube – 80x80x80.



512,000 equations


CARP-CGCARP-CG: ET results: ET results


Future research on Future research on CADDCADD Different internal solvers:

1. Guaranteed convergence?2. Symmetrizability?

Combine CADD with multigrid CFD applications Additional biomedical applicationsCADD with preconditioners Compare CADD with other DD

methods, such as additive Schwarz


CARPCARP Publications PublicationsCARP: SIAM J. Sci. Comput. 27

(2005) 1092-1117.

http://cs.haifa.ac.il/~gordon/carp.pdf

Electron tomography: J. Par. & Dist.

Comput. (2007), in press.

http://cs.haifa.ac.il/~gordon/eltom.pdf

CARP-CG: Submitted for publication.

cadd: component-averaged domain decomposition dan gordon computer science university of haifa rachel...

Documents