combinatorial preconditioners for finite element...

Raymond and Beverly Sackler Faculty of Exact SciencesSchool of Computer Science

Combinatorial Preconditioners for

Finite Element Problems

and

Other Contributions to Numerical

Linear Algebra

Thesis submitted for the degree of Doctor of Philosophyby

Gil Shklarski

This work was carried out under the supervision of

Prof. Sivan Toledo

Submitted to the Senate of Tel Aviv University

August 2008

To my dear wife.

Acknowledgments

First and foremost, I would like to thank my advisor, Professor SivanToledo. Working with Sivan was a truly pleasurable learning experience.Sivan has taught me the fundamentals of Numerical Linear Algebra, andbasically everything I know about academic research. Sivan is a true men-tor; he was always there with novel ideas, with persistence, and with endlessoptimism. It has been a privilege to be a student of Sivan and to shareacademic and technological ideas with him.

I wish to thank Professor Daniel Cohen-Or for the opportunity to workwith his research group and for his support. I also wish to thank my otherco-authors and collaborators: Doron Chen, Haim Avron, Andrei Sharf,Thomas Lewiner, and Miroslav Rozloznık.

Special thanks to my friends and former lab partners Dror Irony andDoron Chen for long discussions and kind advices about academic life.Additional special thanks to my lab partner and former cubical partnerHaim Avron for his optimism and friendship.

Thanks to my friends and lab partners over the years, Aviad Zuc, TuliUchitel, Eran Bida, Michal Spivak, Nir Shasha, Avi Ben Aroya, and EranGal. I wish to thank them for a friendly work atmosphere and for theirindividual friendship.

I wish to use this opportunity to express my gratitude to my high-school mathematics teacher Zari Lemel, of blessed memory, for giving methe passion for mathematics. Zari’s unique teaching experience will becherished by his students forever.

I wish to thank my dear parents and brothers for their support andencouragement over the years. Thanks to my parents for trying to push meforward constantly and from young age. This thesis is also their achieve-ment in this sense.

Last but not least, I wish to thank my dearest friend and loving wifeLiat. Liat has experienced with me all the ups and downs in a PhD studentlife. She was always there with patience, love, and help. This thesis isdedicated to her.

5

Abstract

The main theme of this thesis is the construction of combinatorial pre-conditioners for symmetric positive semidefinite (SPSD) finite element ma-trices. These preconditioners can be used to efficiently solve finite-elementlinear systems and to compute the null space of finite element problems.The preconditioners are applicable to a wide-range of finite-element matri-ces, including matrices arising from finite-element discretizations of bothscalar and vector partial differential equations (e.g., electrostatics and lin-ear elasticity).

This thesis demonstrates that combinatorial preconditioning techniquesthat were traditionally used for the class of symmetric diagonally dominantmatrices can be extended to handle the more general class of SPSD finiteelement matrices. Present combinatorial techniques that handle such ma-trices are all problem-specific or discretization-specific. Both the theoryand the algorithms in this thesis are almost purely combinatorial-algebraicand are oblivious to the details of the underlying physical problem or dis-cretization method.

The main result in this thesis is the development of foundations forcombinatorial preconditioning of finite-element matrices. In particular, itpresents a combinatorial representation for finite-element matrices calledthe rigidity graph and an algebraic-combinatorial theory that relates thisgraph to the null space of such matrices. Based on this theory, it presentsa novel combinatorial-algebraic sparsification method called the fretsaw ex-tension. This method generates preconditioners with essentially the samenull space as the original matrix. These preconditioners were used by Spiel-man and Daitch (2006) as a basis for a provably efficient linear solver fortwo dimensional triangular trusses. In this thesis, these preconditionersare applied to computing the null space of finite element models, includingmodels with equality constraints. Under certain conditions, the fretsaw ex-tension preconditioners can be factored with essentially no fill. Therefore,both the amount of computation and the amount of memory required bythis computation scale linearly with model size. In general, large indus-trial models do not satisfy all these conditions; however, experimentallythe method performs well and outperforms an established method on in-dustrial models, including models with many equality constraints. This isjoint work with Sivan Toledo.

7

Abstract 8

Matrices arising from finite-element discretizations of scalar elliptic par-tial differential equations have certain algebraic properties that can be uti-lized for constructing an effective linear solver. This thesis presents a studyof these properties and a novel solver based on them. The solver usesnovel purely algebraic methods for approximating the element matrices bysymmetric diagonally dominant ones. The resulting matrix is then approxi-mated using standard combinatorial preconditioners. When all the elementmatrices are approximable, which is often the case, the solver is provably ef-ficient. When there are inapproximable elements, the solver identifies thoseelements, and does not sparsify their part in the model. As this portionbecomes larger, this solver automatically behaves more like a direct solver.Experimental results show that on problems in which some of the elementmatrices are ill conditioned, this solver is more effective than an algebraicmultigrid solver, than an incomplete-factorization solver, and than a directsolver. This is joint work with Haim Avron, Doron Chen, and Sivan Toledo.

This thesis also presents a novel application for finite-element discretiza-tion of scalar elliptic partial differential equations. It utilizes the finiteelement discretization for the computer graphics problem of three dimen-sional surface reconstruction from noisy and partial data. The input isa set of points that are close to the surface of a three dimensional body,and some other points that are known to be inside or outside the body.The output is a surface that minimizes a specific penalty function. Theoptimization problem is formulated in a mesh-independent manner, andmapped onto a specific mesh using the finite-element method. This thesispresents an efficient construction of a linear least-squares problem to solvethis optimization problem. The finite-element formulation is the key for theefficiency of the method. It allows the construction of an effective penaltyfunction using a sparse irregular mesh. With a denser regular mesh, theunderlying linear problem would be much more expansive to solve. This isjoint work with Andrei Sharf, Thomas Lewiner, Sivan Toledo, and DanielCohen Or.

Finally, this thesis also contains an additional contribution to numericallinear algebra. It present a partitioned algorithm for reducing a symmetricmatrix to a tridiagonal form, with partial pivoting. That is, the algorithmcomputes a factorization PAP T = LTLT where P is a permutation ma-trix, L is lower triangular with a unit diagonal, and T is symmetric andtridiagonal. The algorithm is based on the column-by-column methods ofParlett and Reid and of Aasen. The partitioning allow the algorithm toexploit modern computer architectures (in particular, cache memories andhigh-performance blas libraries). Experimental results demonstrate thatthe algorithm achieves approximately the same level of performance as thepartitioned Bunch-Kaufman factorization and solve in lapack. This isjoint work with Miroslav Rozloznık and Sivan Toledo.

Contents

Acknowledgments 5

Abstract 7

Chapter 1. Introduction 111.1. Brief Background 121.2. Rigidity in Finite Element Matrices 161.3. The Null Space of Finite Element Problems 221.4. Scalar Elliptic Finite-Element Models 251.5. Finite-Element Formulation for Surface Reconstruction 291.6. Partitioned Triangular Tridiagonalization 32

Chapter 2. Rigidity in Finite-Element Matrices: Sufficient Conditionsfor the Rigidity of Structures and Substructures 36

2.1. Introduction 362.2. The Essential Null Space of a Matrix 392.3. Rigidity Relationships 402.4. Rigidity of Sums 452.5. Null-Space Compatibility 482.6. The Rigidity Graph 532.7. Three Case Studies 552.8. Rigid Sparsifications 602.9. Numerical Examples 752.10. Concluding Remarks 77

Chapter 3. Computing the Null Space of Finite Element Problems 783.1. Introduction 783.2. Overview and Related Work 793.3. The Method in Detail 813.4. Numerical Results 933.5. Conclusions 101

Chapter 4. Combinatorial Preconditioners for Scalar EllipticFinite-Element Problems 102

4.1. Introduction 1024.2. Nearly-Optimal Element-by-Element Approximations 1054.3. Scaling and Assembling Element Approximations 1174.4. Sparsification of the Assembled SDD Approximation 1184.5. Dealing with Inapproximable Elements 120

9

Contents 10

4.6. Asymptotic Complexity Issues 1214.7. Experimental Results 1224.8. Open Problems 132

Chapter 5. Finite-Element Formulation for Surface Reconstruction 1345.1. Introduction 1345.2. Our Approach 1365.3. Constrained FEM Reconstruction 1375.4. Technical Details 1415.5. Results 1435.6. Conclusions 143

Chapter 6. Partitioned Triangular Tridiagonalization 1466.1. Introduction 1466.2. A Variant of Aasen’s Method 1476.3. Our Partitioned Method 1496.4. Implementation 1536.5. Experimental Results 1556.6. Concluding Remarks 157

Bibliography 158

CHAPTER 1

Introduction

Numerical linear algebra plays a central role in many scientific andengineering applications. A typical application starts with a mathematicalmodel of a problem. If the model is continuous, it must first be discretizedso that it can be represented in a computer. Then a solver is applied tothe problem; the solver may simulate a time dependent system, or find thesolution of an optimization problem, for example. Many solvers reduce theproblem to a set or a sequence of simpler problems in linear algebra, such assolving linear systems of equations, solving linear least squares problems,or computing the spectrum of a matrix. An efficient and stable solver forthese linear-algebra subproblems is often a key component in the efficientand accurate solution of the overall problem.

Solving linear systems of equations is a core problem in numerical linearalgebra. Linear systems of equations can be categorized into two majorkinds: dense and sparse. In sparse systems, the coefficient matrix containsa considerable number of zero elements. In dense systems the coefficientmatrix contains too few zeros to be worth exploiting. Sparse and densesolvers are related to each other; a modern sparse solver typically uses adense solver as a subroutine for processing small dense matrix blocks.

There are two techniques for solving both sparse and dense equations:direct and iterative. Direct solvers factor the coefficient matrix into a prod-uct of simpler matrices whose inverse is easy to apply. Iterative solvers startwith an approximate solution of the equations and try iteratively to refineit. Iterative and direct solvers are also related to each other; in many cases,the iterative solution process uses a factored approximation of the coeffi-cient matrix. A brief description of these techniques appears later in theintroduction.

In my research I have worked on many of these interrelated methods.Chapter 5 describes a mathematical model of a problem in computer graph-ics and how the model was discretized. Chapters 2, 3, and 4, which presentthe main theme of this thesis, describe techniques for accelerating sparseiterative solvers. In research that I carried out before starting the PhDprogram, I worked on sparse direct solvers. In [12] I participated in theoptimization and performance evaluation of an unsymmetric sparse directsolver. In [90] I developed a parallel sparse solver for symmetric positivedefinite matrices. This solver uses a high performance dense solver thatwas also developed as part of that research. These contributions are notincluded in this thesis. Finally, Chapter 6 presents my recent work on high

11

1.1. Brief Background 12

performance dense solvers: a novel factorization algorithm for symmetricindefinite matrices with state-of-the-art performance.

The rest of this chapter surveys the results in this thesis. It is aimedto be self-contained; it starts with an informal background, and continueswith descriptions of the actual results, with each section in the introductionsummarizing a chapter in the thesis.

1.1. Brief Background

1.1.1. Finite Element Matrices. The main theme of this thesis isthe efficient solution of linear equations Kx = f , where K and f aregenerated by the finite-element method. A rigorous formulation of thefinite element method is beyond the scope of this introduction. Moreover,most of the results in this thesis do not require the development of exactfinite element formulation, only some general algebraic properties. In thefollowing section we describe those fairly simple properties.

The matrix K is called a stiffness matrix or sometimes a finite-elementmatrix. It is typically a sum K =

∑eKe of sparser matrices Ke called

element matrices. The vector f is typically a vector of forces or currents.Figure 1.1.1 displays an example for such K and f . The figure depicts apiece of two dimensional material divided into elements using a triangu-lar mesh. The same geometric representation can stand for two differentphysical and algebraic problems: (1) A conductive material: Each meshnode corresponds to a single variable representing electrostatic potential.The vector f represents external currents forced into the mesh nodes. Thesolution vector x is the potential created in the mesh nodes under thisenforced current. In other words, x is the potential difference needed togenerate internal currents to match the external currents f . (2) An elasticmaterial: Each mesh node corresponds to two displacement variables. Thevector f represents forces applied on the mesh nodes (two components ateach node). The solution vector x represents the deformation caused bythe force. In other words, x is the displacement needed to generate internalforces in the material to match the external forces f .

The element matrices Ke are almost all zeros except for a boundednumber of nonzero rows and columns. In this thesis we focus on caseswhere the Ke’s are symmetric positive semidefinite (SPSD). Typically, theKe’s have an additional property that we call null-space compatibility. Weexplain this property later. Figure 1.1.2 illustrates the element matrices ofa simple three-element model and their assembly into the stiffness matrixK.

The question that we asked ourselves was “Can we utilize the specialstructure and algebraic properties of finite element matrices to constructan efficient solver for this class of matrices?”.

1.1.2. Solution Methods. There are essentially two major categoriesof solvers for the linear system Kx = f : direct solvers and iterative solvers.


Figure 1.1.1. A piece of two dimensional material dividedinto elements using triangular mesh. Each triangle corre-sponds to a matrix Ke, and the entire material correspondsto K. The mesh nodes correspond to the vector x, and thered arrows correspond to the vector f .

(a) (b)

Figure 1.1.2. Linear elastic triangles in two dimensions.Each element matrix is SPSD, and contains 36 nonzeros in 6nonzero rows and columns. Part (a) illustrates the model andthe variables corresponding to each mesh point, and part (b)shows the corresponding element matrices Ke and theirsum K. Each element matrix is colored in the color of itscorresponding element in the model. Its nonzero rows andcolumns match the variables that its corresponding elementis incident to.

Direct solvers factor the coefficient matrix K into the product of simplematrices. In our case, K is symmetric and positive definite, therefore Kcan be Cholesky factored to the product of a lower-triangular matrix Land its transposed upper-triangular matrix LT . In other cases, LDLT

factorization or LU with partial pivoting factorizations can be applied.After K is factored, a low cost process of forward and backward substitu-tion gives the solution. Sparse direct methods are robust, their runningtime can be bounded using the sparsity pattern of the matrix. Thereare efficient libraries that implement such methods for sparse matrices(e.g., cholmod [48], taucs [90], mumps [4, 5], pspases [73, 92], umf-pack [47], SuperLu [49, 104]). The problem with sparse direct methodsis that in many cases storing the factors requires too much memory, andthat their running time scales poorly. This is demonstrated in the experi-mental section of Chapter 4.


Iterative solvers start with an approximate solution, x(0), perhaps x(0) =0, and then try iteratively to improve it. A series of solutions x(1), x(2), ...is generated, each x(i) is hopefully an improvement of the preceding so-lution x(i−1). The solver stops at step s, when Kx(s) is close enough tof . The common basic solvers are Conjugate Gradients (cg [41, 80]) forSPSD matrices, Minimum Residual (minres [119]) for symmetric matri-ces, and Generalized Minimal Residual (gmres [133]) for general matrices.In general, without additional measures, these iterative solvers can performpoorly. If K is not symmetric, there is even no theoretical bound on thenumber of iterations of these iterative solvers. If K is symmetric and pos-itive definite, the number of iterations (of cg and minres) is bounded by

the ratio√λmax(K)/λmin(K). This ratio is a worst-case Chebyshev-like

bound hiding the nonlinear character of the convergence developed in [94].Nevertheless, for many classes of matrices, additional techniques have beendeveloped, creating an iterative solution process that is robust and signifi-cantly more efficient than direct methods. Some of them are covered in thenext section.

1.1.3. Combinatorial Preconditioning. An easy-to-invert approx-imation of K, denoted by M and called a preconditioner, can dramaticallyimprove the convergence rate of iterative solvers. The idea of precondi-tioning appears in papers as early as [150, 13, 14]. This paradigm was asubject for an extensive research starting the late 1970’s [113]. Over theyears many kinds of preconditioners have been proposed, such as incompletefactorization preconditioners [154, 151, 116, 110, 157, 130, 2, 156, 118,134, 34, 75, 39, 65], multigrid [132, 29, 30] and domain-decompositionpreconditioners [140, 109], sparse approximate inverses [42, 17, 71] andfactored sparse approximate inverses [31, 17, 99].

A common way to use the preconditioner M is to implicitly solve theequation M−1Kx = M−1f instead of the equation Kx = f . In order tohave an efficient solver, there are three requirements from M :

(1) The construction of M from K is efficient.(2) There is a small number of iterations. The number of iterations of

the conjugate gradient method is bounded by the rate√κ(M−1K),

where for a matrix A, κ(A) is the ratio its extreme eigenvaluesκ(A) = λmax(A)/λmin(A).

(3) Applying M−1 is easy, i.e, solving the equation Mz = r is easy.One common way to solve this equation is to factor M using asparse Cholesky factorization.

These requirements are contradicting; setting M = K perfectly satisfies thefirst and second requirements, but contradicts the third one; setting M = Iperfectly satisfies the first and the third requirements, but contradicts thesecond one. Therefore, a good preconditioner should balance between theserequirements.


K G(K) G(M) M

Figure 1.1.3. A construction of a combinatorial precondi-tioner for an SDD matrix. This preconditioner is Vaidya-like preconditioner, sparsifying G(K) into its spanning treeG(M). The matrices K and M are illustrated as networks ofresistors, since SDD matrices with negative off-diagonals canrepresent resistor networks. If every node in the network inK’s illustration had a grounded battery connected to it withpotentials prescribed in x, then Kx is the directed currententering those nodes from the batteries. The same holds forM .

We make the technical note that for singular K and M , the eigenvaluesbound translates to a bound on the set of finite generalized eigenvaluesΛ(K,M) = λ|∃x s.t. Kx = λMx and Mx 6= 0 [18].

We focus on a relatively recent preconditioning technique called com-binatorial preconditioning (and support preconditioning in some cases). Incombinatorial preconditioning, the input matrix K is given a combinatorialrepresentation G(K) (typically a graph or a hyper-graph). Then, a combi-natorial algorithm creates a new combinatorial representation called G(M).Finally, the preconditioner M is constructed from G(M). Figure 1.1.3 il-lustrates this process.

The first family of combinatorial preconditioners was proposed by Vaidyain the early 1990’s [145]. Vaidya’s preconditioners work only for symmetricdiagonally-dominant matrices with negative off-diagonals. His algorithmessentially converts the matrix into a weighted graph; K is a weighted ad-jacency matrix of G(K). Vaidya’s algorithm then drops edges from thisgraph. Vaidya was able to construct provably-good preconditioners for aclass of matrices; the construction algorithm is purely combinatorial andconvergence-rate analysis is partially algebraic but mostly combinatorial(using graph embeddings). The complexity bound does not depend on thecondition number of the input matrix.

In the last decade, Vaidya’s theoretical work has been extended signifi-cantly. Chen and Toledo conducted an experimental evaluation of Vaidya’spreconditioners [37]. Boman, Chen, Hendrickson and Toledo generalizedVaidya’s preconditioners to all symmetric diagonally-dominant matrices [22].

1.2. Rigidity in Finite Element Matrices 16

Boman and Hendrickson proposed a general algebraic framework for ana-lyzing the condition number of combinatorial preconditioners [24] calledsupport theory. Gremban and Miller proposed combinatorial precondition-ers in which M has more rows and columns than K [70, 69]. Maggset al. showed how to construct a sophisticated version of Gremban andMiller-like preconditioner [108]. Guattery [72] and Bern et al. [18] usedsupport theory to bound the condition number of some incomplete factor-izations preconditioners. Howle and Vavasis generalized the precondition-ers of Gremban and Miller to complex systems [83]. Reif proposed andanalyzed Vaidya-like preconditioners, including recursive variants [127].Spielman and Teng [141, 142] improved Vaidya’s preconditioners, mainlyby augmenting a sophisticated spanning-tree construction algorithm withextra edges, and showed how to solve SDD systems in nearly-linear time.Koutis and Miller [100] showed how to solve an SDD system whose graphis planar in linear time and also efficiently in parallel.

From 1991 to 2004, combinatorial preconditioners were only applied tosymmetric diagonally-dominant (SDD) matrices, which do not arise fre-quently in practice. In 2004, both Boman, Hendrickson, and Vavasis [25]and Gupta [74], showed how to apply the combinatorial-preconditioningparadigm to matrices originating from finite-element discretizations of scalarelliptic PDEs. Their techniques apply only for scalar elliptic PDEs. More-over, their works are problem-specific and discretization-specific and cannot be directly applied to more general finite-element models. Thus, theapplication of combinatorial preconditioning to finite-element matrices wasvery limited.

This thesis extends combinatorial preconditioning to finite-element ma-trices arising from both vector and scalar problems. The preconditionersare constructed using algebraic information without problem-specific ordiscretization-specific information.

1.2. Rigidity in Finite Element Matrices

1.2.1. Algebraic theory of rigidity. In Chapter 2, which is basedon a paper that was published in the SIAM Journal on Matrix Analysisand Applications [138], we present an algebraic theory of rigidity for finite-element matrices. This theory serves as a foundation for combinatorialpreconditioning of finite-element matrices.

Figure 1.2.1 illustrates three finite-element models. The model in part(a) marked with K is our original finite-element model. Typically, in com-binatorial preconditioning, the preconditioner M is a sparsified version ofK. Parts (b) and (c) of the figure illustrate two candidate preconditionersM1 and M2. They are similar to K, with some of the elements dropped.

The first question we can ask ourselves is whether one of these candi-dates is fundamentally better than the other. Bounded generalized eigen-values λ(K,M) implies that null(K) = null(M). The geometry illustratedin the figure does not contain enough information to verify this condition.


K M1 M2

Figure 1.2.1. Triangles on the plain. The original finite-element model K and two candidate preconditioners M1 andM2 each constructed by removing elements from the model.

It depends on the physics. In the case of an electrostatic problem, since inboth M1 and M2 the material is fully connected, the only null vector is aconstant vector of potentials. In the case of a linear elasticity problem, thenull spaces are different; in M1 there are only translations and rotations onthe plane, whereas in M2 there is an additional free move around the jointcircled in red. This example shows that geometry alone, or connectivityalone is not enough for analyzing the null-space and certainly not enoughfor constructing a preconditioner.

In our framework we do not want to use explicit physical or geometricalinformation. Using such information will make our algorithms less generaland depend on specific finite-element formulations. The only input for ouralgorithms is the set Ke of element matrices. We handle the null spacequestion in the following way:

(1) We use linear algebra to define relationships between element ma-trices with common nonzero columns.

(2) We use these local relationships to form a graph.(3) We use combinatorial properties of the graph to determine linear

algebraic properties of the assembled matrix K.

We start with an informal definition of the rigidity relationship. Considerelements e and f in a finite-element mesh. Element e is rigid with respectto f if every rigid-body motion of e alone can be extended into exactly onerigid-body motion of e and f together. Figure 1.2.2 illustrates this informaldefinition.

In order to define this relationship formally, we study the null space ofa typical element matrix Ke.

Definition 1.2.1. Let Ne ⊆ 1, . . . , n be the nonzero columns of amatrix Ke with n columns. The trivial null space of Ke is

tnull(Ke) = x|Kex = 0 and xi = 0 for all i /∈ Ne ,and the essential null space of Ke is

enull(Ke) = x|xi = 0 for all i ∈ Ne .


(a) (b)

Figure 1.2.2. Two pairs of elastic triangles on the plane.The two elements in (a) are rigid with respect to each other.The two elements in (b) are not rigid with respect to eachother, since there is a free motion around the hinge in theupper right corner of the structure.

Clearly, the essential and trivial null spaces of a matrix are orthogonaland their union spans the null space of the matrix. Now, we are ready toformally define the rigidity relationship.

Definition 1.2.2. The matrix Ke is rigid with respect to Kf if for everyx ∈ enull(Ke) there is a unique y ∈ enull(Kf ) such that x and y coincideon Ne ∩ Nf . The matrices Ke and Kf are mutually rigid if they are rigidwith respect to each other. Mutual rigidity is marked with Ke ./ Kf .

We thoroughly study this relationship in Chapter 2. Note that its formaldefinition is for general matrices, not necessarily positive semi definite oreven symmetric ones. Mutual rigidity can be tested in a pure algebraic wayand with only local computations. This process is explained theoreticallyin Section 2.3, and a practical routine is given in Section 3.3.1.

We need an additional algebraic tool before we can define a graph basedon this rigidity relationship. This tool is essential for almost all the follow-ing theoretical results. The reason for that is explored in the text itself.

Definition 1.2.3. Let S ⊂ Rn be a linear space. An n-by-n matrixK is S-compatible if every x ∈ enull(K) has a unique extension to y ∈ S(y coincides with x on NK) and if for every y ∈ S, y with all coefficientsoutside NK zeroed is in enull(K).

If K is S-compatible and NK = 1, . . . , n, then we say that K is rigid(in elasticity: has no mechanisms; in electrostatics: connected; etc.).

Figure 1.2.3 illustrates the notion of null space compatibility. Let S ⊂R10 be the linear space of translations and rotations in two dimensionalelasticity with the 5 points on the plane that are illustrated in this figure.The rank of S is 3. The essential null space of the models (a), (b) and(c) are all of rank 3. Their essential null spaces contains translations androtations and no other movements. Therefore, they are all S-compatible.The essential null space of the model (d) is of rank 4 and contains a rotationaround the upper-right joint, this rotation can not be extended into a vectorwhich is a linear combination of translations and rotations of the entireplane. Therefore, model (d) is not S-compatible.


(a) (b) (c) (d)

Figure 1.2.3. Four different models of linear elastic trian-gles on the plane. Each corresponding matrix is 10-by-10.For S ⊂ R10 that contains the translations and rotationsof the plane, models (a), (b) and (c) are S-compatible, andmodel (d) is not S-compatible.

(a) (b)

Figure 1.2.4. Rigidity graph of triangles on the plane. Part(a) shows the rigidity graph in case each triangle representsan elastic triangle on the plane. Part (b) shows the rigiditygraph in case each triangle represents a conductive material.

Now we are ready to define our main combinatorial structure.

Definition 1.2.4. Let K1, K2, . . . , Kk be S-compatible symmetric pos-itive semidefinite n-by-n matrices for some null space S. The rigidity graphG = (V,E) of K1, . . . , Kk is the undirected graph with V = K1, . . . , Kkand

E = (Ke, Kf ) : Ke and Kf are mutually rigid .Figure 1.2.4 illustrates the rigidity graphs of two collections of element

matrices representing two dimensional triangles on the plane. The set (a)is made of elastic triangles and the set (b) is made of conductive triangles.

The following theorems are the main combinatorial results on rigiditygraphs. The first theorem relates combinatorial connectivity to rigidity.

Theorem 1.2.5. Let G be the rigidity graph of a collection K1, K2, . . . , Kk

of S-compatible symmetric positive semidefinite n-by-n matrices. Let K =∑ke=1 Ke. If G is connected, then K is S-compatible. In particular, if⋃ke=1NKe = 1, . . . n, then null(K) = S and K is rigid.

We note that the connectivity of the rigidity graph is only a sufficientcondition for rigidity and in general not a necessary one. Section 2.7 demon-strates this issue. The second theorem relates connectivity of subgraphs tomutual rigidity.


Theorem 1.2.6. Let G be the rigidity graph of a collection K1, K2, . . . , Kk

of S-compatible symmetric positive semidefinite n-by-n matrices. Let H1

and H2 be two connected subgraphs of G that share a vertex Ke. ThenB =

∑Kf∈V (H1) Kf and C =

∑Kf∈V (H2) Kf are mutually rigid.

1.2.2. The Fretsaw Preconditioner. The theory presented in theprevious section gives sufficient conditions for testing rigidity of a structureor a substructure. We now want to use it for preconditioning K. A naturalapproach is to use a matrix assembled from a subset of the original elements.This translates to dropping vertices from the rigidity graph. The sufficientconditions for rigidity can help to determine if we got a preconditioner withthe same null space. Nevertheless, we could not find a way to assure thatthese kinds of preconditioners are easy to factor.

Instead of dropping vertices from the rigidity graph we decided todrop edges. We call the resulting matrices fretsaw extensions. We givehere an informal description of a variant of the fretsaw extension, calledfretsaw-tree extension. Figure 1.2.2 demonstrates this construction for atwo-dimensional elastic material lying on the plane. Consider the modelillustrated in Figure 1.2.2.(a). We compute its rigidity graph G that isillustrated in blue. We Compute a spanning tree T of G. Figure 1.2.2.(b)illustrates such a spanning tree in yellow. Now, we remove edges betweenadjacent elements where there is no edge of T . We do that by adding newslack variables; we do not change the content of the element matrices, wejust augment the matrices with new rows and columns (slack variables) andcarefully switch existing variables with new ones. Figure 1.2.6 illustratesvariable duplication on a simple model. Figure 1.2.2.(c) shows the resultingmodel after duplication. In the illustration, the triangles have been slightlyshrunk to show the places where slits were cut. Now, we assemble the newstructure into a preconditioner F(K) for the original matrix. The pro-cess is purely combinatorial-algebraic and do not use explicit geometricalinformation.

The preconditioning step in iterative solvers involves solving an ordern linear system Mz = r, but our preconditioner F(K) has order m, wherem > n. To use this preconditioner, we augment r with zeros. Denote theaugmented r by r. Then we solve the linear system F(K)z = r. We restrictthe solution z to the original unknowns to form a trial solution z. It can beshown that such preconditioning is equivalent to preconditioning with theSchur complement Schur(F(K)) of the preconditioner, after eliminating theduplicated unknowns (the slack variables added in the process) [70, 24, 18].

We do not give here the formal definition of the fretsaw extension. Wenote that it is defined for rigidity graphs that are not necessarily connectedlike in the fretsaw-tree case. Moreover, the rigidity graph of a fretsawextension is not limited to a tree or to an acyclic structure.

The main theoretical results about fretsaw extensions are:


(a) (b) (c)

Figure 1.2.5. The construction of a spanning-tree fretsawpreconditioner.

Figure 1.2.6. Detaching the rigidity edge between the twoleft elastic triangles. We augment all the 10-by-10 elementmatrices with two zero rows and columns (add x11, x12 asslack variables). We switch rows and columns 1 and 2 in theupper-left element matrix with the new rows and columns 11and 12.

Theorem 1.2.7. Let K =∑k

i=1Ki, where K1, K2, . . . , Kk is a collectionof S-compatible symmetric positive semidefinite n-by-n matrices. Let F(K)be a fretsaw extension of K. Assume that we added ` slack variables addedin the creation of F(K) (i.e., F(K) is an (n+ `)-by-(n+ `) matrix). Let Qbe the (n+`)-by-n identity matrix (the first n columns of an (n+`)-by-(n+`)identity matrix), and let Schur(F(K)) be the Schur complement of F(K)after eliminating the new `slack variables. Assume that NK = 1, . . . , n.If the rigidity graph of K1, K2, . . . , Kk is connected, then

(1) QKQT and F(K) are mutually rigid.(2) null(F(K)) ⊆ null(QKQT ).(3) If Schur(F(K)) exists, then null(K) = null(Schur(F(K))).

This theorem can be extended to the case of a rigidity graph that is notconnected, when using a certain fretsaw-extension construction. We omitits formal statement from this section.

1.3. The Null Space of Finite Element Problems 22

The next result states that a fretsaw preconditioner based on a tree canbe factored with no fill and therefore in linear time. The exact construc-tion is detailed in Section 2.8.3. Informally, it is the process illustrated inFigure .

Lemma 1.2.8. Let K1, K2, . . . , Kk be a collection of S-compatible sym-metric positive semidefinite n-by-n matrices. Let F(K) be a spanning-tree

fretsaw extension of K =∑k

i=1Ki. There is a permutation matrix Γ suchthat all the nonzeros Li,j of the Cholesky factor L of Γ(F(K)+I)ΓT satisfyi, j ∈ NΓQeKeQT

e ΓT for some e (e depends on i and j).

In order to satisfy all the requirements we listed for efficient precon-ditioning we need a bound on λ(K, Schur(F(K))). The following lemmashows that λ(K, Schur(F(K))) ≥ 1.

Lemma 1.2.9. Let K1, K2, . . . , Kk be a collection of S-compatible sym-metric positive semidefinite n-by-n matrices. Let K =

∑ki=1Ki. Let F(K)

be a fretsaw extension of K. Then if Schur(F(K)) exists, and λ is a finitegeneralized eigenvalue of the pencil (K, Schur(F(K))), then λ ≥ 1.

We do not have a general upper bound on λ(K, Schur(F(K))). In com-binatorial preconditioning of SDD matrices, this is usually achieved usinga “path lemma” [18, 45], bounding this eigenvalue using path embeddingtechniques. We have not found a path lemma for a general S.

We have experimented with elastic and conductive two and three dimen-sional models. Our experimental results showed a steady linear convergencebehavior in both preconditioned cg and minres with fretsaw-trees. This issimilar to the convergence behavior of Vaidya’s preconditioners on weightedLaplacians [37]. The results were sometimes comparable and sometimesslightly better than incomplete-Cholesky preconditioner with no-fill.

In [45] Spielman and Daitch have developed a path lemma for a spe-cific case. They studied finite-element matrices that represent two dimen-sional planar trusses made of triangles with some geometric constraints.They have used our fretsaw-extension framework with trees augmentedwith some edges [53]. Their solver is provably efficient and runs withO(n5/4(log2 n log log n)3/4 log 1

ε) operations, when n is the number of joints

in the truss and ε is the prescribed relative error. A path lemma for thethree dimensional case is still missing.

Without a general theoretical bound on λ(K, Schur(F(K))) we decidedto not further pursue this direction for standard preconditioning. The nextsection shows that fretsaw-extension can be useful for a different problem,even without this bound.

1.3. The Null Space of Finite Element Problems

1.3.1. Linear Constraints. When all the Ke’s are compatible with anonempty null space S, the n-by-n matrix K =

∑eKe is singular. Even if

the corresponding rigidity graph is connected, the vectors in S are all null


vectors of K. For example, in linear elasticity even if the rigidity graph isconnected, and therefore the structure is rigid, it can still float in space.The space translation and rotation vectors are in the null space of K.

Typically, this is not a desirable property. The equation Kx = f has noguaranteed single solution. This is often solved by introducing a c-by-n ma-trix C, and a c-vector d, that represent linear constraints that the solutionx must satisfy, namely Cx = d. The matrix C models boundary conditions,in linear elasticity it can model welds, hinges, sliding mechanism, etc. Thiskind of constraints are used by NASTRAN [76].

1.3.2. Computing the null space. In Chapter 3, whose main resultswere submitted for publication in Computer Methods in Applied Mechanicsand Engineering [137], we present a solver that computes the null space

of K and of KC =[KT CT

]T. This is not as useful as solving the linear

system, but an efficient implementation of it is useful:

(1) It can be used to determine the rank of KC . If deficient, a linearsolver can break down or fail to converge.

(2) It can be used in static and dynamic analysis of floating struc-tures [62], for example, where the computed null space is used tosolve a consistent but rank-deficient system of linear equations.

(3) It can be used to construct operators in smooth aggregation [84,148, 149]; it might also be useful in FETI [57, 55] and relateddomain-decomposition methods [109].

The problem of null space computation of the stiffness matrix of finite-element models was studied in [56] and in [120]. In both cases additionalproblem-specific geometric or physical information was used. We do it in apurely algebraic-combinatorial way.

The basic idea is that we use the fretsaw extension to generate a matrixwith essentially the same null space as the original matrix that can befactored very quickly. Then, we can efficiently compute the null space usingrectangular inverse iteration [68, 89]. This involves factoring the (possiblyrectangular) matrix and preforming several iterations with its LU factors.

Given K in its elemental form and (possibly an empty) C we computea fretsaw extension F(K) of K . The rigidity graph of K might not be con-nected. The specific fretsaw extension we use here is based on a spanningforest of the rigidity graph. We do that using an algorithm that indepen-dently computes a tree fretsaw extension for every connected component.We call such an extension a fretsaw-forest extension. Figure 1.3.1 presentssuch an extension. Then we pad C with the appropriate number of columns(corresponding to the new slack variables in F(K)) and compute the null

space N of [F(K)C 0

].


(a) (b)

Figure 1.3.1. An example of a fretsaw-forest construction.The rigidity graph is illustrated in yellow, and its spanningforest in red. The original structure in part (a) was extendedinto the fretsaw-forest extension in part (b). Detailed expla-nation appears in Chapter 3.

The following lemma shows that in order to compute the null space ofKC we just remove the rows corresponding to the new variables in N .

Lemma 1.3.1. Let K =∑

eKe, where Ke is a collection of symmetricpositive semidefinite n-by-n matrices compatible with some S, and let C bea c-by-n matrix. Let F(K) be a fretsaw-forest extension of K. Assumethat we added ` slack variables in the creation of F(K) (i.e., F(K) is an(n+`)-by-(n+`) matrix). Let Q be the (n+`)-by-n identity matrix, and let

N be an (n+ `)-by-s matrix whose columns form a basis for null(

[F(K)CQT

]).

Then, span(QT N) = null(KC).

In many cases, due to numerical issues, the rectangular inverse iterationgenerates a linear space N which contains the null space of F(K) mixedwith additional vectors. As long as the computed null space is not toolarge, this can be solved easily, as shown by the next lemma.

Lemma 1.3.2. For the above K, F(K), Q and C, let N be an (n+`)-by-s

matrix whose columns span a linear space that contains null(

[F(K)CQT

]). Let

V be a matrix, such that span(V ) = null(KCQT N), then span(QT NV ) =

null(KC).

1.3.3. Running Times and Experimental Results. Chapter 3 presentsin detail an algorithm that computes the fretsaw-forest extension in lin-ear time. If the rigidity graph of Ke is connected and C contains onlysingle-unknown equality constraints, we are guaranteed that the square andsymmetric F(K) with the appended CQT can be factored without fill (it isactually a fretsaw-tree extension). This means that in this case the entireprocess can run in linear time. This was also experimentally demonstratedin section 3.4.1.

In typical industrial models C contains multipoint constraints, and therigidity graph might also not be connected. This means that we do not havea theoretical bound on the size of the LU factors and on the running time

1.4. Scalar Elliptic Finite-Element Models 25

of this method. Nevertheless, the existing body of theory suggests that theLU factorization of forest-based fretsaw extension should be cheap relativeto the original finite element model. Our experimental results indicate thatin spite of the lack of complete theoretical analysis, the LU factorization

of

[F(K)CQT

]is indeed inexpensive.

Chapter 3 contains detailed experimental results of our method runningon synthetic simple models and on real industrial models. We compare ourmethod to a direct method that perform rectangular inverse iteration onthe original matrix KC . The experiments indicate that our method clearlyoutperforms the direct method.

We also examined the accuracy of the computed null spaces in bothmethods. We computed approximate relative residuals; for a given nonempty null spaceN we examined the value ‖KCN‖/maxi,j |KC(i, j)|. With-out C and when there was a non empty null space with C our method losesbetween 2 to 5 digits of accuracy compared to the direct method. WhenC made KC non singular, our method almost always correctly detected it.Some problems were ill conditioned to an extent that made rank determi-nation difficult (both in our method and in the direct method).

1.4. Scalar Elliptic Finite-Element Models

1.4.1. SDD Approximation Approach. Our next method for pre-conditioning finite-element matrices works on matrices arising from scalarelliptic partial differential equations. We state here the continuous mathe-matical problem, although our work does not use it directly. Given a do-main Ω which is a bounded open set in Rd , with a boundary Γ = Γ1 ] Γ2,find u : Ω −→ R satisfying

(1.4.1)∇ · (θ(z)∇u) = −f on Ω

u = u0 on Γ1 ,θ(z)∂u/∂n = g on Γ2 .

The conductivity θ is a spatially varying d-by-d symmetric positive definitematrix, f is a scalar forcing function, u0 is a Dirichlet boundary conditionand g is a Neumann boundary condition. Figure 1.4.1(a) illustrates such adomain.

Figure 1.4.1 illustrate a preconditioning approach for such matrices.The finite element matrix K is approximated by a symmetric diagonallydominant matrix L. Then, a standard combinatorial preconditioner M forSDD matrices is used to approximate L. There are two algebraic observa-tions that make this approach useful. The first is the following inequalityfrom [24],

λmax(K,M)

λmin(K,M)≤ λmax(K,L)

λmin(K,L)· λmax(L,M)

λmin(L,M).

This inequality states that if L is a good preconditioner for K and M isa good preconditioner for L, then M is a good preconditioner for K. As


PDE K L M

(a) (b) (c) (d)

Figure 1.4.1. Illustration of the SDD approximation ap-proach. Part (a) illustrates the original scalar elliptic partialdifferential equation. Part (b) illustrates its finite-elementdiscretization; each triangle corresponds to a single Ke, andeach node represents an unknown. The matrix K =

∑eKe.

Part (c) illustrates an SDD approximation L of K. Each tri-angle of resistors corresponds to Le and L =

∑e Le. Part (d)

illustrates M , a preconditioner of L.

explained in Section 1.1.3, there are good combinatorial preconditionersfor SDD matrices. Moreover, SDD matrices are also preconditioned wellusing algebraic multigrid methods [132]. Basically, we only need to findan SDD matrix L that is a good preconditioner for K to get an efficientsolver. The second algebraic observation allows us to do exactly that. Thesplitting lemma [18] shows that the preconditioner L can be constructedelement-by-element. For K =

∑e∈EKe and L =

∑e∈E Le, the lemma gives

the following bounds

λmax(K,L) ≤ maxe∈E

λmax(Ke, Le)

λmin(K,L) ≥ mine∈E

λmin(Ke, Le) .(1.4.2)

These two observations, although not necessarily in this exact formulation,served as a basis for several works. The first group of works by Reitzingeret al. [77, 128, 102] have used complicated and rather expansive algo-rithms to approximate the Ke’s by Le’s. Moreover, their solver was sensitiveto element approximation difficulties. The second group of works by Bo-man, Hendrickson and Vavasis [25], Gupta [74], and Wang and Sarin [153]construct the matrices Le’s using geometrical, physical and discretization-specific information (not using only the K ′es). Their works are thereforelimited to specific finite-element formulations. This limitation is also evi-dent in the fairly limited experiments performed with those preconditioners(i.e., results limited to specific element types).

1.4.2. Our Work. In Chapter 4, which is based on a paper that willappear in the SIAM Journal on Matrix Analysis and Applications [9], we


remove the limitations of these techniques. (Both I and another PhD stu-dent in Tel-Aviv University, Haim Avron, made significant contributionsto this paper.)

(1) We use a novel efficient nearly optimal algorithm for approximatingthe Ke’s by the Le’s. Our algorithm is purely algebraic and worksfor any element type.

(2) We study the theoretical question of approximability.(3) Our solvers smoothly handles approximation difficulties.(4) We implemented a practical solver and performed a thorough com-

parative study.

Our nearly optimal algorithm is based on a novel reduction of the prob-lem of approximating Ke’s by Le’s to the problem of row scaling a (pos-sibly rectangular) matrix to minimize its condition number. (In this case,the condition number is the ratio between its extreme singular values).Then we use an existing approximation algorithm [147] to solve this prob-lem. For each Ke, our approximation algorithm generates an SDD matrix

Lnearly-opte which is both easy to compute and satisfies

λmax(Ke, Lnearly-opte )

λmin(Ke, Lnearly-opte )

≤ n2e

2· λmax(Ke, L

opte )

λmin(Ke, Lopte )

,

where ne is the number of nonzero rows and columns in Ke and Lopte is the

best possible SDD matrix. We have also presented and analyzed additionalelement approximation algorithms. Generally, they all produce good ap-proximations only if Ke is well-conditioned. Although this is often the case,it is not guaranteed. We give here an example of a specific approximationmethod that we call the positive-part approximation.

Definition 1.4.1. LetK be a symmetric positive (semi)definite matrix.We define K+ to be the sdd matrix defined by

(K+)ij =

kij i 6= j and kij < 0

0 i 6= j and kij ≥ 0∑k 6=j − (Ke+)ik i = j .

It turns out that in many cases, κ(K,K+) is small, making K+ a goodapproximation for K. The following lemma gives a theoretical bound forκ(K,K+).

Lemma 1.4.2. Let K be an SPSD ne-by-ne matrix with null(K) =span[1 . . . 1]T . Then null(K+) = span[1 . . . 1]T , and κ(K,K+) ≤ √neκ(K).Moreover, if there exist a constant c and an index i such that ‖K‖inf ≤ cKii,then κ(K,K+) ≤ cκ(K).

The next question we asked ourselves was: “Are all elements well-approximable by SDD matrices?”. As demonstrated above, if Ke is well


K γL+ K γM + K

(a) (b) (c)

Figure 1.4.2. SDD approximation approach with splitting.Part (a) presents the finite-element matrix K. Part (b) illus-trates an SDD approximation L of only a part of K, scaledand added to some inapproximable elements in K markedwith K. Part (c) illustrates the preconditioner L of M scaled

and added to K to form a preconditioner of K.

conditioned, than it is well-approximable. In Chapter 4, we present ma-trices that are ill conditioned, far from SDD matrices, but still can beapproximated well. We also present (ill-conditioned) matrices that can notbe approximated by SDD matrices.

The existence of inapproximable matrices indicates that the basic methodof element-by-element approximation for preconditioning K might fail insome cases. The theoretical bounds in Equation 1.4.2 instantly becomeloose if even a single element is not approximable. This is not only the-oretical, we observed in our experiments that even if a relatively smallportion of the elements are not approximated well, the observed number ofiterations of the preconditioned iterative solver becomes high.

The solution to this problem is the splitting idea, illustrated in Fig-ure 1.4.2. For every element e, we approximate Ke by Le using our nearly-optimal algorithm. Then, we calculate the ratio λmax(Ke, Le)/λmin(Ke, Le).If it is greater than a prescribed constant t, we leave Ke as-is. Now,let L =

∑<t Le be the sum of matrices that were approximated and let

K =∑

>tKe be the sum of matrices that were kept as-is. We combina-

torially sparsify L into M . We use γM + K as a preconditioner for K.The scaling factor γ is quite technical and explained in the text itself. Themethod guarantees a bound on the generalized condition number. Unfor-tunately, the preconditioner is not easy to invert if K is too large.

The last part of our work is a practical implementation of our solverand a comparative study. We have implemented most of the code in Cand used Matlab’s [111] cmex API for a benchmarking interface. Wehave used Vaidya’s combinatorial preconditioner from taucs [37]. We havefactored our preconditioner using cholmod 1.0 sparse Cholesky by TimDavis, with metis [95] fill reducing permutation.

1.5. Finite-Element Formulation for Surface Reconstruction 29

We compared our solver against a direct solver (cholmod 1.0 withmetis), against a preconditioned drop-tolerance incomplete Cholesky solver,and against an algebraic multigrid (AMG) solver (BoomerAMG [79]). Wetested these solvers on various three dimensional models, with various ele-ment types. Here is a brief summary of the results.

(1) Generally, our solver was robust; it produced consistent and con-tinuous results for changing parameters and problem sizes. It wasnot always the fastest, but predictable and reliable.

(2) When the problems were small enough to fit into the memory ofthe test machine (˜8GB) the direct solver was the most robust andfaster than any other method.

(3) For large problems, even very ill-conditioned ones, when all ele-ments are well approximable, our solver worked well, but AMGand drop tolerance incomplete Cholesky were better.

(4) For large problems, with some inapproximable (and therefore ill-conditioned) elements, AMG performed poorly, drop tolerance in-complete Cholesky was unreliable, and our solver was the quickest.

1.5. Finite-Element Formulation for Surface Reconstruction

In Chapter 5, we describe a finite element formulation of a surfacereconstruction problem that arises in computer graphics. This formulation,along with additional techniques for interactive surface reconstruction, waspublished in a SIGGRAPH 2007 paper [136]. The goal of this work isto reconstruct a smooth watertight1 three dimensional surface from partialand noisy scan data. The computer graphics aspect of this problem is dueto Andrei Sharf, Thomas Lewiner, and Daniel Cohen-Or. In this sectionwe focus on my contributions to the solution of this problem, namely thefinite element formulation for the problem and the efficient construction ofa least-squares problem.

The initial input for the problem is a set of scanned points PS thatare on the surface or near it. The input suffers from under sampling ofspecific areas, it has holes, and can have outliers. Moreover, the problemof reconstruction of such a surface is an ill posed problem; there is aninfinite number of surfaces that pass through or near the data points. Inorder to solve ambiguities and to handle the problems with the data points,additional input is created automatically, and can also be controlled by the

user. The additional input comprises two sets of points: P In of points

inside the watertight model and POut of points outside it.Figure 1.5.1 illustrates this process in two dimensions. The original

hummingbird figure is presented in part (a). The scan points are illustratedin part (b). Notice the under-sampling in its beak and upper wing. Part

1A closed-surface model that has all gaps and holes filled to clearly define an inside andoutside.


(a) (b) (c)

(d) (e) (f)

Figure 1.5.1. Illustration of the surface reconstruction intwo dimensions.

(c) illustrates P In and POut in pink and green respectively. These pointswere automatically generated by finding local maximum in an unsigneddistance transform of the scanned points. Part (d) shows the resultingreconstructed surface (line in this case) in yellow. The distortions in theupper wing and in the beak are noticeable. Moreover, around the beak,we got a topologically different reconstruction. Part (e) illustrates the sets

P In and POut after being edited by the user. The new sets lead to theplausible reconstruction in part (f).

The basic idea in the reconstruction, due to Kazhdan et al. [98], is toreconstruct a field u in the entire domain instead of the surface itself. Thesurface will be the zero isosurface of this field. Let d be the computedunsigned distance transform of the scanned points. In our two dimensionalexample, we want

u(x, y) ≈ 0 for (x, y) ∈ PS

u(x, y) ≈ d(x, y) for (x, y) ∈ P In

u(x, y) ≈ −d(x, y) for (x, y) ∈ POut.

We also want a smooth surface. This is achieved by making u smooth. Weselect the following non-smoothness penalty

Ψ(u) =1

2

ˆ ˆ [(∂u

∂x

)2

+

(∂u

∂y

)2]dx dy .

This non-smoothness penalty is exactly the energy that is emitted when urepresents an electrostatic potential in a uniformly conductive material inthe shape of the domain. This observation becomes useful shortly.


The next task is transforming this continuous problem to a discreteproblem that we can solve on a computer. Let M be mesh of triangles(tetrahedrons in three dimensions). The continuous function u is now avector uM , containing a scalar for each mesh node. Using the finite-elementmethod we construct a matrix K = ETE such that Ψ(uM) ≈ uTMKuM =‖EuM‖2

2. This is exactly the stiffness matrix for the scalar Poisson problem

without boundary conditions. The points in PS, P In and POut are notnecessarily points of the mesh. Nevertheless, they are inside a triangle(or a tetrahedron). We represent the value of u in point p using a linearinterpolation vector cp (with 3 or 4 nonzeros for triangles and tetrahedronsrespectively) multiplied by uM . We also set wp to be the weight for eachpoint constraints.

Now, we can combine all the constraints into the following least squaresystem

minuM

∥∥∥∥∥∥∥∥∥∥

Ew1c1

w1c2...

w1c|P |

uM −

0w1t(p1)w2t(p2)

...w|P |t(p|P |)

∥∥∥∥∥∥∥∥∥∥

2

2

,

where t(pi) equals d(pi), −d(pi) or 0 for p ∈ P In, p ∈ POut or p ∈ PS

respectively. We solve this directly using cholmod’s factorization routineon the corresponding normal equation. We also use cholmod’s factor

update/downdate routines for fast addition and removal of points in P In

and POut.This formulation enables us to work with irregular meshes. The finite-

element formulation and the linear interpolation guarantee that the opti-mization problem is presented well for such meshes. The specific mesh wework with is constructed in two phases. In the first phase, and adaptive

octree is constructed on top of the point cloud ( PS, P In, and POut).In the second phase, a dual triangular (tetrahedral) mesh is constructed.This enables us to have a very fine mesh near the points of interest and avery coarse mesh away from them. This makes the underlying linear sys-tem relatively small. Its solution is carried out using a direct solver withtypically less than 70K mesh points. As reported in Section 1.4, for suchsmall problems direct solvers are robust and the most efficient. If we hadused a regular mesh, we would have needed a very fine mesh all over thedomain. The linear systems corresponding to such regular fine meshes aretoo big to handle interactively.

This work has other aspects that are not covered here, and are thecontributions of the other authors of [136]. We state them here for com-pleteness: (1) Identification of topologically unstable regions, directing theuser to add additional constraints there, (2) Tablet scribbling mechanism

1.6. Partitioned Triangular Tridiagonalization 32

to allow adding constraints interactively in three dimensions, and (3) acomparative experiment with other reconstruction methods.

1.6. Partitioned Triangular Tridiagonalization

1.6.1. Triangular Tridiagonalization. Triangular tridiagonalizationis the process of factoring a symmetric matrix A in the following way:

P TAP = LTLT ,

where L is unit lower triangular, T is symmetric tridiagonal and P is apermutation matrix. Given such a factorization of A, we can solve thelinear system Ax = b by solving

P TAPP Tx = P T b

LTLTP Tx = P T b .

The second equation can be solved using a lower triangular solve with L,followed by a tridiagonal solve with T and an upper triangular solve withLT .

This factorization is actually not the commonly used factorization forsymmetric matrices. The commonly used factorization is Bunch-Kaufmanfactorization. This factorization is

P TAP = LDLT ,

where L is unit lower triangular and D is a symmetric block diagonalmatrix with 1-by-1 and 2-by-2 blocks. This kind of factorization is calledsymmetric block-diagonalization,

There are advantages and disadvantages for both approaches. TheBunch-Kaufman factorization reveals the inertia of the matrix, whereasthe tridiagonalization only reduces the inertia finding problem to the prob-lem of computing the inertia of T . Moreover, it is very easy to solve alinear system with D, whereas it is slightly harder to solve with T . On theother hand the pivoting strategy in the tridiagonalization is very simpleand much simpler than in the Bunch-Kaufman factorization. The magni-tude of subdiagonal elements of L in the tridiagonalization are bounded by1, whereas in the Bunch-Kaufman factorization they can grow and lead toaccuracy problems [8].

A 1989 lapack working note by Dongarra and Andreson [7] cited poorperformance of the LTLT factorization as the reason for using partitionedBunch-Kaufman in lapack. The partitioned code used in this work wasapparently lost [8]. This raises the challenge to find an efficient partitionedalgorithm.

When we started this project, there was no partitioned formulation andimplementation available for the bounded Bunch-Kaufman method thatwas developed in [8] to solve the accuracy problems with Bunch-Kaufman.This served as an additional motivation for this work. Recently, after sub-mitting this work for publication in [131] we discovered through private


communication with Nick Higham that he and others have implemented apartitioned version of bounded Bunch-Kaufman.

My interest in dense symmetric factorizations was sparked during my(relatively minor) involvement in the implementation of a symmetric in-definite sparse direct solver [114]. While working on that solver I becameaware of the numerical problems in the Bunch-Kaufman method, which Iwas using as a dense kernel in the sparse solver. Following these problems,and the lack of accurate partitioned algorithms for symmetric indefinitefactorizations, I decided to embark on this project. This contribution istherefore relevant to sparse linear solvers, although it is not related tocombinatorial preconditioning. Chapter 6, which is based on a paper thatwas submitted to the ACM Transactions on Mathematical Software [131],presents this contribution.

1.6.2. Symmetric vs. Unsymmetric Reduced Matrices. The al-gorithm of Parlett and Reid [121] performs triangular tridiagonalization ina fairly natural way. Given a symmetric matrix A, let P2 be a permutationmatrix that switches variable 2 with variable k which is the row number ofthe largest magnitude coefficient in A2:n,1. Then we can write

P2APT2 =

A1,1

[Ak,1v

T][

Ak,1v

]B

,

where v is an (n−2) column vector, and B is an (n−1)-by-(n−1) symmetricmatrix. We can now define the unit lower triangular matrix,

L2 =

10 10 `2 In−2

,

where `2 = −v/Ak,1 if A2:n,1 is not all zeros or a zero (n− 2) column vectorotherwise. It is easy to see that

L2P2APT2 L

T2 =

A1,1

[Ak,1 0 . . .

]Ak,10...

C

,

where C is symmetric. We can continue this reduction process on thesymmetric reduced matrix C, and after (n − 2) steps we get a symmetrictridiagonal matrix T . The resulting interleaved product of permutation ma-trices and unit lower triangular matrices can be represented as the productof a single permutation matrix and a single unit lower triangular matrix.

The Parlett-Reid factorization is right-looking; after creating the nextcolumn of L it updates the trailing submatrix. It performs two rank-1 up-dates per step. Since the reduced matrix is symmetric, they can be appliedonly on half a matrix. This results in a cost of 2

3n3 + O(n2) operations.


The same as unsymmetric Gaussian elimination. It is too expensive. Nev-ertheless, the approach of computing a symmetric reduced matrix will beuseful in our (efficient) partitioned formulation.

The algorithm of Aasen [1] is a bit more complicated. It is described indetail in Chapter 6. In this introduction we only mention several propertiesof this algorithm. Aasen’s algorithm works with the equation P TAP =HLT , where H is the lower Hessenberg matrix LT . It operates in a left-looking way; a column is updated by the previous columns only before it isfactored. This formulation enables Aasen to perform the tridiagonalizationin a cost of 1

3n3 +O(n2) operations.

For convenience of presentation, we mark

A[11] = A1:k,1:k ,

A[12] = A1:k,(k+1):n ,

A[21] = A(k+1):n,1:k , and

A[22] = A(k+1):n,(k+1):n .

(and similarly for L, T , and H).The challenge in creating a partitioned version for Aasen’s algorithm

is that its reduced matrix H [22] L[22] T is not symmetric like the reduced

matrix L[22]T [22] L[22] T in Parlett’s and Reid’s algorithm. This means thatit is not trivial to work with a recursive formulation that fits many otherpartitioned algorithms: “Efficiently process the first k columns, using theprocessed columns update the trailing submatrix A(k+1):n,(k+1):n, now con-tinue recursively on the updated trailing submatrix”. The updated trailingsubmatrix (the reduced matrix) is not symmetric.

1.6.3. A partitioned formulation. Fortunately, there is a simplerelationship between the reduced submatrix in the Aasen case and in theParlett-Reid case. We only state it here, it is explained in detail in Chap-ter 6:

L[22]T [22] L[22] T = H [22] L[22] T − L[21]:,k T

[12]k,:1 L

[22]1,:

T

= A[22] −H [21] L[21] T − L[21]:,k T

[12]k,:1 L

[22]1,:

T.

The first line shows that the non symmetric reduced matrix of Aasen

H [22] L[22] T is different from the symmetric reduced matrix of Parlett-Reid

L[22]T [22] L[22] T only by a rank-1 update.Our partitioned algorithmic step is based on the second line. At each

step we process a batch of k columns using Aasen’s left looking scheme.

We perform a right-looking update on A[22] using the rank-k H [21] L[21] T

and the rank-1 L[21]:,k T

[12]k,:1 L

[22]1,:

Tupdates. We get a symmetric Parlett-Reid

reduced matrix and can restart this process. There are some additionaldelicate details involving pivoting and the fact that we want to update


only half of the symmetric reduced matrix. Nevertheless, the essence of thealgorithm is exactly as described.

Miroslav Rozloznık who is a co-author of our paper has proved the nu-merical stability of our partitioned algorithm. Our algorithm is component-wise backward stable, with a bound which is very similar to the originalbound on Aasen’s algorithm [81]. The stability proof is omitted from thisthesis.

1.6.4. Performance. Our method performs an extra single rank-1 up-date for every k columns compared to Aasen’s original algorithm. The totalrunning time is

1

3

(1 +

1

k

)n3 +O(n2k) .

We have implemented the method, fusing the rank-1 and rank-k updatesinto a single BLAS3 rank-(k + 1) update. This means that only O(n2k) ofthe computation is performed outside BLAS3. Our factorization routinealso cheaply factors T using Givens QR during the factorization of A. Wehave compared our implementation to the Bunch-Kaufman factorizationand solution routines in lapack. Chapter 6 contains detailed descriptionof the experiments. Both our factorization routine and our solve routineperform roughly the same as the corresponding routines in lapack.

Our results show that symmetric tridiagonalization methods can beas efficient as symmetric block-diagonalization. Moreover, the extra costof solving tridiagonal matrices is negligible. This shows that there is noperformance related reason to favor symmetric block-diagonalization oversymmetric tridiagonalization methods.

CHAPTER 2

Rigidity in Finite-Element Matrices: SufficientConditions for the Rigidity of Structures and

Substructures

2.1. Introduction

This chapter1 presents an algebraic-combinatorial theory of rigidity forfinite-element matrices and applies this theory to two important problems:determining whether a finite-element matrix represents a rigid structure,and determining whether a matrix representing a structure and a matrixrepresenting a substructure have the same range and null space. We ad-dresses these problems by providing simple sufficient conditions for rigidityand null-spaces equality, and by providing linear-time algorithms (assumingbounded element degrees) to test these conditions.

Our results employ three new technical tools, one combinatorial and twoalgebraic. One algebraic tool is a purely-algebraic definition of the rigidityrelationships between two rank-deficient matrices2. The other algebraictool is a definition of a finite-element matrix A as a sum of of symmetricsemi-definite matrices Aeke=1 that all satisfy a certain condition. Thecombinatorial tool is a graph, called the rigidity graph, that representsthe rigidity relationships between the terms Ae of a finite-element matrixA =

∑eAe. These tools may be applicable to the solution of other problems

involving finite-element matrices.The concept of rigidity is usually associated with elastic structures and

with finite-element models of such structures. An elastic structure is rigidif any deformation of it that is not a translation and/or rotation requiresenergy. A coin is rigid; a door hinge is not. Our theory of rigidity isconsistent with the traditional concept of rigidity, but it is purely algebraicand more general. By purely algebraic, we mean that our theory usesonly the element matrices Ae and a basis for the rigid body motions (e.g.translations and rotations) of the structure. Our theory and algorithmsdo not depend on the geometry of the structure and on the details of the

1The results in this chapter were also published as a paper in the SIAM Journal onMatrix Analysis and Applications, co-authored by Sivan Toledo [138].This chapter also contains an unpublished extension to Theorem 2.8.11 marked as Corol-lary 2.8.14.2The literature contains another definition of matrix rigidity, originally defined byValiant [146]. Our definition is completely different and unrelated.

36

2.1. Introduction 37

finite-element discretization. Our theory generalizes the concept of rigidityin a natural way from finite-element models of elastic structures to modelsof other physical systems, such as electrostatics.

On the other hand, our theory only provides sufficient conditions forrigidity. Characterizing rigidity exactly is difficult, even if we limit ourattention to specific families of elastic structures. Consider, for examplea structure consisting of struts (elastic bars) connected at their endpointsby pin joints. The struts can only elongate or compress, and the strutsconnected to a pin are free to rotate around the pin. The rigidity of suchstructures in two-dimensions has been extensively studied and is now wellunderstood. However, the conditions that characterize the rigidity of two-dimensional structure are expensive to check [91] and they do not generalizeeasily to three-dimensional trusses and to other structures. Our theory ofrigidity avoids these difficulties by focusing on characterizations that aresimple and general but only sufficient. In fact, structures consisting ofstruts always fail our sufficient conditions.

Our new theory is essentially an algebraic-combinatorial characteriza-tion of finite-element models of structures that are, informally speaking,“evidently rigid”. Models of structures that are rigid due to complex non-local interactions between parts of the structure will usually fail our condi-tions. The main contributions of this work are formal and easily-computedcharacterizations of “evidently-rigid” structures. We, therefore, call struc-tures that pass our test evidently-rigid. We apply these characterizations tothe construction of algorithms that find certain minimally-rigid substruc-tures of a rigid structure.

The results in this chapter are a step toward the generalization of re-sults in spectral graph theory from Laplacians to finite-element matrices.We are particularly interested in an area of spectral graph theory calledsupport theory or support preconditioning. This area is mostly concernedwith constructing an approximation B to a matrix A in three steps: (1)building a graph GA that represents A, (2) approximating GA by a simplergraph GB, and (3) building the matrix B that corresponds to GB. Thegraph GB should be simpler in some way than GA (e.g., smaller balancedvertex separators) and the generalized eigenvalues λ of Ax = λBx shouldnot be very large or very small. Much progress has been made in this area,but only when A is a Laplacian [18, 70, 107, 141, 24], a diagonally-dominant symmetric matrix (i.e., GA is a signed graph) [70, 22], or can bewell approximated by a Laplacian [25].

In this chapter we make three contributions to support precondition-ing of finite-element matrices. First, we provide a reasonable definitionof what a finite-element matrix is: a sum of element matrices whose nullspaces are derived from a single global null space. Second, we provide agraph model of finite-element matrices, and proposes graph algorithms forsparsifying the coefficient matrix A. Three, we provide simple combina-torial conditions that allow us to show that the range and null space of


the sparsified matrix (the preconditioner) B are the same as those of A.The qualitative range and null-space equalities are weaker statements thanquantitative bounds on the generalized eigenvalues, but they are a step to-ward eigenvalue bounds. A weighted rigidity graph may allow us to boundeigenvalues and generalized eigenvalues. The same technical tools may alsobe applicable to the generalization of other results in spectral graph theory,such as Cheeger-type bounds [36, 59, 3].

We use these three contributions to define algebraic methods to sparsifyfinite-elements matrices. One method drops elements from the structure.The other method, called fretsaw preconditioning, cuts some of the con-nections between elements. The fretsaw preconditioning algorithm that wepresent in the chapter constructs, using an almost linear amount of work,a preconditioner that can be applied in linear time in every iteration andwhich has the same null space as the original matrix (the linear cost is withrespect to the number of unknowns assuming a fixed element degree). Theconstruction is purely algebraic: the same algorithm, data structure, andcode work on elasticity and electrostatics, in two and three dimensions.

This chapter is quite technical and fairly complex. It may seem strangethat all of this complexity is needed to prove results that are physicallyintuitive. If a structure is evidently rigid, why is all the algebraic and nota-tional complexity needed? The answer appears to be that the complexity isa result of our insistence on a purely algebraic and combinatorial analysis.We do not rely directly on any physical or continuous properties of thestructures that we analyze. Our analysis reaches the physically-intuitiveconclusions, but the algebraic path toward these conclusions is complex.We believe that the generality and software-engineering advantages of apurely-algebraic approach are worth the complexity of the work. Further-more, the analysis is complex, but the algorithms that we propose are bothgeneral and simple.

The chapter is organized as follows. Finite-element matrices are sumsof very sparse terms called element matrices. Most of the rows and columnsin each element matrix contain only zeros. Such matrices have a trivial nullspace that the zero columns generate, and sometimes another null subspacethat is more interesting. Our study of rigidity is essentially a study of thesenontrivial subspaces. Section 2.2 defines these subspaces and analyzes theirproperties. The combinatorial structure that we use, the rigidity graph, isdefined by rigidity relationships between pairs of element matrices. Theserelationships are defined and explored in Sections 2.3 and 2.4. One of ourultimate goals in this chapter is to show that a connected rigidity graphimplies that the underlying structure is rigid. Unfortunately, this is not truefor collections of arbitrary element matrices; they must have something incommon for their rigidity graph to be useful. This common property iscalled null-space compatibility. Its definition and significance are explainedin Section 2.5. The rigidity graph itself is defined in Section 2.6, along witha proof that a connected rigidity graph implies the rigidity of the structure.

2.2. The Essential Null Space of a Matrix 39

Section 2.7 studies three families of finite-element matrices and their rigiditygraphs, to further illustrate the concepts presented earlier. In Section 2.8 wepresent two methods for sparsifying a finite-element matrix while preservingits null space. The more sophisticated method, called spanning-tree fretsawextension, always leads to simplified finite-element matrices that can befactored with essentially no fill. We present four numerical examples of theuse of spanning-tree fretsaw extension as preconditioners in Section 2.9.We conclude the chapter with a few open problems in Section 6.6.

2.2. The Essential Null Space of a Matrix

Rigidity is closely related to relationships between null spaces. Wetherefore start our analysis with definitions and lemmas concerning thenull space of matrices with zero columns.

Definition 2.2.1. Let A be an m-by-n matrix, let ZA ⊆ 1, . . . n bethe set of its zero columns, and let NA be the set of its nonzero columns.The essential null space of A is the space of vectors x satisfying

• Ax = 0 and• xi = 0 for i ∈ ZA.

The trivial null space of A is the space of vectors x satisfying xi = 0 fori ∈ NA. We denote the two spaces by enull(A) and tnull(A).

Clearly, the essential and trivial null spaces of a matrix are orthogonaland their span (union) is simply the null space of the matrix.

Definition 2.2.2. A restriction of a vector y to the indices NA is thevector

xi =

yi i ∈ NA0 otherwise.

(The restriction is a projection.) An extension with respect to NA of avector x satisfying xi = 0 for i ∈ ZA is any vector y such that yi = xi forall i ∈ NA.

Lemma 2.2.3. Let y be any extension with respect to NA of a vectorx ∈ enull(A). Then y ∈ null(A).

Proof. Let z = y − x, so y = x+ z. Since yi = xi for i ∈ Ne, we havezi = 0 for i ∈ NA. Therefore Az = 0, so Ay = Ax+ Az = 0 + 0 = 0.

Lemma 2.2.4. The restriction of a vector y ∈ null(A) to NA is inenull(A).

Proof. Let y = yN + yZ be a splitting of y into a vector yN withnonzeros only in NA and yZ with nonzeros only in ZA. Clearly, AyN =AyZ = 0. The result follows from the fact that yN is the restriction of y toNA.

Lemma 2.2.5. Let A and B be n-by-n symmetric positive semidefinitematrices. Then null(A+B) = null(A) ∩ null(B).

2.3. Rigidity Relationships 40

Proof. Let x ∈ null(A+B) and suppose for contradiction that Ax 6= 0.A has a decomposition A = LLT . Since Ax 6= 0, we also have LTx 6= 0,so xTLLTx = xTAx > 0. Therefore, xTBx = xT (A + B)x − xTAx =0 − xTAx < 0, a contradiction. Therefore, x ∈ null(A) and similarly,x ∈ null(B). This shows that null(A + B) ⊆ null(A) ∩ null(B). The otherdirection is trivial.

A column that is nonzero in both A and B can be a zero in A+B dueto cancellation. The next lemma shows that this cannot happen when theterms are symmetric positive semidefinite matrices (spsd).

Lemma 2.2.6. Let A and B be n-by-n symmetric positive semidefinitematrices. Then NA+B = NA ∪NB.

Proof. Clearly NA+B ⊆ NA ∪NB. Suppose for contradiction that thelemma does not hold. Then there is a column index j in NA or in NB thatis not in NA+B. Without loss of generality assume that j ∈ NA. Let x bethe jth unit vector. Since j ∈ NA, Ax, which is simply the jth columnof A, is nonzero. But since j 6∈ NA+B, we also have (A + B)x = 0, acontradiction to Lemma 2.2.5.

The last lemma in this section shows the relationship between null-spacecontainment and the sets N and Z.

Lemma 2.2.7. Let A be an m-by-n matrix and let B be an l-by-n matrixwith null(B) ⊆ null(A). Then ZB ⊆ ZA and NA ⊆ NB.

Proof. Let j ∈ ZB and let ej be the jth unit vector. By definition,Bej = 0. By the assumption on the null spaces, Aej = 0. This impliesthat j ∈ ZA. Therefore, ZB ⊆ ZA, so the complements of these sets satisfyNA ⊆ NB.

2.3. Rigidity Relationships

This section introduces the main notion of this chapter: rigidity rela-tionships.

Definition 2.3.1. Let A be an m-by-n matrix A and let B be an `-by-n matrix. A vector y ∈ enull(B) is a mapping of x ∈ enull(A) if yi = xi forall i ∈ NA∩NB. We say that the mapping is rigid if there is only one suchmapping of x. The matrix A is rigid with respect to the matrix B if everyvector x ∈ enull(A) has a rigid mapping y ∈ enull(B). The two matricesare called mutually rigid if they are rigid with respect to each other.

Example 2.3.2. Mutual rigidity does not follow automatically fromone-sided rigidity. Consider, for example,

A =

0 0 0 00 1 −1 00 −1 1 00 0 0 0

, B =

1 −1 0 0−1 1 0 00 0 1 −10 0 −1 1

.


A is rigid with respect to B, because vectors in enull(A) have the form[0 α α 0

]T, and they have a unique extension to vectors in enull(B),

namely[α α α α

]T. But vectors in enull(B), which have the form[

α α β β]T

, are not in null(A) unless α = β.

Example 2.3.3. Let A1 =[1 −1 0 0

]and A2 =

[0 1 −1 0

].

The two matrices are mutually rigid. We have

enull (A1) = span[1 1 0 0

]Tenull (A2) = span

[0 1 1 0

]TTherefore, for every x =

[α α 0 0

]T ∈ enull(A1), there is a unique

y =[0 α α 0

]T ∈ enull(A1), and symmetrically for A2.

Now let A3 =[0 1 0 0

]and A4 =

[0 0 −1 1

]. A1 is not rigid

with respect to either of these two. It is not rigid with respect to A3 becauseenull(A3) = 0, so for an x ∈ enull(A1) there is no rigid y in enull(A3). A1

is not rigid with respect to A4 because for x =[α α 0 0

]T ∈ enull(A1),

any y =[0 0 β β

]Tis in enull(A4), so the mapping is not unique.

We now show how to test whether a matrix A is rigid with respect toanother matrix B. For an m-by-n matrix A, we define ΨA and ΨA to bethe n-by-n diagonal matrices[

ΨA

]jj

=

1 j ∈ NA0 j ∈ ZA

and[ΨA

]jj

=

0 j ∈ NA1 j ∈ ZA .

For two matrices A and B with n columns each, we define ΨA,B to be then-by-n diagonal matrix[

ΨA,B

]jj

=

1 j ∈ NA ∩NB0 otherwise .

Let x be a vector in enull(A). If x has a mapping to y ∈ enull(B), theny must satisfy the equations

ΨBy = 0

By = 0

ΨA,By = ΨA,Bx .

The first two conditions constrain y to be in enull(B) and the third condi-tion constrains y to be a mapping of x. If this linear system is inconsistent,then x has no rigid mapping to y ∈ enull(B), soA is not rigid with respect toB. Even if the system is consistent for all x ∈ enull(A), A is not necessarily

rigid with respect to B. If the coefficient matrix RA,B =[ΨTB

BT ΨTA,B

]Tis rank deficient, the mappings are not unique.

Therefore, to test rigidity we must check that for all x ∈ enull(A),

the vector[0 0 xTΨT

A,B

]Tis spanned by the columns of RA,B, and that


the columns of RA,B are linearly independent. We now derive equivalentconditions, but on a much smaller system. First, we drop rows and columnsZB from the coefficient matrix and rows ZB from y. These rows correspondto equations that constrain yi = 0 for i ∈ ZB. Since these elements of yare not used in any of the other equations, we can drop them withoutmaking an inconsistent system into a consistent one. Also, these columnsare linearly independent, and all the other columns are independent ofthem. Therefore, dropping these |ZB| rows and columns reduces the rankof RA,B by exactly |ZB|; therefore, RA,B is full rank if and only if theremaining rows and columns form a full-rank matrix. Now we drop all thezero rows from the system: Rows NB in the ΨB block of RA,B, the zerorows from the B block, and the zero rows from the ΨA,B block. These rowscorrespond to equations that are consistent for any x and any y; being zero,they do not affect the rank of RA,B.

We assume without loss of generality that columns NA ∩ NB are thelast among the nonzero columns of B. We denote by B the matrix formedby dropping all the zero rows and columns of B and by yNB

the vectorformed by dropping elements ZB from y. (For any n-vector v and a setS ⊆ 1, . . . n, the notation vS means the |S|-vector formed by droppingthe elements of v whose indices are not in S, and similarly for matrices.)Our reduced system is

RA,ByNB=

[B

0 | I

]yNB

=

[0

xNA∩NB

],

where the order of the identity matrix is |NA ∩NB|. To test whether Ais rigid with respect to B, we construct a matrix NA whose columns spanenull(A), and check

(1) whether RA,B has full rank, and(2) whether for every column x in NA,

RA,BR+A,B

[0

xNA∩NB

]=

[0

xNA∩NB

],

where R+A,B denotes the Moore-Penrose pseudoinverse of RA,B.

If B has only few nonzero rows and columns and if the number of columnsin NA is small, then this is is an inexpensive computation. The constructionis illustrated in Figure 2.3.1.

The next three lemmas show the relationship between null-space con-tainment and rigidity.

Lemma 2.3.4. Let A be an m-by-n matrix and let B be an l-by-n matrix.If null(A) ⊆ null(B), then A is rigid with respect to B.

Proof. Let x ∈ enull(A). Therefore, x ∈ null(A) and x ∈ null(B).Define y = ΨBx. We have that xi = yi for all i ∈ NA∩NB. By Lemma 2.2.4,y ∈ enull(B). Therefore y is a mapping of x in enull(B).


=

x

y

ΞA,B

0

0

ΞA,B

B

ΞB

the inner matrix isRA,B

B (with A as anoverlay)

A

Figure 2.3.1. Testing rigidity. The top part of the figureshows the entire linear system, and the bottom part showsthe construction of RA,B.

We now show that y is the unique mapping of x. Let y be a mapping ofx in enull(B). By Lemma 2.2.7, NB ⊆ NA. The equalities yi = xi = yi holdfor every i ∈ NA∩NB = NB. Therefore, y = y, so y is the unique mappingof x in enull(B). This implies that A is rigid with respect to B.


Lemma 2.3.5. Let A be an m-by-n matrix and let B be an l-by-n matrix.If NB ⊆ NA and A is rigid with respect to B, then null(A) ⊆ null(B).

Proof. Let x ∈ null(A). We can write x as y + z where y ∈ enull(A)and z ∈ tnull(A). We have that z ∈ tnull(B), since NB ⊆ NA. Therefore,z ∈ null(B).

We now show that y is also in null(B). Let u be y’s rigid mapping toenull(B). We have that ui = yi for every i ∈ NA ∩ NB = NB. Therefore,we can write y as y = u + u′ where u′i 6= 0 only for i ∈ NA\NB. Itclear that u′ ∈ tnull(B) ⊆ null(B). Therefore, y = u + u′ ∈ null(B) andx = y + z ∈ null(B), which is what we need to prove the lemma.

Corollary 2.3.6. Let A be an m-by-n matrix and let B be an l-by-nmatrix, such that NA = NB. Then A and B are mutually rigid if and onlyif null(A) = null(B).

Proof. Directly follows from Lemma 2.3.4 and Lemma 2.3.5.

The last lemma in this section shows that rigidity relationships aremaintained in certain Schur complements.

Lemma 2.3.7. Let A and B be n-by-n matrices of the form

A =

[A11 00 0

], B =

[B11 B12

B21 B22

]where A11 and B11 are k-by-k matrices for some 0 < k < n. Assume thatA is rigid with respect to B, that NA = 1, . . . , k, that NB = 1, . . . , n,and that B22 is nonsingular. Then null(A11) ⊆ null(B11−B12B

−122 B21) and

A11 is rigid with respect to B11 −B12B−122 B21.

Moreover, if A and B are mutually rigid, then null(A11) = null(B11 −B12B

−122 B21).

Proof. Let x ∈ null(A11). Let x be the vector of size n that equals x inits first k coordinates and that contains zeros in its last (n−k) coordinates.Clearly, x ∈ null(A). Since there are no zero columns in A11 we also havethat x ∈ enull(A). Let y be the rigid mapping of x in enull(B). Theequalities yi = xi = xi hold for all i ∈ 1, . . . , k. Let y be a vector of size(n − k) consisting of the last (n − k) elements of y. Writing the equationBy = 0 in terms of x and y, we obtain[

B11 B12

B21 B22

] [xy

]=

[B11x+B12yB21x+B22y

]= 0

Multiplying the second block row by B−122 gives y = −B−1

22 B21x. Substitut-ing y with −B−1

22 B21x in the first block row, we get B11x−B12B−122 B21x =

0. Therefore, x ∈ null(B11 − B12B−122 B21), so null(A11) ⊆ null(B11 −

B12B−122 B21). The containment of the null spaces, along with Lemma 2.3.4,

shows that A11 is rigid with respect to B11 −B12B−122 B21.

Now assume A and B are mutually rigid (we add the assumption thatB is rigid with respect to A). Let x ∈ null(B11−B12B

−122 B21). Let x be the

2.4. Rigidity of Sums 45

vector of size n that equals x in its first k coordinates and equals −B−122 B21x

in its last (n− k) coordinates. The vector x is in enull(B), since

Bx =

[B11x+B12(−B−1

22 B21x)B21x+B22(−B−1

22 B21x)

]= 0 .

Because B is rigid with respect to A, the vector x has a unique mappingto enull(A). Since NA ⊆ NB, this mapping is ΨAx. Therefore, AΨAx = 0,so x ∈ null(A11). This implies that null(B11 − B12B

−122 B21) ⊆ null(A11).

Therefore, null(B11−B12B−122 B21) = null(A11). This concludes the proof of

the lemma.

2.4. Rigidity of Sums

Finite-element matrices are sums of mostly-zero matrices. This sectionextends our study of rigidity to sums of matrices.

Lemma 2.4.1. Let A and B be symmetric positive-semidefinite n-by-nmatrices. The matrix (A + B) is always rigid with respect to A and withrespect to B.

Proof. By Lemma 2.2.5, null(A + B) ⊆ null(A). Therefore, fromLemma 2.3.4, (A+B) is rigid with respect to A. By symmetry, (A+B) isrigid with respect to B, too.

The previous lemma showed that a sum of spsd matrices is rigid withrespect to the terms of the sum, but the terms are not always rigid withrespect to the sum, even when the terms are spsd. For example, A =[

1 −1−1 1

]is not rigid with respect to A + [ 0 0

0 1 ], because A is rank deficientbut the sum is not. Hence, vectors in enull(A) have no mapping at all tothe essential null space of the sum.

Also, the lemma holds for spsd matrices but not for general matrices.Let A = [ 1 0

0 1 ] and let B =[

0 −10 −1

]. Their sum is A+B = [ 1 −1

0 0 ]. The vector[1 1

]T ∈ enull(A+B), but this vector has no mapping into enull(A) = 0.The next lemma strengthens both the hypothesis and the consequence

of Lemma 2.4.1. It shows that if the terms are mutually rigid, then rigiditybetween the terms and the sum is mutual.

Lemma 2.4.2. Let A and B be mutually rigid symmetric positive-semidefiniten-by-n matrices. Then A and A+B are mutually rigid, and B and A+Bare mutually rigid.

Proof. By Lemma 2.4.1, the sum is rigid with respect to the terms.So all we need to prove is the opposite direction.

Let x 6= 0 be a vector in enull(A). Let y be the rigid mapping of x intoenull(B). We now show that x has a mapping into enull(A + B); we shall


show the uniqueness of the mapping later. We define

wi =

xi i ∈ NAxi = yi i ∈ NA ∩NByi i ∈ NB0 otherwise.

Because NA+B ⊆ NA ∪ NB, we have wi = 0 for i ∈ ZA+B. Therefore, toshow that w ∈ enull(A + B), we only need to show that (A + B)w = 0.This is indeed the case because w is an extension of both x and y, soAw = Bw = 0.

We now show that w is the unique mapping of x into enull(A + B).Suppose that there is another mapping w′ 6= w. Under this supposition,there must be w′i 6= wi for some i ∈ NB\NA, so the restriction y′ of w′ toNBmust be different from y. By Lemmas 2.2.5 and 2.2.4, y′ ∈ enull(B). Thevectors y and y′ are both in enull(B) and both coincide with x on NA∩NB,so they are two different mappings of x, contradicting the hypothesis thatA and B are mutually rigid.

Lemma 2.4.3. Let A and B be symmetric positive-semidefinite n-by-nmatrices, and let C be an m-by-n matrix . C and A+B are mutually rigidif and only if C and A+ αB are mutually rigid for every real α > 0.

Proof. Let α > 0. We first show that enull(A+ αB) = enull(A+B).It is clear that NαB = NB and that αB is symmetric positive-semidefinite.From Lemma 2.2.6 we have NA+αB = NA ∪NαB = NA ∪NB = NA+B. ByLemma 2.2.5,

null(A+ αB) = null(A) ∩ null(αB) = null(A) ∩ null(B) = null(A+B) .

Therefore, enull(A + αB) = enull(A + B). The lemma follows directlyfrom the definition of mutual rigidity and the fact that enull(A + αB) =enull(A+B).

In some special cases, mutual rigidity between sums allows us to inferthat the terms of the sums are mutually rigid and vice versa.

Lemma 2.4.4. Let A, B and C be n-by-n symmetric positive-semidefinitematrices such that NC ∩NA = NC ∩NB = ∅. Then A and B are mutuallyrigid if and only if A+ C and B + C are mutually rigid.

Proof. Assume that A and B are mutually rigid. We show that A+Cis rigid with respect to B + C. By symmetry, B + C is rigid with respectto A+ C, so the two sums are mutually rigid.

Let x ∈ enull(A + C). By Lemma 2.2.5, x ∈ null(A) and x ∈ null(C).Let x be x’s restriction to NA and let x be its restriction to NC . ByLemma 2.2.4, x ∈ enull(A) and x ∈ enull(C). Let y be x’s rigid (unique)


mapping to enull(B). We define the vector

yi =

yi i ∈ NBxi i ∈ NC0 otherwise .

The definition is valid because NC ∩ NB = ∅. We show that y is therigid mapping of x in enull(B + C). Multiplying B + C by y we obtain(B + C)y = By + Cy = By + Cx = 0 + 0 = 0. Since yi = 0 for alli /∈ NB ∪ NC = NB+C , y ∈ enull(B + C). By definition, yi = xi for alli ∈ (NA∩NB)∪NC = (NA∪NC)∩(NB∪NC) = NA+C∩NB+C . Therefore,y is a mapping of x in enull(B + C).

We now show that this mapping is indeed unique. Assume that thereexists u ∈ enull(B + C) that satisfies ui = xi for all i ∈ NA+C ∩ NB+C .We have that ui = xi = yi for all i ∈ NC ⊆ NA+C ∩ NB+C . Let u beu’s restriction to NB. We have u ∈ enull(B) and ui = xi = xi for alli ∈ NA ∩ NB. Therefore, u is a mapping of x in enull(B). Since A andB are mutually rigid, u must equal y. Therefore, u = y and y is the rigidmapping of x in enull(B +C). This shows that A+C is rigid with respectto B + C. Figure 2.4.1 (a) presents this notation graphically.

We now show the other direction. Assume A+C and B+C are mutuallyrigid. We show that A is rigid with respect to B; mutual rigidity follows bysymmetry. The notation for this part of the proof is presented graphically inpart (b) of Figure 2.4.1. Let x ∈ enull(A). SinceNC∩NA = ∅, x ∈ tnull(C).We also have xi = 0 for all i /∈ NA ∪ NC = NA+C , so x ∈ enull(A + C).Let y be x’s rigid mapping to enull(B + C). We show that y is x’s rigidmapping to enull(B). By Lemma 2.2.5, y ∈ null(B). Also, yi = xi = 0 forall i in i ∈ NC ⊆ NB+C . Therefore, yi = 0 for all i /∈ NB, so y ∈ enull(B).By definition, xi = yi for all (NA∩NB) ⊆ NA+C ∩NB+C . This implies thaty is a mapping of x in enull(B).

Finally, we claim that y is the unique mapping of x. Assume that thereexist u in enull(B) that satisfies xi = ui for all i ∈ NA ∩NB. We have thatxi = ui = 0 for all i ∈ NC . Since u is also in enull(B + C) then it is therigid mapping of x ∈ enull(A+C) in enull(B+C) . Because A+B is rigidwith respect to A+ C, we have that u = y. Therefore, y is indeed unique.This implies that A is rigid with respect to B, which concludes the proofof the lemma.

We would like to build larger assemblies of mutually-rigid matrices fromchains of mutual rigidity, but this is not always possible, as the next exam-ple shows.

Example 2.4.5. Let

A =

1 −1 0−1 1 00 0 0

, B =

0 0 00 1 −10 −1 1

, C =

1 0 10 0 01 0 1

.

2.5. Null-Space Compatibility 48

(a) (b)

Figure 2.4.1. An illustration of the notation ofLemma 2.4.4: (a) the vectors defined in the proof ofthe mutual rigidity of A+B and A+C, and (b) the vectorsdefined in the proof of the mutual rigidity of A and B.

These matrices are all spsd, and their essential null spaces are spanned by[1 1 0

]T,[0 1 1

]Tand

[1 0 −1

]T, respectively. The matrices A and

B are mutually rigid, and so are B and C. The essential null space of A+B

is spanned by[1 1 1

]Tand the essential null space of B + C is spanned

by[1 −1 −1

]T. Therefore, C is not mutually rigid with A + B and A

is not mutually rigid with B + C. Moreover, neither A, B, C, A + B, norB +C is mutually rigid with A+B +C, because A+B +C has full rank.This example is inspired by the analysis of signed graphs in [22], whichshows that A+B + C has full rank.

To build larger assemblies of mutually-rigid matrices, we need anothertool.

2.5. Null-Space Compatibility

This section defines and explores a concept that we call null-space com-patibility, which is the tool that allows us to build large assemblies ofmutually-rigid matrices.

Definition 2.5.1. Let S ⊆ Rn be a linear space. A matrix A is calledS-compatible (or compatible with S) if every vector in enull(A) has a uniqueextension into a vector in S, and if the restriction of every vector in S toNA is always in enull(A).

Definition 2.5.2. Let A be an n-by-n matrix. If A is S-compatible andNA = 1, . . . , n, then we say that A is S-rigid. This condition is equivalent


to null(A) = S and NA = 1, . . . , n. If S is clear from the context, wesimply say that A is rigid.

Given a basis for S, we can easily check the compatibility of a matrixA. Let the columns of N be a basis for S, and let NA be a basis forenull(A). A is compatible with S if and only if NA = ΨANA and ΨAN havethe same range. This can be checked numerically using the singular valuedecompositions of the two matrices, for example.

Example 2.5.3. Let S = span[1 1 1

]T. The matrices A and B from

Example 2.4.5 are compatible with S, but C is not. Note that the mutualrigidity of A and C together with the S-compatibility of A do not implyS-compatibility for C. The matrix A + B from the same example is alsocompatible with S, and since NA+B = 1, 2, 3, A+B is rigid.

Lemma 2.5.4. Let A be an S-compatible matrix. Let N be a matrixwhose columns form a basis for S. Then enull(A) = span(ΨAN).

Proof. We first show that enull(A) ⊆ span(ΨAN). Let x ∈ enull(A).Since A is S-compatible, x has a unique extension w in S. By definition,there exists a vector y such that w = Ny. Substituting w in the equationx = ΨAw, we get x = ΨANy. Therefore, x ∈ span(ΨAN) so enull(A) ⊆span(ΨAN).

We now show that span(ΨAN) ⊆ enull(A). Let x = ΨANy ∈ span(ΨAN).Define w = Ny ∈ S. Since A is S-compatible, x = ΨAw ∈ enull(A). Thisshows that span(ΨAN) ⊆ enull(A).

The definition of null-space compatibility is related to the definition ofmutual rigidity, but it defines compatibility with respect to a space, notwith respect to a particular matrix having that space as a null space. Hereis the relationship of S-compatibility with mutual rigidity.

Lemma 2.5.5. Let S ⊆ Rm be a linear space, and let B be some ma-trix with no zero columns whose null space is S. Another matrix A isS-compatible if and only if A and B are mutually rigid.

Proof. The equivalence follows from the fact that enull(B) = null(B) =S (because ZB = ∅) and from the fact that NA ∩NB = NA.

If the dimension of S is small, the S-compatibility test given after Def-inition 2.5.2 can be much more efficient than the test for mutual rigiditygiven earlier.

Example 2.5.6. Two matrices that are both compatible with some nullspace S are not necessarily mutually rigid. For example

A =

1 −1 0 0−1 1 0 00 0 0 00 0 0 0

and B =

0 0 0 00 0 0 00 0 1 −10 0 −1 1


are both compatible with S = span[1 1 1 1

]T, but they are not mutu-

ally rigid. Also, their sum is not S-compatible. Since NA+B = 1, 2, 3, 4,enull(A + B) = null(A + B), so A + B is S-compatible if and only ifnull(A+B) = S. However,

enull(A+B) = enull

1 −1 0 0−1 1 0 00 0 1 −10 0 −1 1

= span

1 01 00 10 1

6= S = span

1111

.

The next two lemmas are key results that will allow us to build largeassemblies of mutually-rigid matrices.

Lemma 2.5.7. Let A and B be mutually rigid, both S-compatible forsome null space S. Then A+B is also S-compatible.

Proof. Let u be a vector in enull(A+B), let x be the restriction of uto NA, and let y be the restriction to NB. Clearly, y is the rigid mappingof x in enull(B). Let w be the extension of x to a vector in S. We claimthat w is a unique extension of u to S. If w is not an extension of u, thenthey must differ on an index in NB \ NA, so the restriction of w to NB issome y′ 6= y. The vector y′ is also a mapping of x in enull(B). But bothy and y′ 6= y are mappings of x in enull(B), a contradiction to the mutualrigidity of A and B.

We now show that w is the unique extension of u to S. If there isanother extension, its restriction to NA must differ from x, so it cannot bean extension of u.

We now show that the restriction u of a vector w ∈ S is in enull(A+B).The restriction x of u to NA is also the restriction of w to NA, so Ax =Au = Aw = 0. Similarly for the restrictions toNB. Therefore (A+B)u = 0,so u ∈ enull(A+B).

We now introduce a technical lemma that shows how to transform anull-space extension of a vector into the rigid mapping of the same vector.

Lemma 2.5.8. Let A and B be mutually rigid matrices, both compatiblewith some null space S. Let x ∈ enull(A) and let w ∈ S be its uniqueextension to S. The vector ΨBw is the rigid mapping of x to enull(B). Inparticular, if some vector u is the rigid mapping of x to enull(B), then theunique extension of u to S is w.

Proof. Let y = ΨBw. We first show that y is the rigid mapping ofx to enull(B). The vector y is in enull(B), since B is S-compatible andw ∈ S. From the definition of w and y we have that xi = wi = yi for alli ∈ NA ∩ NB. Therefore, y is a mapping of x in enull(B) and it is uniquebecause A and B are mutually rigid.

Let u be the rigid mapping of x to enull(B). Since this mapping isunique, u = y = ΨBw. The vector w is an extension of u to S. The matrixB is S-compatible so this extension is unique.


The following lemma is the main result of this section. Compare thislemma to Example 2.4.5: in the example, the three matrices were not allcompatible with some null space S; the conclusion of this lemma does nothold in that example.

Lemma 2.5.9. Let A, B, and C be symmetric positive-semidefinite n-by-n matrices, all compatible with some null space S. Let A and B bemutually rigid and let B and C be mutually rigid. Then A+ B and C aremutually rigid (and similarly for B + C and A).

Proof. We first show that A + B is rigid with respect to C. Let ube in enull(A + B). By Lemma 2.5.7, the matrix A + B is S-compatible.Let w be u’s extension to S. Define x = ΨT

Aw, y = ΨTBw and z = ΨT

Cw.By definition, u = ΨT

A+Bw and therefore zi = ui for all i ∈ NA+B ∩ NC .Since C is S compatible, z ∈ enull(C). Therefore, z is a mapping of u inenull(C).

We show that z is unique. The matrices A + B and A are mutuallyrigid according to Lemma 2.4.2. According to Lemma 2.5.8, x is the uniquemapping of u in enull(A), y is the unique mapping of x in enull(B) and z isthe unique mapping of y in enull(C). Therefore, z is the unique mappingof u in enull(C) and A+B is rigid with respect to C.

We show now that C is rigid with respect to A + B. Let z be inenull(C). The matrix C is S-compatible. Let w be z’s extension to S.Define x = ΨT

Aw, y = ΨTBw and u = ΨT

A+Bw. By definition, z = ΨTCw and

therefore zi = ui for all i ∈ NA+B ∩ NC . Therefore, u is a mapping of z inenull(A+B).

We show that u is unique. According to Lemma 2.5.8, y is the uniquemapping of z in enull(B), x is the unique mapping of y in enull(A) andu is the unique mapping of x in enull(A + B). Therefore, u is the uniquemapping of z in enull(A+B). This implies that C is rigid with respect toA+B and concludes the proof of the lemma.

Example 2.5.10. This example shows that A and C are not neces-sarily mutually rigid, even if A and B are mutually rigid and B and Care mutually rigid and all three are S-compatible. Instead of constructingthe matrices here, we refer the reader to the construction of Section 2.7.3,where we construct matrices that model elastic triangles. Suppose that thethree matrices are 10-by-10, that A corresponds to a triangle (p1, p2, p3) (piis a point in the plane; all the points are different), B to (p2,, p3, p4), andC to (p3, p4, p5). We assume that the three triangles are disjoint exceptfor shared edges. The analysis of Section 2.7.3 shows that A and B aremutually rigid and B and C are mutually rigid but A and C are not.

The next lemma characterizes the rigidity and S-compatibility of certainlarger sums.

Lemma 2.5.11. Let A and B1, B2, . . . , Bk be symmetric positive-semidefinitematrices, all compatible with some null space S. Let A and Bi be mutually


rigid for i = 1, . . . , k. Then A and A +∑k

i=1Bi are mutually rigid and

A+∑k

i=1 Bi is S-compatible.

Proof. We prove the lemma by induction on k. The case k = 1is trivial by Lemma 2.4.2 and Lemma 2.5.7. We assume that the claimis correct for k smaller than n and show that it is correct for k = n.By the inductive assumption, A and A +

∑n−1i=1 Bi are mutually rigid and

A +∑n−1

i=1 Bi is S-compatible. A and Bn are mutually rigid. Therefore,

by Lemma 2.5.9 we have that Bn and A + A +∑n−1

i=1 Bi = 2A +∑n−1

i=1 Bi

are mutually rigid. By lemma 2.4.3, Bn and A +∑n−1

i=1 Bi are mutually

rigid. Therefore, by Lemma 2.5.9, we have that A and Bn+A+∑n−1

i=1 Bi =A+

∑ni=1Bi are mutually rigid. By Lemma 2.5.7, we also have that Bn +

A +∑n−1

i=1 Bi = A +∑n

i=1Bi is S-compatible. This concludes the proof ofthe lemma.

The following lemma shows that the property of null-space compatiblitysimplifies considerably the mutual-rigidity test.

Lemma 2.5.12. Let A and B be S-compatible symmetric positive semi-definite n-by-n matrices. Let N be an n-by-` rank-` matrix whose columsspan S. The matrices A and B are mutually rigid if and only if ΨA,BN hasfull rank `.

Proof. We first assume that ΨA,BN has rank ` and show that A isrigid with respect to B. Mutual rigidity follows by symmetry. Let x ∈enull(A). We can write x = ΨANyA, where yA is some length-` vector.Define u = ΨBNyA. By definition, u ∈ enull(B) and since

ΨA,Bu = ΨA,BΨBNyA = ΨA,BNyA = ΨA,BΨANyA = ΨA,Bx ,

u is a mapping of x.We now show it is unique. Let u ∈ enull(B), such that ΨA,Bu = ΨA,Bx.

By definition, u = ΨBNyB for some yB. Clearly,

ΨA,Bu = ΨA,BΨBNyB = ΨA,BNyB .

Therefore,ΨA,BNyA = ΨA,Bx = ΨA,Bu = ΨA,BNyB ,

andΨA,BN(yA − yB) = 0 .

Since ΨA,BN has full rank `, yA = yB, so u = u. This implies that A isrigid with respect to B.

We show now the other direction. Assume A and B are mutually rigid.We note that for every 0 6= n ∈ S, ΨAn 6= 0. Otherwise, both n and 0 areextentions of 0 ∈ enull(A), which is impossible since A is S-compatible.

Suppose for contradiction that ΨA,BN is rank deficient. Therefore, thereexists an `-vector y 6= 0, such that ΨA,BNy = 0. Since N has full rank,Ny 6= 0. By the previous paragraph, ΨANy 6= 0. Note that the mappingof ΨANy to enull(B) is ΨBΨANy = ΨA,BNy = 0. Therefore, the mapping

2.6. The Rigidity Graph 53

of both the zero vector and of ΨANy to enull(B) is the zero vector. Thiscontradicts the assumption that A and B are mutually rigid.

The following lemma is a direct consequence of the last lemma and isusefull in certain situations.

Lemma 2.5.13. Let A, B and C be S-compatible symmetric positivesemidefinite n-by-n matrices. If A and B are mutully rigid and NB ⊆ NC,then A and C + αB are also mutually rigid for all α ≥ 0.

Proof. Let α ≥ 0 and let N be an n-by-` rank-` matrix whose columsspan S. The matricesA andB are mutually rigid, therefore, by Lemma 2.5.12,ΨA,BN has full rank. Since C and αB are spsds, by Lemma 2.2.6, NB ⊆NC+αB. Therefore, ΨA,C+αBN contains the rows of ΨA,BN and has a fullrank. By the other direction of Lemma 2.5.12, we get that A and C + αBare mutually rigid.

2.6. The Rigidity Graph

Mutual rigidity relationships in a collection of S-compatible spsd ma-trices define a graph structure that we can use to demonstrate the rigidityof finite-element matrices.

Definition 2.6.1. Let A1, A2, . . . , Ak be S-compatible symmetric pos-itive semidefinite n-by-n matrices for some null space S. The rigidity graphG = (V,E) of A1, . . . , Ak is the undirected graph with V = A1, . . . , Akand

E = (Ae, Af ) : Ae and Af are mutually rigid .We could also define the rigidity graph of a collection of matrices that

are not necessarily S-compatible, but Example 2.4.5 suggests that such adefinition might not have interesting applications. On the other hand, theS-compatibility requirement in the definition enables an important result,which we state and prove next.

Lemma 2.6.2. Let G be the rigidity graph of a collection A1, A2, . . . , Akof S-compatible symmetric positive semidefinite n-by-n matrices. Let H bea connected subgraph of G, let V (H) denote the vertex set of H, and letAe be a vertex in H. Then Ae and

∑Af∈V (H) Af are mutually rigid, and∑

Af∈V (H) Af is S-compatible.

Proof. Let T be a depth-first-search tree of H whose root is Ae. De-note by T1, T2, . . . , Tc the trees in forest formed from T by removing Ae.We show by induction on the height h of T that the following claims holds:Ae and Ae+

∑ci=1

∑Af∈Ti

Af are mutually rigid, and Ae+∑c

i=1

∑Af∈Ti

Afis S-compatible.

The claim holds trivially for k = 1 (a single-vertex tree), because Ae isS-compatible and is mutually rigid with itself.

Now, we assume that the inductive claim is correct for trees with heighth or less and we show it is correct for trees with height h + 1. Let T be a

2.6. The Rigidity Graph 54

tree of height h + 1 whose root is Ae, vertex, and let T1, T2, . . . , Tc be thesubtrees defined above. The height of every Ti is h or less. Let Ai be theroot vertex of Ti, and let Fi be the forest of A′is descendants. By definition,Ae and Ai are mutually rigid. By the inductive claim on Ti, we have thatAi +

∑Af∈Fi

Af is S-compatible and mutually rigid with Ai. We note that

all the sums of the form∑Af are symmetric and positive-semidefinite.

Therefore, by Lemma 2.5.9 Ae and Ai+(Ai+∑

Af∈FiAf ) = 2Ai+

∑Af∈Fi

Afare mutually rigid. By Lemma 2.4.3, we have that Ae and Ai+

∑Af∈Fi

Af =∑Af∈Ti

Af are mutually rigid for every i. By Lemma 2.5.11, we have that

Ae and Ae +∑c

i=1

∑Af∈Ti

Af are mutually rigid and Ae +∑c

i=1

∑Af∈Ti

Afis S-compatible. This concludes the proof of the lemma.

The next result generalizes the previous lemma.

Theorem 2.6.3. Let G be the rigidity graph of a collection A1, A2, . . . , Akof S-compatible symmetric positive semidefinite n-by-n matrices. Let H1

and H2 be two connected subgraphs of G that share a vertex Ae. ThenB =

∑Af∈V (H1) Af and C =

∑Af∈V (H2) Af are mutually rigid.

Proof. According to Lemma 2.6.2, B and Ae are mutually rigid andso are C and Ae. By Lemma 2.5.9, we have that B and Ae+C are mutuallyrigid. The sum Ae + C equals

∑Af∈V (H2)\Ae

Af + 2Ae. By Lemma 2.4.3,

we have that B and∑

Af∈V (H2) Af = C are mutually rigid.

The next theorem shows that the rigidity graph can sometimes tell usthat a finite-element matrix is rigid in the sense that its null space is exactlyS. This is only a sufficient condition; it is not necessary.

Theorem 2.6.4. Let G be the rigidity graph of a collection A1, A2, . . . , Akof S-compatible symmetric positive semidefinite n-by-n matrices. Let A =∑k

e=1Ae. Let N be a matrix whose columns form a basis for S. If G

is connected, then enull(A) = span(ΨAN). In particular, if⋃ke=1NAe =

1, . . . n, then null(A) = S and A is rigid.

Proof. According to Lemma 2.6.2, A is S-compatible. Therefore, byLemma 2.5.4, enull(A) = span(ΨAN).

If⋃ke=1NAe = 1, . . . n, then ΨAN = N and enull(A) = null(A).

Therefore, null(A) = enull(A) = span(ΨAN) = span(N) = S. By defini-tion, A is rigid.

When the rigidity graph is not connected, the null space may or may notbe S. To show that a disconnected rigidity graph sometimes correspondsto a null space larger than S, consider

A1 =

[1 −1 0 0−1 1 0 00 0 0 00 0 0 0

]and A2 =

[0 0 0 00 0 0 00 0 1 −10 0 −1 1

].

Both are compatible with S = span[1 1 1 1

]T, but they are not mu-

tually rigid. Therefore, their rigidity graph consists of two disconnected

2.7. Three Case Studies 55

vertices. The null space of A1 + A2 is spanned by both[1 1 1 1

]Tand[

1 1 −1 −1]T

, so it is indeed larger than S, even though NA1 ∪ NA2 =1, 2, 3, 4. Examples in which the rigidity graph is not connected but thenull space of the sum is S are more complex; we show an example later inthe chapter, in section 2.7.3.

2.7. Three Case Studies

This section presents three families of S-compatible spsd matrices fortwo different Ss. One is well known and we present it without proofs. Thesecond and third are more complex and we present them in full.

2.7.1. Laplacians. The first family of matrices that we present con-sists of Laplacians, matrices that are often used in spectral graph theory(e.g. [115]) and in other areas. The results in this sections are all adapta-tions of known results, so we omit the proofs. All the matrices and vectorsare of order n.

Definition 2.7.1. For a pair (k, j) of indices, 1 ≤ k < j ≤ n, we definethe vector u(k,j),

u(k,j)i =

+1 i = k

−1 i = j

0 otherwise .

We define the (k, j) edge matrix using A(k,j) = u(k,j) u(k,j) T .

Lemma 2.7.2. Let , 1 ≤ k < j ≤ n and let A(k,j) be an edge matrix.Then

(1) A(k,j) is symmetric positive-semidefinite.(2) NA(k,j) = k, j.(3) A(k,j) is compatible with S1 = span

[1 1 · · · 1

]T.

The next lemma gives a sufficient and necessary condition for two edgematrices to be mutually rigid.

Lemma 2.7.3. The edge matrices A(i,j) and A(k,`) are mutually rigid ifand only if |i, j ∩ k, `| ≥ 1.

Laplacians are sums of edge matrices. They are often defined using anundirected graph G = (1, 2, . . . , n, E),

A(G) =∑

(i,j)∈E

A(i,j) .

Each edge matrix A(i,j) is then associated with an edge (i, j) ∈ E in thegraph G. Weighted Laplacians are sums of positively-scaled edge matricesand correspond to graphs with positively weighted edges. Lemma 2.7.3states that two edge matrices are mutually rigid if and only if the corre-sponding edges are incident on at least one common vertex.


The rigidity graph of A(i,j)|(i, j) ∈ E is a dual of G,

Gdual = (E, (e, f) : e and f share a vertex in G) .The rigidity graph of Laplacians is special in that its connectivity is

not only a sufficient condition for the rigidity of the Laplacian, as shownin Theorem 2.6.4, but also a necessary condition.

Lemma 2.7.4. Let G = (1, 2, . . . , n, E) be an undirected graph. IfA(G) =

∑E A

(i,j) is rigid then the rigidity graph Gdual of A(i,j)|(i, j) ∈ Eis connected.

Proof. We first show that if A(G) is rigid, then G is a connected graph.Assume for contradiction that G is not connected. Therefore, there are twononempty sets of vertices S and S = 1, . . . , n\S that are disconnected.Define the vector x,

xi =

+1 i ∈ S−1 i ∈ S .

By definition, A(i,j)x = 0 for every (i, j) ∈ E. Therefore, A(G)x =∑

(i,j)∈E A(i,j)x =

0. The vector x is in enull(A(G)) and has no extension to S1. This contra-dicts the assumption that A(G) is rigid, since this assumption implies thatit is S1-compatible. Therefore, G is a connected graph.

It is clear that if G is connected, then Gdual is connected. This concludesthe proof of the lemma.

All the results on the rigidity of Laplacians hold for weighted Laplacians.

2.7.2. Elastic Struts in Two Dimensions. The second family ofmatrices model a collection of pin-jointed struts. Such a collection mayform a rigid structure called a truss (e.g., a triangle in two dimensions) ora non-rigid structure called a mechanism (e.g., two struts connected by apin). The rigidity graph of such a structure, however, is never connected:it has no edges at all. Therefore, the rigidity graph can never show thatthe underlying structure is rigid. This shows that our analysis indeed onlyprovides sufficient, but not necessary conditions. We carry out the anal-ysis nontheless to give another example of how to derive mutual rigidityconditions.

We note that there is a combinatorial technique that can determinewhether such a structure is rigid, under a technical assumption on thegeometrical location of the pins. The structure is modeled by a graph inwhich vertices correspond to pins (assuming there is a pin at the end ofeach strut) and in which edges correspond to struts. If the pins are an anappropriately-defined general position, then several equivalent conditionson the graph characterize exactly the rigidity of the structure [88, 101,106, 155, 91]. These conditions can be tested in O(n2) operations [78].

Our technique is more general but less sharp than these techniques. Itapplies to any finite-element matrix, but it only provides sufficient con-ditions for rigidity. In the cases of two-dimensional struts, our sufficient


conditions are never satisfied. We show later in this section that our tech-nique does work for other families of elastic structures.

Definition 2.7.5. ([144]) Let P = plnl=1 be a set of different pointsin the plane, pl = (xl, yl). For every pi 6= pj, let v(i,j) be the 2P -by-1 vectordefined by

v(k,j)i =

(xk − xj)/rk,j i = 2k − 1

(yk − yj)/rk,j i = 2k

−(xk − xj)/rk,j i = 2j − 1

−(yk − yj)/rk,j i = 2j

0 otherwise

,

where rk,j =√

(xk − xj)2 + (yk − yj)2. We define the (i, j) strut matrix to

be A(i,j) = v(i,j) v(i,j) T .

Definition 2.7.6. Given a collection P = plnl=1 of points in the plane,we define the translation and rotation vectors

N(x)P =

1010...10

, N

(y)P =

0101...01

, and N

(r)P =

−y1

x1

−y2

x2...−ynxn

.

The planar null-space of the collection is SP = spanNP , where NP =[N

(x)P N

(y)P N

(r)P

].

The next lemma shows that the strut matrices are SP compatible.

Lemma 2.7.7. Let P = plnl=1 be a set of different points in the planethat share no x coordinates and no y coordinates. Let A(i,j) be an (i, j)-strutmatrix for some i and j. Then

(1) A(i,j) is symmetric and positive semidefinite.(2) NA(i,j) = 2i− 1, 2i, 2j − 1, 2j.(3) A(i,j) is SP -compatible.

Proof. The first two claims in the lemma follow directly from thedefinition of A(i,j). We show that A(i,j) is SP -compatible by showing thatthe columns of ΨA(i,j)NP form a basis for enull(A(i,j)).

A direct calculation, which we omit, shows that AΨA(i,j)NP = 0. Thepoints pi and pj are different, so the rank of ΨA(i,j)NP is 3. The rank ofA(i,j) is 1, so its essential null space has dimension 3. Therefore, ΨA(i,j)NP

spans enull(A(i,j)), so A(i,j) is SP compatible.

The following lemma indicates that the rigidity graph of a collectionof strut matrices contains only trivial edges (self loops, which are alwayspresent).


Lemma 2.7.8. Let P = plnl=1 be a set of different points in the planethat share no x coordinates and no y coordinates. Let A(i,j) and A(k,`)

be two strut matrices. A(i,j) and A(k,`) are mutually rigid if and only ifi, j = k, `.

Proof. Assume A(i,j) and A(k,`) are mutually rigid. By Lemma 2.5.12,rank(ΨA(i,j),A(k,`)NP ) = 3. Therefore, using Lemma 2.7.7, |2i − 1, 2i, 2j −1, 2j ∩ 2k − 1, 2k, 2`− 1, 2`| ≥ 3. This implies that i, j = k, `.

The other direction is immediate; a matrix is always mutually rigid withitself.

2.7.3. Elastic Triangles in Two Dimensions. We now study an-other family of matrices that also arise in two-dimensional linear elasticity,matrices that model triangular elements. The rigidity graph of such a col-lection can be connected, so the rigidity graph can sometimes tell us thatthe structure is rigid. There are also cases in which the structure is rigidbut its rigidity graph is not connected.

Definition 2.7.9. Let P = plnl=1 be a set of different points in theplane, pl = (xl, yl), let v(i,j) and A(i,j) be defined as in Definition 2.7.5. Forthree different points pi, pj, and pk, we define the (i, j, k) element matrixin this family to be

A(i,j,k) = A(i,j) + A(i,k) + A(j,k) = v(i,j) v(i,j) T + v(j,k) v(j,k) T + v(k,i) v(k,i) T .

We note that this definition of A(i,j,k) is numerically different than the def-inition that results from standard finite-elements analysis (but the sparsitypatterns and the null spaces are the same).

The next lemma is the equivalent of Lemma 2.7.7. We omit the proof,which is similar to the proof of Lemma 2.7.7.

Lemma 2.7.10. Let P = plnl=1 be a set of affinely independent pointsin the plane. Let A be an (i, j, k) element matrix. Then

(1) A is symmetric and positive semidefinite.(2) NA = 2i− 1, 2i, 2j − 1, 2j, 2k − 1, 2k.(3) A is SP -compatible, where SP is the planar null space defined in

Definition 2.7.6.

The following lemma characterizes mutual rigidity between SP -compatiblematrices.

Lemma 2.7.11. Let P = plnl=1 be a set of different points in the plane.Let A and B be SP -compatible matrices. Assume that there exist i and j 6= isuch that 2i− 1, 2i, 2j − 1, 2j ⊆ NA ∩NB. Then, A and B are mutuallyrigid.

Proof. By Lemma 2.5.12, it is sufficient to show that ΨA,BNP isrank-3. Indeed, it is easy to validate that since the points are different,rank(ΨA,BNP ) > 3.


(a) (b)

(c) (d)

Figure 2.7.1. Triangular plane elements and their rigiditygraphs (in blue).

The last lemma of this section shows how to construct the rigidity graphfor this family of matrices.

Lemma 2.7.12. Let P = plnl=1 be a set of different points in the plane.Let A = A(i,j,k) and let B = A(p,q,r) be element matrices Then, A and B aremutually rigid if and only if |i, j, k ∩ p, q, r| ≥ 2.

Proof. If |i, j, k ∩ p, q, r| ≥ 2, there exist m 6= l such that 2m−1, 2m, 2l−1, 2l ⊆ NA∩NB. Then, by Lemma 2.7.11, A and B are mutuallyrigid.

If A and B are mutually rigid, by Lemma 2.5.12, ΨA,BNP is rank-3.Therefore, |NA ∩ NB| ≥ 3. By Lemma 2.7.11, |2i − 1, 2i, 2j − 1, 2j, 2k −1, 2k ∩ 2p − 1, 2p, 2q − 1, 2q, 2r − 1, 2r| ≥ 3. Therefore, |i, j, k ∩p, q, r| ≥ 2.

Informally speaking, Lemma 2.7.12 shows that edges in the rigiditygraph correspond to pairs of triangles whose mutual rigidity is evident:they share a side. The lemma can be generalized to higher dimensions:elastic elements are mutually rigid if and only if they share a face. Forelasticity, this may be a trivial statement, but it shows that our algebraicdefinition of mutual rigidity indeed captures the domain-specific notion ofrigidity.

Figure 2.7.1 shows a few examples of triangular plane elements and theirrigidity graphs. The structures in cases (a) and (b) are not rigid, and the

2.8. Rigid Sparsifications 60

rigidity graph is not connected. Case (c) is rigid, and the rigidity graph isconnected. Case (d) is rigid, but the rigidity graph does not show it; thegraph is not connected. This shows, again, that connectivity of the rigiditygraph is not a necessary condition for rigidity.

2.8. Rigid Sparsifications

Our next goal is to sparsify a matrix A defined as a sum of S-compatiblesymmetric positive semidefinite n-by-n matrices, but without changingnull(A). By sparsify we mean that linear systems of the form Bz = r,where B is the sparsification of A, should be easier to solve than linearsystems of the form Ax = b. In this sense, B is sparser than A if it hasa subset of the nonzeros of A, or if the pattern graph of B has smallerbalanced vertex separators than the pattern graph of A [64, 105]. Theremay be other meanings.

2.8.1. Dropping Vertices from the Rigidity Graph. Perhaps themost obvious way to sparsify A =

∑eAe is to drop some of the element

matrices from the assembly. This section gives a condition that guaranteesthat the subset-sum has the same range and null space as A. The analysisis inductive: it analyzes the effect of dropping one element at a time.

Lemma 2.8.1. Let G be the rigidity graph of a collection A1, A2, . . . , Akof S-compatible symmetric positive semidefinite n-by-n matrices. Let H1, H2, . . . , H`be the connected components of the G. Assume that NHi

∩ NHj= φ for

every i 6= j.Let A =

∑kf=1Af and let

B =k∑

f=1f 6=e

Af = A− Ae

for some vertex Ae. If

(1) G and the rigidity graph of B have the same number of connectedcomponents, and

(2) NA = NB,

then the matrices A and B have the same null space.

Proof. By Corollary 2.3.6 and by the condition NA = NB, it is suffi-cient to show that A and B are mutually rigid. By Lemma 2.4.1, A = B+Aeis rigid with respect to B. All that is left to show is that B is rigid withrespect to A.

Assume without loss of generality that Ae is in H1. Let

C =∑Af∈H1

f 6=e

Af .


The first condition of the lemma implies that H1\Ae is a nonempty con-nected subgraph of G. Therefore, there exist a vertex Ac in H1\Ae suchthat Ae and Ac are mutually rigid. Applying Theorem 2.6.3 to Ae, Acand to H1\Ae shows that Ae + Ac and C are mutually rigid. There-fore, by Lemma 2.4.2, C and C + Ae + Ac are mutually rigid. Finally, byLemma 2.4.3, C and C + Ae are mutually rigid.

Let D =∑`

i=2

∑Af∈Hi

Af . We have that ND ∩ NC = φ and ND ∩NC+Ae = φ, since NHi

are pairwise disjoint. By Lemma 2.4.4, we have thatC+D and C+Ae +D are mutually rigid. Therefore, A = C+Ae +D andB = C +D are mutually rigid, so null(A) = null(B).

In particular, if G is connected, then the lemma allows us to drop ele-ment matrices only as long as the rigidity graph of the remaining elementsremains connected. Clearly, there are cases where we can drop an elementmatrix that would disconnect the rigidity graph without changing the nullspace of the sum. In this case dropping the element violates the sufficientcondition stated in Lemma 2.8.1, but without actually changing the nullspace. For example, dropping A(2,5,6) from the structure shown in Fig-ure 2.7.1.(c) leads to the structure shown in Figure 2.7.1.(d), which is alsorigid, but has a disconnected rigidity graph.

The examples shown in Figure 2.7.1 parts (a) and (b) show that thelemma does not hold if the NHi

are not mutually disjoint. Dropping A(3,4,5)

from the structure shown in part (a) of the figure gives the structure shownin part (b). The rigidity graphs of both structures have the same numberof connected components, 2, and NA = NA−A(3,4,5) . But the null space ofthe structure in (a) has dimension 4 (rigid body motions and a rotationaround p3), where as the null space of the structure in (b) has dimension 6(separate rigid body motions for the two elements).

If we use Lemma 2.8.1 to construct a preconditioner B by repeatedlydropping element matrices from the sum A =

∑iAi, the generalized eigen-

values of (B,A) are clearly bounded from above by 1, since for any λ thatsatisfies Bx = λAx for an x 6∈ null(A) we have

λ ≤ maxx

Ax 6=0

xTBx

xTAx

= maxx

Ax6=0

xT(∑

i∈S⊂1,...kAi

)x

xT(∑k

i=1Ai

)x

≤ 1 .

2.8.2. Dropping Edges from the Rigidity Graph by FretsawExtensions. We now show and analyze a more sophisticated sparsificationtechnique that drops edges from the rigidity graph. This technique allowsus to algebraically express the physical action of cutting slits in a structurewithout changing its qualitative behavior (its null modes) and its material


properties. The technique drops edges from the rigidity graph of a givenstructure, then modifies the indices of nonzero rows and columns in elementmatrices to simulate the action of cutting slits along element boundarieswhere edges have been dropped. Figure 2.8.1 illustrates the cutting ideaand its connection to dropped edges in the rigidity graph. The techniqueis quite complex to define and analyze.

Definition 2.8.2. An extension mapping for matrices with n rows isa length ` vector s =

[s1 s2 · · · s`

]of integers in the range 1 to n. The

master extension matrix P of an extension mapping s is an (n + `)-by-nmatrix with a single 1 in each row,

P(s)ij =

1 i ≤ n and j = i

1 i > n and j = si−n0 otherwise .

An extension matrix Q(s) of an extension mapping s is an (n+`)-by-n witha single 1 in each column and at most a single 1 in each row such that

P(s)ij = 0 implies Q

(s)ij = 0. When the mapping is clear from the context, we

drop the superscript from the extension matrices.

In the product Q(s)A of an extension matrix Q(s) and an arbitrary ma-trix A, each row of the product is either all zeros or a copy of some rowof A, and each row of A is mapped to a row of the product. In particular,row i of A is either mapped to row i of the product or to row n+ j wheresj = i.

The following lemma states some properties of extension matrices andtheir relation to the projection matrices ΨA. We omit its proof.

Lemma 2.8.3. Let P = P (s) and Q = Q(s) be a master extension matrixand an extension matrix for some extension mapping s of length `. Let Abe an n-by-n symmetric matrix and let In be the n-by-n identity matrix.Then QTQ = P TQ = QTP = In and ΨQAQT = QΨAQ

T .

Definition 2.8.4. The extension of a subspace S ⊆ Rn under an exten-sion mapping s is the subspace span

(P (s)N

)where N is a matrix whose

columns form a basis for S. To keep the notation simple, we abuse thenotation and denote this space by P (s)S.

Lemma 2.8.5. The space P (s)S depends only on P (s) and on S, not on aparticular basis N . That is, for any two matrices N1 and N2 whose columnsspan S, span

(P (s)N1

)= span

(P (s)N2

).

Proof. Let x ∈ span(P (s)N1

). There exists a vector y such that

x = P (s)N1y. Since N1y ∈ S, there exists a vector z such that N1y = N2z.Therefore, x = P (s)N1y = P (s)N2z ∈ span(P (s)N2). This implies thatspan

(P (s)N1

)⊆ span

(P (s)N2

). Equality follows by symmetry.


Note that span(P (s)N

)is essentially the same space as N , but with

some coordinates replicated. This observation serves as an intuition for theproofs of the following two lemmas.

The following lemma shows that the extension of S-compatible sym-metric positive semidefinite matrix retains its essential properties.

Lemma 2.8.6. Let A be an S-compatible symmetric positive semidefi-nite matrix, and let Q = Q(s) be an extension matrix for some extensionmapping s. Then QAQT is symmetric positive semidefinite and compatiblewith P (s)S.

Proof. The matrix QAQT is symmetric since A is symmetric. Foran arbitrary vector x, let y = QTx. We have that xTQAQTx = yTAy ≥0, since A is positive semidefinite. This implies that QAQT is positivesemidefinite.

We now show that QAQT is compatible with P (s)S. Let N be amatrix whose columns form a basis for S. It is sufficient to show thatspan(ΨQAQTP (s)N) = enull(QAQT ).

By Lemma 2.8.3,

ΨQAQTP (s)N = QΨAQTP (s)N = QΨAN .

Therefore, span(ΨQAQTP (s)N) = span(QΨAN). Let E be a matrix whosecolumns form a basis for enull(A). Since A is S-compatible, span(ΨAN) =span(E). Moreover, since Q has full rank, span(QΨAN) = span(QE).Therefore, span(ΨQAQTP (s)N) = span(QE).

Now, it is sufficient to show that span(QE) = enull(QAQT ) in order toprove the lemma. In the special case, where Q is an (n + `)-by-n identitymatrix, this equality directly follows the fact that span(E) = enull(A).In the general case, Q is a row permutation of the (n + `)-by-n identitymatrix, and the equality holds since renaming the variables does not changeanything.

Similarly-extended matrices maintain their rigidity relationship.

Lemma 2.8.7. Let Aiki=1 be a collection of S-compatible symmetric

positive semidefinite matrices, and let Qe = Q(s)e and Qf = Q

(s)f be extension

matrices for some s, such that [Qe] : ,j = [Qf ] : ,j for every j ∈ NAe ∩ NAf.

Then QeAeQTe is rigid with respect to QfAfQ

Tf if and only if Ae is rigid

with respect to Af .In particular, if Q is the (n+`)-by-n identity matrix (the first n columns

of an (n+`)-by-(n+`) identity matrix; such a matrix is an extension matrixfor any s), then (Ae, Af ) is an edge of the rigidity graph of Aiki=1 if andonly if (QAeQ

T , QAfQT ) is an edge of the rigidity graph of QAiQTki=1.

Proof. We start by some immediate definitions and observations. LetN be an n-by-` matrix whose columns form a basis for S. Let P (s) be the


corresponding master extention matrix. By Lemma 2.8.6, the matrix P (s)

is a basis for P (s)S, and QeAeQTe and QfAfQ

Tf are P (s)S-compatible.

By Lemma 2.5.12, the matrices Ae and Af are mutually rigid if and onlyif ΨAe,Af

N is rank-`. Since Qe is full-rank, ΨAe,AfN is rank-` if and only if

QeΨAe,AfN is rank-`. A simple computation reveals that ΨAeQ

TeQfΨAf

=ΨAe,Af

. Therefore, by applying Lemma 2.8.3 we get

QeΨAe,AfN = QeΨAeQ

TeQfΨAf

N

= QeΨAeQTeQfΨAf

(QTf P

(S))N

= ΨQeAeQTeΨQfAfQ

TfP (S)N

= ΨQeAeQTe ,QfAfQ

TfP (S)N .

Therefore, ΨAe,AfN is rank-` if and only if ΨQeAeQT

e ,QfAfQTfP (s)N is rank-`.

Applying Lemma 2.5.12 again, we get that ΨQeAeQTe ,QfAfQ

TfP (s)N is rank-`

if and only if QeAeQTe and QfAfQ

Tf are mutually rigid. This concludes the

proof of the lemma.

We are particularly interested in certain extensions, described by thefollowing definition.

Definition 2.8.8. Let Aiki=1 be a collection of S-compatible sym-metric positive semidefinite matrices, let s be an extension mapping, andlet Qiki=1 be a collection of extension matrices for this s. Let G be the

rigidity graph of Aiki=1, and let G be the rigidity graph of QiAiQTi ki=1.

If

• For every j ∈ NA, there is some i such that j ∈ NAiand [Qi]j,j = 1,

• for every connected component Ai1 , . . . , Aim of G, the matrices

Qi1Ai1QTi2, . . . , QimAimQ

Tim are a connected component of G and

vice versa, and• for every connected component Ai1 , . . . , Aimof G, there is at least

one j ∈ i1, . . . , im such thatQj is the (n+`)-by-n identity matrix,

then we say that QiAiQTi ki=1 is a fretsaw extension of Aiki=1. (The

rationale behind the name fretsaw is explained below.) When the Qis are

clear from the context, we use F(A) to denote∑k

i=1QiAiQTi ; we call this

matrix the fretsaw extension of A =∑k

i=1Ai. For an extension mapping oflength |s| = `, and an n-by-n matrix A, we call F(A) the (n+ `)-by-(n+ `)fretsaw extension of A.

We note that the first fretsaw condition ensures that NA ⊆ NF(A).

Transforming an extended matrix B =∑Q

(s)i Ai Q

(s)i

Tback to A =∑

Ai is simple, as shown in the next lemma.

Lemma 2.8.9. Let A1, A2, . . . , Ak be a collection of n-by-n matrices.Let P be a master extension matrix for some extension mapping s and


let Qiki=1 be a collection of extension matrices for this s. Denote A =∑ki=1Ai and B =

∑ki=1 QiAiQ

Ti . Then A = P TBP .

Proof. By Lemma 2.8.3, P TQi = I, so

P TBP =k∑i=1

P TQiAiQTi P =

k∑i=1

Ai = A .

Definition 2.8.10. Let B be an (n+ `)-by-(n+ `) matrix. Partition

B =

[B11 B12

B21 B22

]such that B11 is n-by-n and B22 is `-by-`. If B22 is nonsingular, we denoteschur`(B) = B11−B12B

−122 B21. When ` is clear from the context, we simply

use schur(B).

The following theorem is the main structural result of this section. Thetheorem characterizes the relationship of the null spaces of a matrix andits fretsaw extension. In particular, it lists conditions that guarantee thepreservation of the null space under a fretsaw extension.

Theorem 2.8.11. Let A =∑k

i=1Ai, where A1, A2, . . . , Ak is a collectionof S-compatible symmetric positive semidefinite n-by-n matrices, let Q bethe (n + `)-by-n identity matrix, and let F(A) be a fretsaw extension ofA. Assume that NA = 1, . . . , n. If the rigidity graph of A1, A2, . . . , Ak isconnected, then

(1) QAQT and F(A) are mutually rigid.(2) For every x ∈ enull(QAQT ), there exist a vector y ∈ enull(F(A))

such that ΨQAQT y = x.(3) null(F(A)) ⊆ null(QAQT ).(4) If schur(F(A)) exists, then null(A) = null(schur(F(A))).

Proof. Let Qiki=1 be the collection of the extension matrices usedin F(A). By Lemma 2.8.6, the matrices in collections QiAiQ

Ti ki=1 and

QAiQTki=1 are compatible with P (s)S. By definition, the rigidity graph ofQiAiQ

Ti ki=1 is connected. By Lemma 2.8.7 and the assumption that the

rigidity graph of Aiki=1 is connected, the rigidity graph of QAiQTki=1

is also connected. By definition, the rigidity graph of QAiQTki=1 sharesat least one vertex with the rigidity graph of QiAiQ

Ti ki=1. Therefore, by

Lemma 2.6.3, QAQT and F(A) are mutually rigid.The second part of the lemma follows the fact that QAQT is rigid with

respect to F(A) and that NQAQT = NA ⊆ NF(A).The matrix F(A) is rigid with respect to QAQT . We also have that

NQAQT = NA ⊆ NF(A). Therefore, by Lemma 2.3.5, null(F(A)) ⊆ null(QAQT ).This proves the third part of the lemma.

The fourth part of the lemma follows from Lemma 2.3.7 and the factthat QAQT and F(A) are mutually rigid.


The following lemma shows that the null space is also preserved whenadding an spsd matrix. This is the building block which later enables us toextend the last theorem to cases where the rigidity graph is not connected.

Lemma 2.8.12. Let A =∑k

i=1Ai + B, where A1, A2, . . . , Ak is a col-lection of S-compatible symmetric positive semidefinite n-by-n matrices, Bis a positive semidefinite n-by-n matrix , and let G be the rigidity graph ofAiki=1. Let F(A) be an (n+`)-by-(n+`) fretsaw extension of Aiki=1, andlet Q be the corresponding (n + `)-by-n identity matrix. If G is connectedthen

null(F(A) +QBQT ) ⊆ null(QAQT ) .

Proof. Assume G is connected. Let A denote∑k

i=1Ai, then A =

A+B. By Theorem 2.8.11, null(F(A)) ⊆ null(QAQT ). Therefore,

null(F(A)) ∩ null(QBQT ) ⊆ null(QAQT ) ∩ null(QBQT )(2.8.1)

For every two spsd square matrices of the same order C and D, null(C +

D) = null(C)∩ null(D). Clearly, QBQT ,F(A) and QAQT are spsd matri-ces. Therefore,

null(F(A) +QBQT ) = null(F(A)) ∩ null(QBQT )

and

null(QAQT ) = null(QAQT +QBQT ) = null(QAQT ) ∩ null(QBQT ) .

Combining the last two equalities with equation 2.8.1, we get

null(F(A) +QBQT ) ⊆ null(QAQT ) .

The following lemma shows that certain fretsaw extensions of a matrixcollection with a rigidity graph with two connected components also havethe desirable null space containment properties.

Lemma 2.8.13. Let K = A + B, where A =∑k1

i=1Ai, B =∑k2

i=1Bi,and A1, A2, . . . , Ak1 and B1, B2, . . . , Bk2 are collections of S-compatible sym-metric positive semidefinite n-by-n matrices. Let G be the rigidity graph ofAik1i=1

⋃Bik2i=1. Assume G is exactly two connected components, one is

Aik1i=1 and the other is Bik2i=1.Let F(A) be an (n+`1)-by-(n+`1) fretsaw extension of Aik1i=1, and let

Q1 be the corresponding (n+ `1)-by-n identity matrix. Let F(Q1BQT1 ) be a

fretsaw extension of Q1BiQT1

k2i=1. Let Q2 be the corresponding (n+`1+`2)-

by-(n+ `1) identity matrix. then

(1) Q2F(A)QT2 + F(Q1BQ

T1 ) is a fretsaw extension of K.

(2) null(Q2F(A)QT2 + F(Q1BQ

T1 )) ⊆ null(Q2Q1KQ

T1Q

T2 ).

Proof. The first part follows directly from the way the fretsaw exten-sion of K. It clearly satisfies all the conditions of Definition 2.8.8.


By Lemma 2.8.12,

(2.8.2) null(F(A) +Q1BQT1 ) ⊆ null(Q1KQ

T1 ) .

and similarly,

(2.8.3) null(F(Q1BQT1 ) +Q2F(A)QT

2 ) ⊆ null(Q2(Q1BQT1 + F(A))QT

2 ) .

By equation 2.8.2,

(2.8.4) null(Q2(F(A) +Q1BQT1 )QT

2 ) ⊆ null(Q2Q1KQT1Q

T2 ) .

Combining equations 2.8.3 and 2.8.4, we get

null(F(Q1BQT1 ) +Q2F(A)QT

2 ) ⊆ null(Q2Q1KQT1Q

T2 ) ,

which concludes the proof of the lemma.

A proof technique similar to the proof in the last lemma can be used toshow that fretsaw extensions that are constructed rigidity-component at atime have the desirable null space containment properties. This result issummarized in the next corollary.

Corollary 2.8.14. Let A =∑k

i=1 Ai, where A1, A2, . . . , Ak is a collec-tion of S-compatible symmetric positive semidefinite n-by-n matrices, andlet G be the rigidity graph of Aiki=1. Let Hjcj=1 be the connectivity com-

ponents of G. Let A(j) =∑

Ai∈V (Hj) Ai .

Let F(A(1)) be an (n+ `(1))-by-(n+ `(1)) fretsaw extension of A(1), andlet Q(1) be the corresponding (n+ `(1))-by-n identity matrix. Let F(A(2)) be

an (n+`(1)+`(2))-by-(n+`(1)+`(2)) fretsaw extension of Q(1)A(2) Q(1) T , andlet Q(2) be the corresponding (n + `(1) + `(2))-by-(n + `(1)) identity matrix.

Let F(A(j)) be an (n +∑j−1

t=1 `(j))-by-(n +

∑j−1t=1 `

(j)) fretsaw extension of

Q(j−1) · · ·Q(1)A(j) Q(1) T · · · Q(j−1) T , and let Q(j) be the corresponding (n+∑jt=1 `

(j))-by-(n+∑j−1

t=1 `(j)) identity matrix.

Let Q = Q(1) · · ·Q(c) and

F(A) = Q(c) · · ·Q(2)F(A(1)) Q(2) T · · · Q(c) T

+Q(c) · · ·Q(3)F(A(2)) Q(3) T · · · Q(c) T

+ . . .

+F(A(c)) .

Then,

null(F(A)) ⊆ null(QAQT ) .

2.8.3. Constructing a Fretsaw Extension from a Spanning Tree.We now show a practical way to construct nontrivial fretsaw extensions.The extensions that we build here are essentially as sparse as possible: wecan factor F(A) with no fill. Our simple spanning-tree extensions may notbe effective preconditioners (the generalized eigenvalues may be large), but


the construction shows that there are efficient algorithms for constructingnontrivial fretsaw extensions.

Let A = A1, A2, . . . , Ak be a collection of S-compatible symmetricpositive semidefinite n-by-n matrices. Let G = (A, E) be the rigidity graphof A. Without loss of generality, we assume that G is connected (otherwisewe repeat the construction for each connected component of G). The con-struction builds the Qis by columns. This introduces a slight notationaldifficulty, since we do not know the number of rows in the Qis until theconstruction ends. We use the convention that the columns are tall enough(nk is tall enough) and then chop the Qis to remove the rows that are zeroin all of them. We denote by er the rth (long enough) unit vector.

We use a spanning tree T ofG to define an extension F(A). We initializea variable r to n. This variable stores the index of the last nonzero row inthe Qis. We also initialize the extension mapping s = 〈〉. The algorithmiterates over the column indices j ∈ 1, . . . , n (in any order). In iterationj, we construct column j of Q1 . . . , Qk.

We begin iteration j by setting the jth column of Q1 to ej. This ensuresthat Q1 is an identity matrix, so the third fretsaw condition is automaticallysatisfied.

We now construct the set V (j) = Ai ∈ A|j ∈ NAi of elements that are

incident on the index j. We also construct the subgraph G(j) = (V (j), E(j))of T that is induced by V (j). We partition G(j) into its connected compo-nents and process each component separately. The ordering of the compo-nents is arbitrary, except that if A1 ∈ V (j), then we process the componentcontaining A1 first. Let Ai1 , Ai2 , . . . , Aim ⊆ V (j) be the vertices of thenext component to be processed. If this component is the first componentof G(j), then we set the jth columns of Qi1 , . . . , Qim to ej. Otherwise, weincrement r, set the jth columns of Qi1 , . . . , Qim to er, and concatenate thecurrent index j to the end of s.

This process specifies the jth column of every Qi such that j ∈ NAi.

We complete the construction of the Qis by setting the jth column of everyQi such that j 6∈ NAi

to ej. A pseudocode for the this process can be foundin Algorithm 1.

Sometimes the row/column indices of A =∑Ai have a natural group-

ing. For example, in problems arising in two-dimensional linear elasticity,each point in the geometry of the discrete structure is associated with twoindices, an x-direction index, say j1, and a y-direction index, say j2. Thisusually implies that G(j1) and G(j2) are identical graphs. In such cases, itmay be wise to extend A consistently : we order the connected componentsof G(j1) and G(j2) consistently, which means that [Qi] : ,j1

= ej1 if and only

if [Qi] : ,j2= ej2 . A consitent extension has a physical interpretation in

terms of slits in the material, whereas an inconsistent extention may haveno straightforward physical meaning.

Figure 2.8.1 illustrates the construction of a spanning-tree fretsaw-extended matrix for a structure consisting of linear elastic elements in two


(a) (b)

(c) (d)

Figure 2.8.1. The construction of a spanning-tree fretsaw-extended matrix. (a) The elements of the original structureand its rigidity graph; elements are mutually rigid iff theyshare a side. (b) A spanning tree T of the rigidity graph.(c) The structure induced by the spanning tree; duplicatednodes are marked by small circles. In the illustration, thetriangles have been slightly shrunk to show how rigidity re-lationships have been severed, but the element matrices areonly permuted, so they still model the original triangles. (d)The fretsaw-extended structure.


Algorithm 1 Pseudocode for the construction of fretsaw spanning-treeextensions.

TreeFretsawExtension(A, G)

. A = A1, A2, . . . , Ak is a collection of S-compatible SPSD n-by-nmatrices

. G = (A, E) is the rigidity graph of AT ←A spanning tree of G

r ← n

For j ∈ 1, . . . , n do . in any order

. Construct column j of Q1 . . . , Qk

Set the jth column of Q1 to be ej

V (j) ← Ai ∈ A|j ∈ NAi . the elements that are incident on the

index j

E(j) = (Ap, Aq) ∈ T |Ap, Aq ∈ V (j)G(j) ← (V (j), E(j)) . the subgraph of T that is induced by V (j)

G(j)1 , G

(j)2 , . . . , G

(j)cj ← The connectivity components of G(j)

. The order is arbitrary, except that if A1 ∈ V (j), then A1 ∈ G(j)1

For all Ap ∈ G(j)1 , set the jth columns of Qp to ej

For G(j)i ∈ G

(j)2 , . . . , G

(j)cj do . in any order

r ← r + 1

For all Ap ∈ G(j)i , set the jth columns of Qp to er

End For

For i ∈ 1, . . . , k do

If j 6∈ NAi, then set the jth columns of Qi to ej

End For

End For

dimensions. The figure explains the rationale behind the term fretsaw.A fretsaw is a fine-toothed saw held under tension, designed for cuttingthin slits in flat materials, such as sheets of plywood. When applied totwo dimensional elastic structures, like the one shown in Figure 2.8.1, thespanning-tree fretsaw construction appears to cut the original structure likea fretsaw.

Once the Qis are constructed, we form F(A) =∑k

i=1QiAiQTi . The

next theorem shows that the Qis are extension matrices for some s andthat F(A) is a fretsaw extension of A.

Theorem 2.8.15. Let Aiki=1 be a collection of S-compatible symmet-ric positive semidefinite n-by-n matrices, let s be the extension mapping


that our spanning-tree algorithm constructs, and let Qiki=1 be the exten-

sion matrices for this extension. Then F(A) =∑k

i=1 QiAiQTi is a fretsaw

extension of A.

Proof. We first show that the Qis are indeed valid extension matricesfor a single s. We can construct an s step-by-step along the constructionof the Qis. We initialize s = 〈〉. When we increment the value r, weconcatenate the current index j to the end of s. By definition, Qi containsa nonzero in (r, j) if and only if s contains the value j in its (r−n)th position.Therefore, the Qis are consistent with P (s). Moreover, by definition, everycolumn j of a matrix Qi is either ej or er where r > n and r is unique inamong the columns of Qi. Therefore, the Qis are valid extension matrices.

Let G be the rigidity graph of Aiki=1, and let G be the rigidity graph ofQiAiQ

Ti ki=1. We show now that every connected componentAi1 , Ai2 , . . . , Aim in

G, the matrices Qi1Ai1QTi1, Qi2Ai2Q

Ti2, . . . , QimAimQ

Tim form a connected

component in G and vice versa. Let Ai1 , Ai2 , . . . , Aim be a connected com-ponent of G. Let T be the spanning tree used to create Qi1 , Qi2 , . . . , Qim .Let Ap and Aq be to matrices adjacent in T . For every index j ∈ NAp∩NAq ,

Ap and Aq belong to the same connected component of G(j). Therefore, Qp

and Qj coincide on their jth column. Therefore, by Lemma 2.8.7, QpApQTp

andQqAqQTq are mutually rigid. Therefore, Qi1Ai1Q

Ti1, Qi2Ai2Q


Tim

is a connected component in G. In a similar manner ifQi1Ai1QTi1, Qi2Ai2Q


Tim

form a connected component in G, by Lemma 2.8.7, Ai1 , Ai2 , . . . , Aim is aconnected component in G.

There are two additional properties that need to be verified in orderto show that F(A) is a fretsaw extension. By definition, the constructionensures that there is at least one Qi1 for every connected component whichis an (n + l)-by-n identity matrix. The second property that we needto show is that NA ⊆ NF(A). Let j ∈ NA. Let Ap be a matrix in the

connected component ofG(j) that was processed first. By definition, columnj of Qp contains ej. Therefore, j ∈ NQpApQT

p. Lemma 2.2.6, ensures that

j ∈ NF(A).

2.8.4. Perfect Elimination Orderings for Spanning-Tree Fret-saw Extensions. The spanning-tree fretsaw construction is motivated byan elimination ordering that guarantees that all the fill occurs within thenonzero structure of the element matrices. If [Ae]i,j 6= 0 for all i, j ∈ NA,

this ordering is a no-fill ordering of F(A).The analysis in the proof is closely related to the analysis of clique

trees. The spanning-tree fretsaw is a tree of cliques that satisfies the clique-intersection property defined by Blair and Peyton [21]. If the elementswere guaranteed to be maximal cliques, then Theorem 3.2 of [21] wouldguarantree a no-fill elimination ordering. But we are not certain whetherelement matrices are always maximal cliques in the graph of the assembledmatrix, so [21, Theorem 3.2] is not directly applicable.


Lemma 2.8.16. Let A = A1, A2, . . . , Ak be a collection of S-compatiblesymmetric positive semidefinite n-by-n matrices. Let F(A) be a spanning-

tree fretsaw extension of A =∑k

i=1Ai. There is a permutation matrix Γsuch that all the nonzeros Li,j of the Cholesky factor L of ΓF(A)ΓT satisfyi, j ∈ NΓQeAeQT

e ΓT for some e (e depends on i and j).

Proof. We root the spanning tree T of the rigidity graph at A1 andtake φ to be a postorder of this rooted tree. That is, φ is an ordering ofthe element matrices in which the leaves of the rooted tree appear first,followed by parents of leaves, etc. We construct an elimination ordering γincrementally. Initially, γ = 〈〉 is an empty ordering. Let Ae be the nextunprocessed element matrix in φ, and let Af be the parent of Ae in therooted tree (if it has a parent). Let i1, . . . , im be the indices in

NQeAeQTe\(NQfAfQ

Tf

⋃γ)

(with γ taken to be a set of already ordered indices). We concatenate〈i1, . . . , im〉 to γ in an arbitrary order. The permutation matrix Γ is thematrix that corresponds to γ. That is, Γi,γi

= 1.Now that we have specified Γ, we show that it limits fill as claimed.Claim A. Let i ∈ 1, . . . , n, n + 1, . . . , n + `. Let j be the column in

P (s) such that P(s)i,j 6= 0 (recall that every row in P (s) contains exactly one

nonzero). We denote by G(j)i the connected component of G(j) in which j

is mapped to i. The graph G(j)i is a connected subgraph of G(j) which is an

induced subgraph of a rooted tree. Therefore, G(j)i is itself a rooted tree.

We claim that i is added to γ during the processing of the root Ah of G(j)i .

Proof of Claim A. We first show that if i is added to γ, then it is added

during the processing of the root of G(j)i .Let Ae be the element during the

processing of which we add i to γ. We first show that Ae ∈ G(j)i . Clearly,

i ∈ NQeAeQTe. Therefore, j ∈ NAe . By the definition of G(j), Ae ∈ G(j), and

by the definition of G(j)i , Ae ∈ G(j)

i . Now suppose for contradiction that Aeis not the root of G

(j)i . Then Ae has a parent Af in G

(j)i . Because Af is

in G(j)i , i ∈ NQfAfQ

Tf, so the algorithm would not have added i to γ during

the processing of Ae. Therefore, Ae is the root of G(j)i .

To complete the proof of Claim A, we show that i is added to γ. Suppose

for contradiction that it is not. When we process the root Ah of G(j)i , i 6∈ γ.

But i cannot be in NQfAfQTf, where Af is the parent of Ah in the global

rooted tree. If it was, then j ∈ NAf, so Af would be in G(j), and because

it is connected to Ah, it must also be in G(j)i . But Ah is the root of G

(j)i ,

so i 6∈ NQfAfQTf. Therefore, i is added to γ. This concludes the proof of

Claim A.


Claim B. Just before Af is processed, γ is exactly the set

γ =i : i ∈ NQeAeQT

efor some Ae that appears before Af in φ

\(NQfAfQ

Tf∪i : i ∈ NQgAgQT

gfor some Ag that appears after Af in φ

).

Proof of Claim B. The claim follows by induction from the process ofconstructing γ and from the fact that φ is a postorder of the rooted tree.

Claim C. If Lr,c 6= 0 then r, c ∈ NΓQeAeQTe ΓT for some e.

Proof of Claim C. If Lr,c 6= 0, then either[ΓF(A)ΓT

]r,c6= 0 or there is

some ı < r, c such that Lr,ı 6= 0 and Lc,ı 6= 0. The first condition cannotviolate Claim B, because if

[ΓF(A)ΓT

]r,c6= 0 then there is some e such

that[ΓQeAeQ

Te ΓT

]r,c6= 0, so r, c ∈ NΓQeAeQT

e ΓT .

If for some r and c we have Lr,c 6= 0 because of the second condition,then let ı be the minimal index such that Lr,ı 6= 0 and Lr,ı 6= 0 for allsuch (r, c) pairs. This definition of ı guarantees that

[ΓF(A)ΓT

]r,ı6= 0 and[

ΓF(A)ΓT]c,ı6= 0. Define i = γı, r = γr and c = γc.

Let Af be the element during the processing of which i was added toγ. Because ı < r, c, when i is added to γ, r and c are not yet in γ.We claim that r and c are in NQfAfQ

Tf; if true, Claim C holds. Suppose

for contradiction that r 6∈ NQfAfQTf. By Claim B, r ∈ NAg for some Ag

that appears after Af in φ. Because[ΓF(A)ΓT

]r,ı6= 0 and because Ag is

symmetric, we must also have i ∈ NAg . But this implies that i cannot beadded to γ when Af is processed (by Claim B). This concludes the proofof Claim C and of the entire proof.

2.8.5. Quantitative analysis. Lemma 2.8.11 showed that if the rigid-ity graph of a finite-element matrixA =

∑Ae is connected and if schur(F(A))

exists for a fretsaw extension F(A), then A and schur(F(A)) have the samerange and null space. We now strengthen this result and show that the gen-eralized eigenvalues of (schur(F(A)), A) are bounded from above by 1. Wenote that schur(F(A)) can be implicitly used as a preconditioner; in thepreconditioning step of an iterative linear solver, we can solve a linear sys-tem whose coefficient matrix is F(A), not schur(F(A)) [70, Section 3.3]. Inparticular, the previous section showed that we can factor a spanning-treefretsaw-extension F(A) with essentially no fill.

Lemma 2.8.17. Let A1, A2, . . . , Ak be a collection of S-compatible sym-metric positive semidefinite n-by-n matrices. Let A =

∑ki=1Ai. Let F(A)

be a fretsaw extension of A. Then if schur(F(A)) exists, and λ is a finitegeneralized eigenvalue of the pencil (schur(F(A)), A), then λ ≤ 1.

Proof. We partition F(A) into

F(A) =

[B11 B12

BT12 B22

],


where B11 is n-by-n and B22 is `-by-`. By the assumption that schur(F(A))exists, B22 is symmetric positive definite. In this notation, schur(F(A)) =B11 − B12B

−122 B

T12. Let P be the (n + `)-by-n master extension matrix

corresponding to the fretsaw extension F(A) and let Q be the (n+ `)-by-nidentity matrix.

Let λmax be the maximal finite generalized eigenvalue of the pencil(schur(F(A)), A) and let x be the corresponding eigenvector. We let

x =

[x

−B−122 B21x

]and multiply it by F(A),

F(A)x =

[B11x+B12(−B−1

22 B21x)B21x+B22(−B−1

22 B21x)

]=

[schur(F(A))x

0

]=

[λmaxAx

0

]= λmaxQAQ

T x .

Multiplying both sides by xT , we obtain xTF(A)x = λmaxxTQAQT x.

We now show that xTF(A)x ≤ xTQAQT x. For a length-` vector y, de-

fine the function f(y) =[xT yT

]F(A)

[xy

]. We note that f(−B−1

22 B21x) =

xTF(A)x. For an arbitrary y,[xT yT

]F(A)

[xy

]=

[xT yT

] [B11 B12

BT12 B22

] [xy

]= xTB11x+ yTBT

12x+ xTB12y + yTB22y

= xT(B11 −B12B

−122 B

T12

)x+ xTB12B

−122 B

T12x

+yTBT12x+ xTB12y + yTB22y

= xT(B11 −B12B

−122 B

T12

)x

+(y +B−1

22 BT12x)TB22

(y +B−1

22 BT12x).

Because B22 is positive definite, f(y) is minimized at y = −B−122 B

T12x.

By Lemma 2.8.9, xTQAQT x = xTQ(P TF(A)P )QT x = xTP TF(A)Px.By the definition of a master extension matrix, the vector Px containsthe vector x in its first n coordinates, so Px =

[xT zT

]for some z and

xTP TF(A)Px = f(z). Since −B−122 B

T12x minimizes f ,

xTQAQT x = xTP TF(A)Px = f(z) ≥ f(−B−122 B

T12x) = xTF(A)x

This implies that λmax ≤ 1 and concludes the proof of the lemma.

2.9. Numerical Examples 75

0 50 100 150 200 250 300 35010-8

10-6

10-4

10-2

100

102

# Iterations

Resid

ual

A Poisson problem: 1012 nodes, 1012 unknowns

no preconditiongfretsaw-treecholinc(0)(3)

(2)(1)

(1)(2) (3)

0 200 400 600 800 1000 1200 140010-8

10-6

10-4

10-2

100

102

# Iterations

Res

idua

l

Linear Elasticity: 1012 nodes, 2024 unknownsno preconditiongfretsaw-treecholinc(0)

(2)(3)

(1)

(1)

(2)

Figure 2.9.1. A triangularization of a two-dimensional do-main (top) and convergence plots for two problems dis-cretized on this domain. The triangularization used in theplots is finer than the one shown in the top part of the fig-ure. The graph on the left shows the convergence of itera-tive solvers on a discretization of a Poisson problem, and thegraph on the right shows convergence on a linear-elasticityproblem, both with constant coefficients.

2.9. Numerical Examples

In this section we present experimental results that indicate that fretsaw-tree sparsifications can be used as preconditioners. We do not claim thatthey are particularly effective. Our only goal in this section is to demon-strate that fretsaw-tree sparsifications can be used computationally as pre-conditioners. The results presented in this section also suggest that thequalitative convergence behavior of fretsaw-extension preconditioners issimilar to that of Vaidya’s precondioners when applied to weighted Lapla-cians [37].

Figure 2.9.1 shows convergence results for an iterative solver (precondi-tioned conjugate gradients) with a fretsaw-tree preconditioner. The figure

2.9. Numerical Examples 76

no preconditiongfretsaw-treecholinc(0)

0 2000 4000 6000 8000 10000 1200010-3

10-2

10-1

100

101

102

103

104

105

106Linear Elasticity, 3516 nodes, 10548 unknowns

no preconditiongfretsaw-treecholinc(0)(3)

(2)(1)

(1)

Figure 2.9.2. A tetrahedral meshing of a three-dimensionalcylinder (top) and convergence plots for two problems dis-cretized on a similar but longer cylinder (bottom). The left-most part of the cylinder was removed to reveal the irregularmeshing inside.

shows results for two different physical two-dimensional problems that wediscretized on the same triangulated mesh. One problem was a Poissonproblem and the other a linear-elasticity problem, both with constant co-efficients and with Neumann (natural) boundary conditions. In each case,we constrained one or three unknowns belonging to a single triangle totransform the coefficient matrix into a non-singular one.

Each graph shows convergence results for three conjugate-gradientssolvers: with no preconditioning, with no-fill incomplete-Cholesky precon-ditioning (denoted cholinc(0) in the graphs), and with fretsaw-tree precon-ditioning. The fretsaw trees for the two problems are different, of course,because the rigidity graphs are different. We chose to compare the fretsaw-tree preconditioner with a no-fill incomplete-Cholesky preconditioner be-cause both are equally sparse.

Figure 2.9.2 shows similar plots for two three-dimensional problems de-fined on a cylinder, a Poisson problem and a linear elastic problem. Thematerial is isotropic, but the coefficients are variable; the material constantis 1+10x+1000y+1000z, where x ∈ [0, 35] and y, z ∈ [0, 0.2]. The tree thatwe constructed was a maximum spanning tree, where the weights chosenheuristically to reflect the norm and condition of element matrices.

The results show that fretsaw trees can be used as preconditioners.The experiments are too limited to fully judge them, but the experimentsdo indicate that they are not worse than another no-fill preconditioner.Two other observations on the graphs are (1) the fretsaw is better thanincomplete Cholesky on the two-dimensional Poisson problem, but the two

2.10. Concluding Remarks 77

preconditioners are comparable on the other problems, and (2) the steadylinear convergence behavior of the fretsaw trees is similar to the convergencebehavior of Vaidya’s preconditioners on weighted Laplacians [37].

2.10. Concluding Remarks

To keep the chapter readable and of reasonable length, we have omittedfrom it several topics, which we plan to cover in future works.

• Element matrices that represent boundary conditions. In much ofthis chapter, we have assumed that all the element matrices arecompatible with S. This means, in particular, that the elementmatrix is singular. In many practical computations, boundaryconditions are added to remove the singularity. We kept the dis-cussion focused on singular matrices to reduce clutter. We explorethe handling of boundary conditions in the next chapter.• Fretsaw constructions other than spanning-tree fretsaw extensions.

Previous work on combinatorial preconditioners indicates, boththeoretically and experimentally, that tree and tree-like precondi-tioners are not effective; augmented trees and other constructionsusually work better. We have developed augmented-spanning-treefretsaw extension algorithms for Laplacians, but they do not pro-vide better theoretical results than existing combinatorial prcon-ditioners for Laplacians. We decided to exclude them from thischapter.

In addition, there are several interesting problems that we have not yetsolved. The most interesting one is proving lower bounds on the general-ized eigenvalues of (schur(F(A)), A) and finding fretsaw constructions thatensure that this bound is not too small. A particularly interesting questionis whether this can be done by assigning weights to the edges of the rigiditygraph. In [45], Spielman and Daitch give a bound on the generalized eigen-values of (schur(F(A)), A) for the case of planar trusses made of triangleswith some geometric constraints. They have used the fretsaw-extensionframework with trees augmented with some edges [53]. Their solver isprovably efficient and runs with O(n5/4(log2 n log log n)3/4 log 1

ε) when n is

the number of joints in the truss and ε is the prescribed relative error. Anextension to the three dimensional case and to more general finite-elementmatrices is still missing.

Another question is what other results from spectral graph theory canbe generalized to finite-element matrices as defined in this chapter, andwhether the rigidity graph, perhaps weighted, would be useful in such gen-eralizations.

CHAPTER 3

Computing the Null Space of Finite ElementProblems

3.1. Introduction

The computation of the null space of finite element matrices is an im-portant task in many industrial applications. In this chapter1 we show amethod to efficiently compute the null space of symmetric positive semidef-inite finite-element matrices, possibly with additional linear equality con-straints. Our method is strictly algebraic and does not use the geometry ofthe underlying model; the only information needed is the model in elemen-tal form and a representation of the linear constraints. Our method worksfor any model with symmetric semidefinite elements. It works both whenthe constraints are symmetrically added to the finite-element matrix andwhen the constraints are appended to the symmetric finite-element matrixto create a rectangular matrix.

In structural mechanics the null space of a finite-element matrix corre-sponds to the zero energy modes of the structure. Such zero energy modesare important in static and dynamic analysis of floating structures [62], forexample, where the computed null space is used to solve a consistent butrank-deficient system of linear equations. Certain domain decompositionmethods also require the computation of the null space of floating substruc-tures. This is the case for FETI [57, 55], for the balancing domain decom-position method [109], and for some direct flexibility methods [58, 32].

Although most finite-element analyses are performed on non-singularmatrices, a nonempty null space can also be introduced due to an error inthe modeling phase. A model with two few constraints (boundary condi-tions) can lead to a singular coefficient matrix. If the linear solver factorsthe matrix with a backward stable factorization (such as LU with partialpivoting), the singularity can be detected fairly easily either as a break-down in the factorization process or using inverse iteration [89]. The failedfactorization process, is however, expensive. Other types of linear solvers,

1The results in this chapter were also submitted for publication as a paper in the jour-nal Computer Methods in Applied Mechanics and Engineering, co-authored by SivanToledo [137].This chapter also contains several technical algorithmic and theoretical details that wereomitted from the journal version.

78

3.2. Overview and Related Work 79

especially iterative ones, do not cope well with singular or nearly singu-lar matrices, which can cause the solver to converge extremely slowly ornot at all. Our method can address this issue by quickly checking if a fi-nite element model is sufficiently constrained; the scaling of our null spacecomputation routine is approximately linear.

Finally, algebraic multigrid methods such as smooth aggregation [84,148, 149] also benefit from an efficient computation of the null space ofsemidefinite finite-element matrices.

The remainder of the chapter is organized as follows. The next sectiongives an overview of our method and relate it to existing work. Section 3.3describes our method in detail. We present numerical results in Section 3.4.Finally, we summarize our results in Section 3.5.

3.2. Overview and Related Work

Let K =∑

eKe be an n-by-n finite-element matrix, where all the Kesare symmetric positive semidefinite element matrices. Let C be a c-by-nrow block of linear constraints (C may be empty). Our ultimate goal is tocompute the null space of the rectangular matrix

KC =

[KC

],

or of the symmetric square matrix[K CT

C

].

We compute the null space using a two phase process. In the first phasewe efficiently compute an (n+`)-by-(n+`) matrix F(K). The matrix F(K)is an easy-to-factor approximation of K that preserves its null space in (ina specific technical sense that we describe later). This theory behind thisapproximation was developed in an Chapter 2 (which was also publishedas [138]). The algorithmic aspects are described below in Section 3.3. Inthe second phase we compute the null space of

K =

[F(K)

C

],

where C is the matrix C padded with ` zero columns. We then extract thenull space of K from the computed null space of K. Any reliable method forcomputing the null space of K is appropriate here, including the methodsin [40, 63, 126].

The key insight in our method is to form K such that factoring it issignificantly easier than factoring KC . Once a matrix is factored using abackward-stable factorization, computing its null space is relatively easy,using inverse power iterations; this technique works well even for rectan-gular matrices [68]. But factoring KC (or even just K) is expensive forthree-dimensional models. Our method addresses this by replacing K by

3.2. Overview and Related Work 80

F(K), which preserves the null space but is easier to factor (even thoughits dimension is higher).

Our method uses a combinatorial relationship called mutual rigidity be-tween the Kes. This relationship, which we define in Section 3.3.1, is closelyrelated to element relationships defined in several papers; two mutuallyrigid elements are defined as mechanism free adjacent elements in [120],as elasticity connected elements in [44], and as adjacent in the natural as-sociation graph in [96, 97]. This relationship is also closely related to themechanism buster algorithm in [55].

There is, however, a fundamental difference between the element-relationshipdefinition in this thesis and the definitions in the papers cited above. Ourdefinition is purely algebraic; it uses only the information in the elementmatrices. All the other definitions use geometrical information, discretiza-tion specific information, or assumptions on the physics that the elementsmodel. Therefore, our method is more general and can be used with manyfinite-element modeling techniques. Moreover, our method is not restrictedto structural mechanics; it works equally well in electrostatic problems, forexample.

Our method also uses the inter-element relationship differently thanearlier methods. We use the rigidity relations and the underlying graphto combinatorially sparsify K. In essence, we form the stiffness matrixof another physical structure, which has the same null space as the givenstructure but is much easier to analyze (because its stiffness matrix is easierto factor).

The specific problem of computing the null space of large semidefinitesparse matrices arising in structural mechanics was explored in [56] and in[120]. In both cases the null space of rigid substructures was computed us-ing additional geometrical information. The method in this chapter workswithout this information, and therefore may be suitable for broader usagescenarios.

The rigidity relationship, as well as the similar relationships mentionedabove, is empty for skeletal truss structures. This means that other meth-ods should be used in this case. For two-dimensional trusses, there existsa combinatorial algorithm to determine the rigidity of a truss [91] (undera slightly different definition of rigidity), but it is not clear whether thealgorithm can be extended to three dimensions.

Several papers [40, 63, 126] proposed algorithms for finding the nullspace of another matrix arising in structural mechanics, the equilibriummatrix in the force method. More recent papers [96, 97] presented effi-cient algorithms to compute the null space of the equilibrium matrix, whenthe matrix was computed using specific finite-element formulation and spe-cific element types. The equilibrium matrix is underdetermined (has morecolumns than rows), whereas our work focuses on computing the null spaceof another family of matrices, namely stiffness matrices and such matrices

3.3. The Method in Detail 81

with appended constraints. These matrices are square or overdetermined(have more rows than columns) but possibly rank deficient.

Finally, we mention another related group of algorithms that can beused to compute the null space of sparse rectangular matrices such as KC .These methods are based on rank revealing QR factorizations [117, 123,10]. These works compute a QR factorization in a careful way to han-dle rank deficiency. The QR factorization can then be used to accuratelycompute a null space basis for the original matrix. Computing the QRfactorization of KC , however, is very slow. Nevertheless, these methodscan be used to compute the null space of K. Our method uses an LUfactorization rather than a QR, since sparse LU factorization are usuallymuch faster than sparse QR factorizations.

3.3. The Method in Detail

We start with some notation. We represent a finite element modelas collection of elements of the form e = (ke, Ie) where ke is an ne-by-neelement matrix in local variables and Ie is a vector of size ne containing theglobal indices corresponding to the local variables. The matrix Ke is the n-by-n element matrix in global coordinates. In other words, Ke(Ie,Ie) = ke.The linear constraints of the model, if there are any, are given in a c-by-nblock of rows denoted by C. This is all the information that we need forout method.

Let K =∑

eKe , the steps for computing the null space of KC =[KT CT

]Tare:

(1) Compute the rigidity graph, a combinatorial graph in which ver-tices represent elements and edges represent rigidity relationships.

(2) Use the rigidity graph to extend every element e = (ke, Ie) into an

extended element e = (ke, Ie).(3) Assemble the elements e into an (n+`)-by-(n+`) matrix F(K).

(4) Pad C into C =[C 0

]and append to F(K), to create

K =

[F(K)

C

].

(5) Compute the LU factorization of K with partial pivoting.

(6) Compute the null space of K using subspace inverse iteration, us-ing its LU factors.

(7) Restrict the null space of K to its first n coordinates and orthog-onalize.

The theoretical idea behind this is that x ∈ null(K) if and only if

QTx ∈ null(

[KC

]) ,

where Q is the (n + `)-by-n identity matrix (an identity extended with `zero rows).


The rest of this section describe these steps in detail and explains thetheoretical justification for the method.

3.3.1. Rigidity Graphs. We define an undirected weighted combina-torial graph which we call the rigidity graph (see Definition 2.6.1). Thevertices in this graph correspond to the elements and the edges connectcertain pairs of elements. Informally, this graph is the dual of the finite-element mesh. Figure 3.3.1(c) illustrates the rigidity graph of the model inFigure 3.3.1(b).

More formally, the edges of the graph represent a local relation betweenpairs of elements. We call this relation mutual rigidity relationship. Therigidity relationship can be defined and computed in a purely algebraicway (but if the element types are known, not just the (ke, Ie)’s, then mutualrigidity can be determined by the element type and adjacency in the mesh).We next show how to algebraically determine whether two given elementmatrices are mutually rigid.

We denote by è the dimension of the null space of ke, and we let Ne

be an ne-by-è matrix whose columns are a basis for the null space of ke.In structural analysis in 2 and 3 dimensions, è is 3 and 6 respectively; inelectrostatic problems, è is 1 in any dimension.

Let e and f be two elements. Let Ce,f be the common variables thatappear both in Ie and If . We denote by Ce→f and Cf→e the |Ce,f |-vectorsthat contain the indices of the common variables in Ie and If respectively.In other words, Ie(Ce→f ) = If (Cf→e) = Ce,f . We also define Ne→f andNf→e as the rows subset matrices Ne(Ce→f , :) and Nf (Cf→e, :) respectively(the colon selects all the columns, as in Matlab).

The elements e and f are mutually rigid if the following conditions hold:

(1) The element matrices ke and kf have the same null space dimension(i.e., è = `f ).

(2) The number of the shared variables |Ce,f | ≥ è.(3) The columns of Ne→f and Nf→e are linearly independent.(4) The columns of Ne→f span the columns of Nf→e and vice versa.

If e and f are mutually rigid, we add an edge (e, f) to the rigidity graph.We assign a weight to the edge: the number of shared variables |Ce,f |. (Inother applications of the rigidity graph other weights may be appropriate.)

In many cases, we know an a-priori grouping of the variables into gridpoints. This knowledge can accelerate the generation of the rigidity graph.If it is not known (or not passed to the software layer that performs thisoperation) we treat each variable as a different grid point. In any case,the acceleration is only by a bounded constant factor, namely the numberof variables per grid point. For every element e, let Pe be the list of itsgrid points. In case there are d variables per each grid point the length ofPe equals the lengths of Ie divided by d. We assume that variables thatcorrespond to a single grid point are consecutive in Ie (and in the globalvariable mapping). This means that the variables that correspond to point


(a) (b)

(c) (d)

(e) (f)

Figure 3.3.1. The construction of the fretsaw extension forsimply supported planar elastic triangles. (a) The originalmodel problem. (b) Its finite element discretization. Thesmall rectangles represent boundary conditions in the formof simply supported grid points. (c) The rigidity graph. (d)The spanning forest (tree in this case). (e) The addition ofslack variables. New slack variables are marked with blackcircles. The elements where slightly shrunk to illustrate theaddition of slack variables. In practice the element matricesare the same. (f) The resulting finite element model.

p are numbered (p− 1)d + 1, (p− 1)d + 2, . . . , pd. Under this assumption,the Pe’s can be easily computed from the Ie’s. In the case each variablerepresents a different grid point, each Pe is simply Ie.


We determine the set of mutually-rigid element pairs as follows. Webegin by computing the eigen-decomposition of each ke, which gives usthe è’s and the Ne’s. The next challenge is to find all the element pairsthat satisfy conditions 1 and 2 above; that is, the pairs that share boththe dimension of the null space and enough unknowns. For each such pairwe also need to compute Cf→e and Ce→f to be able to verify conditions 3and 4.

This can be done in the following way. For every grid point p, let Epbe the set of elements that are incident on p. We compute the ordered listof pairs Lp, which contains the set of elements ei that are incident on p(which is exactly Ep) and for each ei it also contains the index in Pei

of thepoint p. The list Lp is sorted according to the element numbers. It can beformally defined as follows,

Lp = 〈(ei, ti)|ei ∈ Ep and Pei(ti) = p〉 ,

where ei+1 > ei for every i. The construction of the unsorted Lp’s canbe done simultaneously in a single pass over the Pe’s. Then, each Lp canbe sorted according to the element numbers. Since the size of each Lp isbounded, the entire process requires linear time.

Now, we are ready to generate the list of element pairs (e, f) with non-empty Ce,f , along with the corresponding Ce→f and Cf→e. We go overall the elements in ascending order. For the current element e we do thefollowing:

(1) We construct an array R that contains an entry for every element.Each entry Rf contains Ce,f , Ce→f and Cf→e. They are all initial-ized to empty lists.

(2) We construct an empty list Y that stores the indices of the non-empty entries in R.

(3) We go over every grid point p ∈ Pe. For each grid point p we dothe following:(a) We remove the first pair from the list Lp. Since the Lp’s are

ordered, the removed pair contains the element e.(b) For every pair (f, t) ∈ Lp, we add p into Ce,f , t into Cf→e,

and the index of p in Pe into Ce→f . If Rf is empty, we add finto Y .

(4) We go over all f ∈ Y(a) If d|Ce,f | < è, or è 6= `f then (e, f) are not mutually rigid

and we skip to the next element in Y .(b) Otherwise, we need to test conditions 3 and 4.

(i) We have Ce→f and Cf→e, therefore we can form Ne→fand Nf→e.

(ii) We compute the reduced singular values decompositions(svds) of Ne→f and Nf→e: Ne→f = Ue→fΣe→fV

Te→f and

Nf→e = Uf→eΣf→eVTf→e.


(iii) We test whether Ne→f is full rank by checking whetherthe ratio of its smallest to largest singular values is lessthan machine ε (≈ 10−16 in double precision), and sim-ilarly for Nf→e.

(iv) We test condition 4 by computing ‖Uf→eUTf→eNe→f −

Ne→f‖F and ‖Ue→fUTe→fNf→e − Nf→e‖F . If both are

small enough, then condition 4 holds and we output thepair (e, f).

(5) We zero all the nonempty entries of R by traversing over Y . Then,we discard Y .

The correction of step 4.(b).iv is due to the fact that a vector b is in therange of a matrix A if and only if AA+b = b ([19, Proposition 6.1.7]) , thatUf→eU

Tf→e = Nf→eN

+f→e , and that Ue→fU

Te→f = Ne→fN

+e→f .

The number of operations needed to complete steps 1–3 is proportionalto the total number of grid points that where added to all the Ce,f ’s duringthe process. Since the number of elements that are incident on a single gridpoint is bounded by a constant, the total operations number is proportionalto the input size. The computation in step 4 also scales linearly with themodel size, assuming the element matrix size is bounded by a constant.

In practice we first find the common è for the given model, then weonly process the elements that share this specific è. This means that everyelement f with `f 6= è will be a singleton in the rigidity graph.

In some models there are element pairs (e, f) such that Ie ⊆ If . Wedetect such pairs during the computation of the rigidity graph and sum eachpair into a single element by summing the two original element matrices.This process can be performed efficiently using a single pass on the data.We construct an additional directed graph whose nodes are the containingand contained elements. We call this graph the containment graph. Thereis a directed edge (e, f) from e to f in this graph if and only if Ie ⊆ If .We insert all the nodes in the graph into a priority queue based on theirincoming degree. At each step we remove from the queue the elemente with the minimal incoming degree in the containment graph (which isalways zero). Let k be its outgoing degree, and let f1, f2, . . . , fk be theelements that are connected to e. We add the matrix 1

kAe into each of

the Afi’s, discard the element e from the containment graph, update the

incoming degree of the Af ’s in the queue, and also remove e from therigidity graph. If the outgoing degree k is 0 we do nothing and continue.We continue this process until the queue is empty. Clearly, the sum of thenew set of element matrices is equal to the sum of the original elementmatrices. Lemma 2.5.13 shows that under reasonable assumptions, if Agwas mutually rigid with Ae, and Ie ⊆ If , then it is also mutually rigid withAf and with 1

kAe + Af . Therefore, the resulting rigidity graph after this

last process is a valid rigidity graph.


(a) (b)

Figure 3.3.2. Detaching a rigidity edge by adding slackvariables. The element matrices remain unchanged (in localcoordinates), but in the local-to-global index mapping of thedark element, new slack variables X11 and X12 replaced theoriginal variables X5 and X6.

We close this section with a short discussion about mutual rigidity re-lationship in the context of structural mechanics. The relationship is com-puted algebraically, but a rule of thumb that works for common elementtypes is that two elements that share grid points are mutually rigid if andonly no mechanism exists when examining a body consisting of only thesetwo elements. For example, two struts connected at one joint are never mu-tually rigid (Section 2.7.2); Two elastic triangles in the plane constructedby three struts each are mutually rigid if they share a side (share two gridpoints) (Section 2.7.3); elastic tetrahedrons in space built out of six strutseach are mutually rigid if they share a face (share three grid points); solidtetrahedral elements that are connected in a single grid point are mutuallyrigid if they share the three displacement variables and three rotation vari-ables in that grid point, but are not mutually rigid if they share only thedisplacement variables.

3.3.2. Fretsaw Extensions and Null-Space Preservation. In thesecond phase of the algorithm, we compute a maximum weight spanningforest of the rigidity graph and form a finite-element model of a structurewhose rigidity graph is this forest. We use the given element matricesbut drop some of the inter-element continuity constraints. Figure 3.3.1(d)illustrates the spanning forest (tree in this case) of the rigidity graph inFigure 3.3.1(c).

The basic step in the computation of the new model is removing a rigid-ity edge between two adjacent elements. In order to detach such elementswe add new slack variables to the model. Figure 3.3.2 illustrates this step.We now show how to efficiently allocate those new slack variables to theelements so that the modified model will be the exact forest we computed.

Let F be the maximum weight spanning forest of the rigidity graph.We first compute the connected components of F . We store for each ele-ment e the index Ce of its connected component. We also randomly selecta representative element for each connected component. These elementswill not be altered during the entire process and are needed to ensure thesatisfaction of the conditions of Definition 2.8.8. For every element e we


denote by Fe = e1, e2, . . . , eme the set of me elements that are adjacentto e in F . We define r to be the highest variable index used so far in theprocess. We initialize it to n (the highest variable index in K).

Next, we perform the following operation for every grid point p. (Asbefore, if we do not have an a-priori grouping of the variables into gridpoints we treat each variable as a different grid point. The slow down isonly by a bounded constant factor)

(1) As in the previous section, let Ep be the set of elements that areincident on p. We partition Ep according to the connected com-ponents of F (stored for every e ∈ Ep in Ce).

(2) For each part Ep,i of Ep we perform the following steps:(a) We construct the induced subgraph Gp,i of F with the vertex

set Ep,i. The edges of Gp,i are exactly

(e, f) | e ∈ Ep,i, f ∈ Fe .

(b) We compute the connected components ofGp,i. Let V(1)p,i , . . . , V

(np,i)p,i

be the vertex sets of the connected components.(c) If np,i > 1 then we need to detach the elements in different

V(j)p,i s. We do that by adding new slack variables and exchang-

ing the variables corresponding to p in all of the V(j)p,i s except

for one.We first select the set V

(k)p,i of elements that are not changed.

If Ep,i contains a representative element of a global connected

component, then we select the k such that V(k)p,i contains this

representative. Otherwise, we select k arbitrarily.Let Dp be the number of variables that correspond to p (Dp

can be 1 if no knowledge of variable grouping to grid pointsexist). For every j 6= k, we do the following:

(i) We go over all e ∈ V(j)p,i , we switch the Dp variables

corresponding to p in Ie with the variables (r + 1), (r +2), . . . , (r +Dp).

(ii) We let r ← r +Dp.

We denote the resulting Ie by Ie. See Figure 3.3.3 for an illustrated examplefor this process. Steps 1, 2(a) and 2(b) can be fused into a single pass onthe data.

Let ` be the number of new variables that were introduced in the pro-cess above. We now assemble the resulting finite-element model into the(n + `)-by-(n + `) matrix F(A). This matrix is called a fretsaw-forestextension (fretsaw extension is generally defined in Definition 2.8.8). Fig-ure 3.3.1(f) reveals the reasoning behind this name.

The next theorem shows that under a fairly natural condition, the nullspace of the fretsaw extension is a subset of the null space of the originalstiffness matrix. The condition that guarantees this is called element com-patibility. Let S ⊆ Rn be a linear space. An element matrix ke is called


(a) (b)

Figure 3.3.3. Slack variable allocation mechanism. Parts(a) and (b) illustrate a model before and after the addition ofslack variables respectively. The rigidity graph is illustratedin yellow and its spanning forest is illustrated in red. Therigidity forest has two connected components illustrated inpink and light yellow. The rigidity forest around grid pointsA,B and C is not connected. Grid point A represents thebasic case. There is only one connected component of theforest adjacent to A, and it is not connected. Therefore,slack variables are added. We do not add slack variables forpoint B since the induced rigidity forest for every connectedcomponent of the rigidity forest is connected. We add one setof slack variables for point C since the rigidity forest is con-nected in the pink connected component and not connectedin the yellow connected component.

S-compatible if every vector ve in null(ke) has a unique extension into avector v in S (by extension we mean that v(Ie) = ve, and if the restrictionv(Ie) of every vector v in S is always in null(ke). In linear elasticity, ele-ment matrices are compatible with the space of rotation and translations.In electrostatic problems, element matrices are compatible with the spaceof constant vectors.

Theorem 3.3.1. Let K =∑

eKe, where Ke is a collection of sym-metric positive semidefinite n-by-n matrices that are compatible with someS ⊆ Rn, let Q be the (n+`)-by-n identity matrix, and let F(K) be a fretsaw-forest extension of K, as constructed by the algorithm above. Then,

(1) There exist an (n + `)-by-n matrix P such that P TF(K)P = Kand QTP = I.

(2) null(F(K)) ⊆ null(QKQT ).

The first part of the theorem is exactly Lemma 2.8.9. The second partof the theorem is a slight extension of Theorem 2.8.11, which can be foundas corollary 2.8.14 in this thesis.

The sum K =∑

eKe of n-by-n spsd finite-element matrices is usuallysingular, so Kd = f has an infinite number of solutions or none at all. Inorder to be able to form a numerical problem with a unique solution, linearequality constraints are usually added to K. Let C be a c-by-n matrix. Apossible way to get a nonsingular system is by symmetrically adding the


block C to K. The following lemma shows that this indeed restricts thenull space.

Lemma 3.3.2. Let K be an n-by-n spsd matrix, and let C be a c-by-nmatrix. Then,

null(

[K CT

C 0

]) =

[uv

]| u ∈ null(K) ∩ null(C) and v ∈ null(CT )

.

Proof. Let K be the symmetric matrix[K CT

C 0

].

We first let u =[uT vT

]T, where u ∈ null(K)∩ null(C) and v ∈ null(CT ).

Therefore,

Ku =

[K CT

C 0

] [uv

]=

[Ku+ CTv

Cu

]=

[00

].

This shows that u ∈ null(K) and the containment in one direction.We now show the containment in the other direction. Let u = u =[

uT vT]T ∈ null

(K)

. Expanding the equation Ku = 0, we get

Ku+ CTv = 0

Cu = 0 .

Therefore, u ∈ null(C). Moreover, multiply the first equation by uT fromthe left and get

uTKu+ uTCTv = 0 .

Since uTCT = (Cu)T = 0, we get uTKu = 0. Since K is spsd thenKu = 0, so u ∈ null(K). Therefore, u ∈ null(K) ∩ null(C). Moreover,CTv = Ku+CTv = 0, so v ∈ null(CT ). This shows the containment in theother direction and concludes the proof of the lemma.

Another possibility for getting a non singular system is adding a c-by-n

block of new rows C to K. Clearly, null([KT CT

]T) = null(K)∩ null(C).

The following lemma shows that the fretsaw-forest extension also preservethe null space under such equality constraints.

Lemma 3.3.3. Let K =∑

eKe, where Ke is a collection of symmetricpositive semidefinite n-by-n matrices, let Q be the (n + `)-by-n identitymatrix, let F(K) be a fretsaw-forest extension of K, and let C be a c-by-nmatrix. Then,


null(

[F(K)CQT

]) ⊆ null(

[KC

]QT ) .

Proof. Let

x ∈ null(

[F(K)CQT

]) .

By definition, F(K)x = 0 and CQTx = 0. By Theorem 3.3.1, since x ∈null(F(K)), then x ∈ null(QKQT ). Therefore, QKQTx = 0 and KQTx =QTQKQTx = 0. Therefore,[

KC

]QTx =

[KQTxCQTx

]= 0 ,

which concludes the proof of the lemma.

The containment in the null-space and the special padding structuresenable us to get a stronger characteristic which is summarized in the nextlemma.

Lemma 3.3.4. For the above K, F(K), Q and C, let N be an (n+`)-by-

s matrix whose columns form a basis for null(

[F(K)CQT

]), then span(QT N) =

null(

[KC

]).

Proof. Let r ∈ Rs. We first show that QT Nr ∈ null([KT CT

]T) . By

definition, Nr ∈ null(

[F(K)CQT

]). By Lemma 3.3.3, Nr ∈ null(

[KT CT

]TQT ).

Therefore,[KT CT

]TQT Nr = 0 so QT Nr ∈ null(

[KT CT

]T). This

shows that null([KT CT

]T) ⊇

QT Nr | r ∈ Rs

.

We now show the containment in the other direction. Let x ∈ null([KT CT

]T).

By Theorem 3.3.1, there exists an (n + `)-by-n matrix P such that K =P TF(K)P and QTP = In. By the definition of x,[

KC

]x = 0[

P TF(K)PCQTP

]x = 0[

P TF(K)CQT

]Px = 0 .


Since F(K) is spsd, there exist a matrix U , such that F(K) = UTU . Thisimplies that Px ∈ null(F(K)), since

P TF(K)Px = 0

xTP TF(K)Px = 0

xTP TUTUPx = 0

‖UPx‖2 = 0

UPx = 0

UTUPx = 0

F(K)Px = 0 .

Therefore, [F(K)CQT

]Px = 0 .

By the definition of N , there exists a vector r such that Px = Nr. Mul-tiplying the last equality by QT , we get x = QTPx = QT Nr. This showsthe other containment direction and concludes the proof of the lemma.

3.3.3. Rectangular Inverse Iteration. Once we construct the aug-mented fretsaw-forest matrix K, we compute a basis N for its null spaceusing rectangular inverse iteration [68]; see [89] for additional backgroundon inverse iteration. This process requires the sparse LU factorization withpartial pivoting of K. Once this factorization is computed, the algorithmusually performs a few iterations involving solving linear systems of equa-tions of the form UX = B where X and B have only a few columns;between every two iterations, the algorithm orthogonalizes X. In somerare cases the algorithm becomes somewhat more complex, but not moreexpensive. As long as the dimension of the computed null space is small,the orthogonalization is not a dominant cost.

In practice, there are cases where there are zero columns and rows in Kand therefore also in F(K) and K. In these cases, we first remove the zero

rows and columns from K. This has no algebraic or theoretical significance,but was needed for our specific implementation of the rectangular inverseiteration.

Another issue that arises in practice is the treatment of single pointconstraints. Single point constraints correspond to rows in C (and in K)with only a single nonzero each. In exact arithmetic, the correspondingvariables in the null space are always zero. In practice, the inverse iterationprocess assigns small values to these variables that depend on the scaling ofthe single nonzero rows in C. In order to handle this numerical behavior,and to avoid the question of proper scaling of the single nonzero rows in C,we chose to force a zero in the corresponding variables by removing thosecolumns from K.


The inverse iteration result N is an (n + `)-by-s matrix. Our method

truncates the last ` rows of N and then orthonormalize it. The result is ann-by-s matrix N . (Note that s can be smaller than s). Lemma 3.3.4 guar-

antees that the computed null space is exactly the null space of[KT CT

]T.

It is intuitive that the sparsified structure is weaker than the originalstructure. This can also be quantified theoretically in a purely algebraicway: By Lemma 2.8.17 and using [24, Proposition 6.1] the generalizedeigenvalues λ(F(K), QKQT ) are bounded by 1. This behavior affects theinverse iteration process. For a given threshold, the inverse iterations out-puts vectors which are linear combinations of null vectors of K and of vec-tors that correspond to small singular values. In other words, the generatedN spans a linear space that contains the null space of K and some other

vectors. Fortunately, we can still recover the null space of[KT CT

]T. The

following lemma shows exactly how to do this.

Lemma 3.3.5. For the above K, F(K), Q and C, let N be an (n+`)-by-s

matrix whose columns span a linear space that contains null(

[F(K)CQT

]). Let

V be a matrix, such that span(V ) = null(

[KC

]QT N), then span(QT NV ) =

null(

[KC

]).

Proof. Let U be an s-by-smatrix such that span(NU) = null(

[F(K)CQT

]).

By Lemma 3.3.4, span(QT NU) = null([KT CT

]T).

Let x ∈ null([KT CT

]T). By definition, there exists a vector r such

that x = QT NUr. By the definition of x,[KC

]x = 0[

KC

]QT NUr = 0 .

Since Ur ∈ null([KT CT

]TQT N), there exist a vector t such that Ur =

V t. Therefore, x = QT NV t. This shows that null([KT CT

]T) is contained

in span(QT NV ).In order to complete the proof, we need to show that every vector in

span(QT NV ) is also in null([KT CT

]T). Let x = QT NV t for an arbitrary

t. By the definition of V ,[KC

]x =

[KC

]QT NV t = 0t = 0 .

This shows that x ∈ null([KT CT

]T) and concludes the proof of the

lemma.

3.4. Numerical Results 93

The Lemma implies the following final step in the algorithm: compute

an orthonormal basis for null([KT CT

]TQT N). This is usually inexpen-

sive since N only has a few columns. If we denote by V the matrix whose

columns spans this null space, then the null space of[KT CT

]Tis exactly

QT NV .In general, triangular factorizations (Cholesky, LDLT , LU , or QR) of

large finite-elements problems are expensive. But our method is efficient be-cause triangular factorizations of forest-based fretsaw extensions are sparseand cheap to compute. In particular, if C consists of single-unknown con-straints and the rigidity graph of K is connected, then K can be factoredusing sparse symmetric factorizations with no fill outside the original ele-ment matrices (Lemma 2.8.16).

The analysis of the cost of the factorization in Chapter 2 cannot bedirectly applied, however, the LU factorization of K is usually cheap tocompute and produce factors with small amount of fill. There are threegaps between that analysis and the computation that we use here. First, theanalysis in Chapter 2 assumes that the rigidity graph is connected. But inmany real-world models, even models with many solid elements, the rigiditygraph is disconnected. As the number of connected components in therigidity graph increases, so does the fill in the LU factors that our methodcomputes. Second, the analysis in Chapter 2 assumes that C consists ofsingle-unknown equality constraints; if this is not the case, the structure ofC introduces additional fill that is hard to characterize theoretically andmight be substantial. Third, the analysis in Chapter 2 bounded fill andwork in symmetric factorization (Cholesky or LDLT ), whereas rectangularinverse iteration requires an LU factorization. Existing theory on fill insparse LU factorization [21, 28, 52, 61, 85] cannot be directly applied toour forest or tree-based fretsaw extensions. Nevertheless, the existing bodyof theory suggests that the LU factorization of tree-based fretsaw extensionshould be cheap relative to the original finite element model.

Our experimental results, which are reported below, indicate that inspite of the lack of complete theoretical analysis, the LU factorization ofK is indeed inexpensive.

3.4. Numerical Results

We have implemented a prototype of the fretsaw extension method.Our implementation runs under Matlab version 7.4 [112], although allits computation-intensive parts are implemented in C (and are invoked

by Matlab through its cmex interface). The code factors K using Mat-lab’s lu function with pivoting threshold 1.0 (partial pivoting), which usesUMFPACK version 5.0 [46]. However, we use a C function to solve sparsetriangular linear systems; it is much faster than matlab’s \ operator. Run-ning times were measured on a 64-bit Linux 2.6 system with a 1.6 GHz


Figure 3.4.1. Model problem for the scaling experiments.

AMD Opteron 242 processor and 8 GB of RAM. The reported runningtimes are wall-clock times.

The numerical experiments whose results are reported in this section all

compute the null space of KC =[KT CT

]T. The inverse iteration code is

the same as in [68, Section 5], with threshold 10−8 and the initial estimateof the dimension of the null space is set to the common `e of the model.2

We refer to our method in this section as the fretsaw-forest method. Itsoutput is the n-by-s matrix N . We compare the fretsaw-forest method witha naive method to compute the null space of KC , which performs inverseiteration on the triangular factors of KC itself. We refer to this algorithmas the direct method below. The result of this experiment is also an n-by-smatrix N whose columns are an orthonormal basis of the null space of KC .

We compare the accuracy of the methods using two metrics. The firstis discrete: the dimension s of the discovered null-space. The second is con-tinuous and contains more information: the relative error ‖KCN‖2/‖KC‖2.We compute the numerator ‖KCN‖2 but approximate the denominator‖KC‖2 by the maximal absolute value of elements in KC .

3.4.1. Performance Scaling. In the first set of experiments we com-pute the null space of a uniform symmetric elastic k-by-k-by-k cubes, withno constraints (namely, K = KC). The cubes are discretized using a nonuniform mesh of tetrahedra. Each tetrahedron is built out of 6 struts. Eachvertex of a tetrahedron is a joint in this struts’ structure. Figure 3.4.1 showssuch a cube. We used Tetgen version 1.4 [139] to generate the mesh. Thetetrahedra have a minimum radius-edge ratio of 2 and a volume boundedby 10. Each tetrahedra is modeled by a single 12-by-12 element matrix.Table 1 summarizes the models’ characteristics.

Figure 3.4.2(a) presents the running times of the fretsaw-forest methodand the direct method as a function of the problem size. It shows that the

2This code (but using Matlab’s triangular solver) is publicly available athttp://www.tau.ac.il/~stoledo/research.html.


Cube side size n |e| nnz(K)103

11 264 245 528 2,646 4,048 9036 5,739 9,458 21053 15,576 27,853 61869 34,179 63,763 141482 54,069 103,052 228691 75,507 145,423 3228

Table 1. Model statistics for the scaling experiments. Theparameter n stands for the number of unknowns, |e| is thenumber of elements, and nnz(K) is the number of nonzerosin the stiffness matrix.

0 1 2 3 4

x 106

0

100

200

300

400

500

600

NNZ(K)

Tim

e (s

ec)

Total Running Times

Fretsaw−ForestDirect

0 1 2 3 4

x 106

0

0.5

1

1.5

2x 10

8

NNZ(K)

NN

Z(L

)+N

NZ

(U)

Factor Fill


(a) (b)

0.5 1 1.5 2 2.5 3

x 106

0

50

100

150

200

NNZ(K)

Tim

e (s

ec)

Fretsaw−Forest Time Breakdown

Fretsaw ConstructionFactrozationInverse Iteration

0.5 1 1.5 2 2.5 3

x 106

0

100

200

300

400

500

600

NNZ(K)

Tim

e (s

ec)

Direct Time Breakdown

FactrozationInverse Iteration

(c) (d)

Figure 3.4.2. Performance of scaling experiments set. Part(a) shows the total running time of the null space computa-tion. Part (b) shows the total number of nonzeros in the LUfactors. Parts (c) and (d) show the breakdown of the runningtimes into the different phases of the algorithm.

proposed method is highly effective for problems of modest to large sizes.Figure 3.4.2(c) demonstrates the linear scaling behaviour of this method;The running time of the entire process is linear in the problem size and so


are the independent running times of the fretsaw-forest construction, fac-torization and inverse iterations operations. The scaling behaviour for thedirect method is clearly super-linear, as demonstrated by Figure 3.4.2(d).

The reason for the linear factorization and inverse-iteration times in thefretsaw-forest case is presented in Figure 3.4.2(b). This figure presents thenumber of nonzeros in the LU factors in both the fretsaw-forest and the di-rect method. The nonzeros number in the LU factors of the fretsaw-forestsparsification appears to be linear in the problem size. This demonstratesthat the fretsaw sparsification is effective with respect to the number ofnonzeros in LU factorizations, not only with respect to nonzeros in sym-metric factorizations. A closer inspection of the data indicates that theLU factors do fill somewhat. This is not surprising given that the orderingthat UMFPACK computes is different from the ordering that Lemma 2.8.16assumes, and given that we use partial pivoting.

The null space dimension in all the experiments was correctly computedto be 6. Figure 3.4.3 shows the relative errors in the scaling experiment.The relative error of the direct method was always 10−10 or less. The rela-tive error in our method is worse: the method looses between 3 to 5 digitsof accuracy relative to the direct method, but in all cases it is still 10−6

or less. This loss of accuracy is caused primarily by the fretsaw approxi-mation itself, not because we compute the null vectors of K inaccurately:the relative error in the null vectors of K is almost always under 10−10.The approximate values of ‖K‖2 and ‖KC‖2 were very close. Because ofthe missing continuity relations in the fretsaw approximation, F(K) hassmaller nonzeros eigenvalues than K. The small eigenvectors of F(K) mixwith the true null vectors of the structure, which F(K) and K share, andcause the computed eigenvectors to be less accurate. In other words, theloss of accuracy appears to be due to the ill conditioning of F(K), whichshrinks the spectral gap between numerically zero and numerically nonzeroeigenvalues.

3.4.2. Industrial Structural Analysis Models Experiments. Inthe second set of experiments we used industrial real-world structural me-chanics models. The models were produced by two different versions ofNastran, MSC.Nastran and NX.Nastran. All the models contain multi-ple element types and most of them contain single-point and multi-pointconstraints. Table 2 summarizes the main properties of the models.

We first computed the null space of these models without the con-straints. In such experiments the null-space is not empty but contains atleast the rigid-body motions of the body. In these experiments the rigiditygraph is often not connected, so the fretsaw-forest methods indeed generatea forest and not a tree. The number of connected components is shown inTable 2.

Figure 3.4.4 shows the performance of the fretsaw-forest method andthe direct method in this case. We first focus on the factor-fill graph (part


0 1 2 3 4

x 106

10−14

10−12

10−10

10−8

10−6

NNZ(K)E

rror

nor

m

Relative Error

Fretsaw−ForestDirectFretsaw−Only

Figure 3.4.3. Relative errors in the scaling experi-ment. The red (circles) and green (triangles) lines present‖KCN‖2/maxi,j |KC(i, j)|. The blue (Xs) line present

‖KN‖2/maxi,j |K(i, j)|.

Name Description n |e| nnz(K)103 nrows(C) #SPC #CC

cyl Cylinder 648 150 41 0 108 1lg0d Half Cylinder bracket 51,264 8,665 2,149 1248 512 2lg0m Transmission part 99,927 18,681 7,322 540 466 3lg0q Cube 31,125 28,260 786 0 0 1lg0r Brake 90,738 21,024 5,341 1,338 144 338md0c Exhaust Manifold 52,938 9,231 2,769 4212 0 142vl0d Crank Shaft 149,775 45,200 10,715 96 337 50vl0g T-Joint 136,242 27,081 10,314 0 1909 2vl0r Brake 126,399 27,365 8,598 4851 132 82plan10g Unknown 660 100 28 0 72 1sm0d Unknown 12,444 2,276 572 0 0 2nxn plm Aircraft fuselage section 39,432 9,162 1,991 11,640 386 16

Table 2. Model statistics for the industrial experiments set.The parameter n stands for the number of unknowns witha nonempty column in K, |e| is the number of elements,nrows(C) is the number of rows in C, #SPC is the num-ber of single point constraints that correspond to nonemptycolumns in K, nnz(K) is the number of nonzeros in the stiff-ness matrix (without the constraints), and the parameter#CC stands for the number of connected components in therigidity graph.

(b) of the figure). As in the scaling experiments, the fill in the LU factorsin the fretsaw-forest method is close to linear, and in the direct-method isclearly not linear; in most cases it is an order of magnitude higher. Theslightly non-linear behaviour of the fill in the fretsaw-forest method maybe due to the fact that the rigidity graph is not connected, which makesthe fretsaw sparsification less effective.


0 2 4 6 8 10

x 106

0

100

200

300

400

NNZ(K)

Tim

e (s

ec)

Total Running Times


0 2 4 6 8 10

x 106

0

0.5

1

1.5

2x 10

8

NNZ(K)

NN

Z(L

)+N

NZ

(U)

Factor Fill


(a) (b)

0 2 4 6 8 10

x 106

0

50

100

150

200

NNZ(K)

Tim

e (s

ec)



0 2 4 6 8 10

x 106

0

100

200

300

400

NNZ(K)

Tim

e (s

ec)



(c) (d)

Figure 3.4.4. Performance of industrial experiments setwithout constraints. Part (a) shows the total running timeof the null space computation. Parts (c) and (d) show thebreakdown of the null space computation times. Part (b)shows the total number of nonzeros in the LU factors.

We now direct our attention to the running times breakdown graphs inFigure 3.4.4(c) and Figure 3.4.4(d). The factorization times in both figuresare consistent with the data in the fill graphs. The inverse iterations time isalso related to the size of the fill, but is not always monotone and smoothas in the scaling experiments. A closer inspection reveals that the nonmonotonicity is caused by differences in the null space dimensions and inthe number of inverse iterations. Models with larger null space dimensionsdemand more time in every iteration and more iterations to converge.

Finally, the overall performance of our method, shown in Figure 3.4.4(c),demonstrates that our new method is faster than the direct method, espe-cially on large models.

Figure 3.4.5 shows the performance of the methods with constraints.The added constraints increase the fill in the LU factors and increase therunning times. See, for example, the bump around 2·106, which correspondsto the matrix ’nxn plm’, which has a large number of constraints. On onematrix, ’vl0d’, the direct method ran out of memory memory whereas thefretsaw-forest method ran to completion. The graphs in this figure do not


0 2 4 6 8 10

x 106

0

100

200

300

400

500

NNZ(K)

Tim

e (s

ec)

Total Running Times


0 2 4 6 8 10

x 106

0

0.5

1

1.5

2

2.5

3x 10

8

NNZ(K)

NN

Z(L

)+N

NZ

(U)

Factor Fill


(a) (b)

0 2 4 6 8 10

x 106

0

50

100

150

200

NNZ(K)

Tim

e (s

ec)



0 2 4 6 8 10

x 106

0

100

200

300

400

500

NNZ(K)

Tim

e (s

ec)



(c) (d)

Figure 3.4.5. Performance of industrial experiments setwith constraints. Part (a) shows the total running time of thenull space computation. Parts (c) and (d) show the break-down of the null space computation times. Part (b) showsthe total number of nonzeros in the LU factors.

include the results for this matrix. Nevertheless, the qualitative behavior issimilar to the behavior in Figure 3.4.4: even with constraints, the fretsawmethod is faster and requires less memory (especially on larger models).

Table 3 presents the accuracy and the relative errors of the methods,with and without constraints. The table indicates that our method returneda null space with the same dimension as that returned by the direct methodin all but four cases. The relative errors behaved in a similar way to therelative errors in the scaling experiments; in most cases the method looses2–5 digits of accuracy relative to the direct method.

We now explore the four cases where our method and the direct methodcomputed different number of null space basis vectors (marked with doubleasterisks in the table). In all the cases, the relative error that was achievedby both methods was very small. Therefore, the inconsistency is clearlya threshold setting issue. The output of the inverse iteration method ishighly sensitive to the threshold. Further experiments with different inverseiteration threholds produced different numbers of null-space vectors bothin the direct and fretsaw-tree methods, all with small relative errors.


Without Constraints With ConstraintsFretsaw-Forest Direct Fretsaw-Forest Directsize rel-err size rel-err size rel-err size rel-err

plan10g 6 4.4e-10 6 1.4e-12 0 - 0 -cyl 5** 4.2e-7 6 3.6e-14 0 - 0 -

sm0d 6 6.3e-7 6 6.1e-10 No constraints availablelg0q 1 2.1e-11 1 9.5e-13 1 2.1e-11 1 9.5e-13lg0d 12 1.7e-5 12 1.8e-10 0 - 0 -md0c 36 1.2e-7 36 5.9e-10 3 1.4e-10 3 3.9e-14lg0r 8 2.3e-18 8 4.5e-18 5** 2.6e-8** 4 3.7e-8lg0m 6 3.3e-7 6 5.3e-11 0 - 0 -vl0r 4 5.9e-18 4 1.7e-18 0 - 0 -vl0g 6 8.3e-8 6 2.3e-9 0 - 0 -vl0d 10** 4.1e-7** 6 7.6e-10 11 2.3e-1 Out of memory

nxn plm 1 3.5e-22 1 3.5e-22 8** 1.1e-5 0 -Table 3. Computed null space sizes and relative errors inthe industrial experiments set. The relative error (rel-err) is‖KCN‖2/maxi,j |KC(i, j)|. In cases where the computed nullspace size was 0, the relative error is irrelevant and markedwith a hyphen. In cases there was inconsistency between thereturned results of the the methods the result is marked withdouble asterisks.

On ’nxn plm’ we obtained consistent results in the case without con-straints, but the results varied when we changed the inverse-iteration thresh-old from 10−8 to 10−6 and to 10−4. This is another example for the difficultywith the inverse-iterations method. Moreover, when we added constraintswe got more null vectors which is really not what one expects. We exploredthe computation of null(

[KT CT

]QT N). The singular values there are all

very small (and indeed the relative error is small) and singular vectors withvery small residuals where detected as null vectors. It is consistent with theinverse-iteration sensitivity to thresholds that we saw in the experimentswith this matrix with no constraints.

The last experiment that we discuss is ’vl0d’ with constraints. This isthe only experiment with significantly large relative error. Further experi-ments with different thresholds for this matrix, with and without constraintrevealed that the computed null space is very sensitive to the threshold. Inthe case without constraints, the direct method exhibited the same behav-ior.

We note that when the dimension of the computed null space is toolarge, this can be easily identified by calculating the relative error. Apractical system can automatically detect this error and then try to runthe same inverse iteration method with a different threshold.

3.5. Conclusions 101

We also note that the loss of 2–5 digits of accuracy in the computed nullvectors is inherent in the fretsaw-forest approximation, but the reliabilityissues are due to the iterative solver that we use, which is based on inverseiteration. We comment on this further below.

3.5. Conclusions

We presented a novel method for computing the null space of finiteelement models, including models with equality constraints. The methodis purely algebraic (but requires access to the element matrices, not just tothe assembled matrix). It does not require access to the geometry of themodel.

Our definition of rigidity relationship, which lies at the core of the newmethod, might be useful in generalizing the methods of [55, 120, 44, 96,97] by eliminating the need for geometric or domain specific knowledge,but we did not check this in detail.

Our experiments show that the method scales extremely well on 3-dimensional model problems, leading to work and memory requirementsthat are linear in the size of the model. The method performs well andoutperforms an established method on industrial models as well, includingmodels with many equality constraints. The method’s performance is sensi-tive to the structure of the multipoint constraints in the problem. This factis demonstrated in our experiments with the industrial models. This is adrawback to our method in problems where the structure of the multipointconstraints yields a very large decomposition fill.

The computed null vectors are 2–5 decimal digits less accurate thanthose computed by the established method, which is computationally moreexpensive. Both methods are somewhat sensitive to a threshold parameterin the iterative solver that we use, which is based on inverse iteration.

We believe that an iterative solver that is more sophisticated than thesimple inverse iteration solver may be more robust. Such a solver would usethe fretsaw approximation as a preconditioner, but would use true residuals(whereas the inverse iteration solver that we use now uses residuals with

respect to K, not K). We leave this to future research.

CHAPTER 4

Combinatorial Preconditioners for Scalar EllipticFinite-Element Problems

4.1. Introduction

This chapter1 presents novel combinatorial preconditioners for scalar el-liptic finite-element problems based on element-by-element symmetric diag-onally dominant approximations. Symmetric diagonally dominant matricesare relatively easy to precondition. This observation has led two groups ofresearchers to propose linear solvers that are based on element-by-elementapproximation of a given coefficient matrix by a diagonally dominant ma-trix. The diagonally dominant approximation is then used to construct apreconditioner, which is applied to the original problem. Before we de-scribe their proposals and our new contributions, however, we describe theformulation of the problem.

These existing techniques, as well as our new algorithm, use a givenfinite-element discretization of the following of problem: Find u : Ω −→ Rsatisfying

(4.1.1)∇ · (θ(z)∇u) = −f on Ω

u = u0 on Γ1 ,θ(z)∂u/∂n = g on Γ2 .

The domain Ω is a bounded open set of Rd and Γ1 and Γ2 form a partitionof the boundary of Ω. The conductivity θ is a spatially varying d-by-dsymmetric positive definite matrix, f is a scalar forcing function, u0 is aDirichlet boundary condition and g is a Neumann boundary condition.

We assume that the discretization of (4.1.1) leads to an algebraic systemof equations

Kx = b .

The matrix K ∈ Rn×n is called a stiffness matrix, and it is a sum of elementmatrices, K =

∑e∈EKe. Each element matrix Ke corresponds to a subset

of Ω called a finite element. The elements are disjoint except perhaps for

1The results in this chapter will also appear as a paper in the SIAM Journal on Ma-trix Analysis and Applications, co-authored by Haim Avron, Doron Chen, and SivanToledo [9]; several examples and minor proofs will appear in a technical report [11].Most of the research was carried out by myself and another PhD student from Tel-AvivUniversity, Haim Avron, in a collaborative effort. These results will also appear inHaim’s thesis.

102


their boundaries and their union is Ω. We assume that each element matrixKe is symmetric, positive semidefinite, and zero outside a small set of nerows and columns. In most cases ne is uniformly bounded by a small integer(in our experiments ne is 4 or 10). We denote the set of nonzero rows andcolumns of Ke by Ne.

For the problem (4.1.1), all the element matrices are singular with rank

ne−1 and null vector[1 1 · · · 1

]T. This is a key aspect of techniques for

approximating element matrices by diagonally dominant ones: the methodsonly works when element matrices are either nonsingular or are singular

with rank ne − 1 and null vector[1 1 · · · 1

]T.

While the general approach of element-by-element (EBE) precondition-ing was proposed as early as [87], Reitzinger and his collaborators werethe first to propose element-by-element approximation by symmetric anddiagonally dominant (sdd) matrices [77, 128, 102]. They proposed twoelement-approximation techniques. In one technique, the approximationproblem is formulated as an optimization problem, in which one tries tominimize the generalized condition number of a given element matrix Ke

and an sdd matrix Le; a generic optimization algorithm is then used to findLe. The second technique uses symbolic algebra to formulate the approx-imation problem, and uses symbolic-algebra algorithms to find Le. Bothmethods are quite expensive and slow. A row or a column that is zero inKe is also zero in Le, so the Le’s are also very sparse. Once the Le’s arefound, they are assembled. When all the Le’s are sdd, their sum L is alsosdd. The matrix L is then used to construct an algebraic multigrid solver,which is used as a preconditioner for K.

Boman, Hendrickson, and Vavasis proposed a different element-by-elementapproximation technique [25]. They used geometric information about theelements and the values of θ to directly construct Le. The approximationalgorithm is inexpensive. They show that under certain conditions on thecontinuous problem and on the finite-element mesh, the approximationsare all good. They proposed to use L to construct a combinatorial pre-conditioner rather then apply multigrid to L. This proposal was based onthe observation that over the last decade, several provably good combina-torial graph algorithms for constructing preconditioners for sdd matriceshave been developed [145, 69, 70, 22, 141, 142, 53, 143, 108]. Someof them, as well as some combinatorial heuristics, have been shown to beeffective in practice [37, 93, 125, 129, 60, 124].

In this chapter, we extend this paradigm in two ways. First, we pro-pose a novel, effective, and purely algebraic method for approximating anelement matrix Ke by an sdd matrix Le. Our approximation algorithm isrelatively inexpensive and provably good: the spectral distance between Ke

and Le is within a n2e/2 factor of the best possible for an sdd approxima-

tion of Ke, where ne is the number of nonzero rows and columns in Ke. Inparticular, this means that our algorithm produces good approximationswhenever the algorithm of Boman et al. does. Furthermore, there exist


element matrices that are ill conditioned and that are far from diagonaldominance (in the sense that a small perturbation of their entries cannotmake them diagonally dominant), which our algorithm approximates well.

Vavasis has shown [152] that some of the results in this chapter can beused to find an optimal approximation of an element matrix Ke by an sddmatrix Le. Since his method is computationally expensive it is not clear ifit is effective in a practical solver. Our code uses our provably good, cheapto compute, but suboptimal approximation instead.

Our second contribution to this paradigm is a technique to handle prob-lems in which some of the element matrices cannot be well approximatedby sdd matrices. This may arise because of anisotropy in θ, or becauseof ill-shaped elements, for example. Our algorithm splits the elements intotwo sets: the set E(t) in which Le is a good approximation of Ke, and therest (where t is a parameter that determines how good we require approx-imations to be.) We then scale and assemble the good element-by-elementapproximations to form L =

∑e∈E(t) αeLe. Next, we use a combinatorial

graph algorithm to construct an easy-to-factor approximation M of L. Fi-nally, we scale M and add γM to the inapproximable elements, to formγM +

∑e 6∈E(t) Ke. We factor this matrix and use it as a factored precondi-

tioner for K =∑

eKe.The splitting idea is simple, but there is no easy way to exploit it in most

preconditioning paradigms. In Reitzinger’s method, for example, one couldbuild an algebraic multigrid solver for L =

∑e∈E(t) αeLe, but how would

one incorporate the inapproximable elements into this solver? There is noobvious way to do this. An algebraic multigrid solver for γM+

∑e 6∈E(t) Ke is

unlikely to be much better as a preconditioner than an algebraic multigridsolver for K, because γM +

∑e 6∈E(t) Ke is far from diagonal dominance

unless all the elements are approximable.The reason that the splitting idea works well with combinatorial pre-

conditioners is that combinatorial preconditioning algorithms sparsify thesdd matrix L that is given to them as input. The sparsification makes theCholesky factorization of the sparsified M much cheaper and sparser thanthe factorization of L. If most of the elements are approximable, adding∑

e6∈E(t) Ke to γM is likely to yield a preconditioner that is still cheap tofactor.

Once the Cholesky factor of the preconditioners are computed, we useit in a preconditioned symmetric Krylov-subspace solver such as ConjugateGradients [41, 80], symmlq, or minres [119]. For most of the combina-torial algorithms that we can use to construct M , it is possible to showthat the preconditioner is spectrally close to K. The spectral bounds givea bound on the number of iterations that the Krylov-subspace algorithmperforms.

4.2. Nearly-Optimal Element-by-Element Approximations 105

Experimental results that explore the performance and behavior of oursolver show that the solver is highly reliable. In particular, on some prob-lems other solvers, including an algebraic-multigrid solver and an incomplete-Cholesky solver, either fail or are very slow; our solver handles these prob-lems without difficulty.

The rest of the chapter is organized as follows. Section 4.2 presents ourelement-approximation method. The scaling of the element-by-element ap-proximation is presented in Section 4.3. The combinatorial sparsificationphase is described in Section 4.4 and the handling of inapproximable ele-ments in Section 4.5. The costs associated with the different phases of thesolver are described in Section 4.6. Experimental results are presented inSection 6.5. We mention some open problems that this research raises inSection 4.8.

4.2. Nearly-Optimal Element-by-Element Approximations

In this section we show how to compute a nearly optimal sdd approx-imation Le to a given symmetric positive semidefinite matrix Ke that iseither nonsingular or has a null space consisting only of the constant vec-tor.

4.2.1. Defining the Problem. Let S be a linear subspace of Rn (theresults of this section also apply to Cn, but we use Rn in order to keepthe discussion concrete). We denote by RS ⊆ Rn×n the set of symmetricpositive (semi)definite matrices whose null space is exactly S.

Definition 4.2.1. Given two matrices A and B in RS, a finite gen-eralized eigenvalue λ of (A,B) is a scalar satisfying Ax = λBx for somex 6∈ S. The generalized finite spectrum Λ(A,B) is the set of finite gener-alized eigenvalues of (A,B), and the generalized condition number κ(A,B)is

κ(A,B) =max Λ(A,B)

min Λ(A,B).

(This definition can be generalized to the case of different null spaces,but this is irrelevant for this chapter.) We informally refer to κ(A,B) asthe spectral distance between A and B.

We refer to the following optimization problem as the optimal symmetricsemidefinite approximation problem.

Problem 4.2.2. Let A be a symmetric positive (semi)definite matrixwith null space S and let B1, B2, . . . , Bm be rank-1 symmetric positive semi-definite matrices. Find coefficients d1, d2, . . . , dm that minimize the gener-alized condition number of A and

Bopt =m∑j=1

d2jBj


under the constraint null(Bopt) = null(A), or decide that the null spacescannot match under any dj’s. We can assume without loss of generalitythat all the Bj’s have unit norm.

A slightly different representation of the problem is useful for charac-terizing the optimal solution. Let Bj = ZjZ

Tj , where Zj is a column vector.

Let Z be an n-by-m matrix whose columns are the Zj’s. Thenm∑j=1

d2jBj =

m∑j=1

d2jZjZ

Tj = ZDDTZT

where D is the m-by-m diagonal matrix with dj as its jth diagonal element.This yields an equivalent formulation of the optimal symmetric semidefiniteapproximation problem.

Problem 4.2.3. Given a symmetric positive (semi)definite n-by-n ma-trix A with null space S and an n-by-m matrix Z, find an m-by-m diagonalmatrix D such that null(ZDDTZT ) = S and that minimizes the generalizedcondition number κ(A,ZDDTZT ), or report that no such D exists.

We are interested in cases where range(Z) = range(A), where the prob-lem is clearly feasible.

Vavasis has pointed out [152] that minimizing κ(ZTDTDZ,A) is equiv-alent to finding a diagonal nonnegative matrix S that minimizes max Λ(ZTSZ,A)such that min Λ(ZTSZ,A) ≥ 1, and taking D = S1/2. He showed that thisoptimization problem can be solved as a convex semidefinite programmingproblem: find the minimum t and the corresponding S for which the con-straints min Λ(tA, ZTSZ) ≥ 1 and min Λ(ZTSZ,A) ≥ 1 can be satisfiedsimultaneously. Convex semidefinite programming problems can be solvedin polynomial time. An algorithm based on this observation can find anoptimal sdd approximation, but applying it might be costly. We have de-cided to pursue a different direction by finding a suboptimal but provablygood approximation using a fast and simple algorithm.

4.2.2. From Generalized Condition Numbers to Condition Num-bers. The main tool that we use to find nearly optimal solutions to Prob-lem (4.2.2) is a reduction of the problem to the well studied problem ofscaling the columns of a matrix to minimize its condition number.

Definition 4.2.4. Given a matrix A, let σmax be the largest singularvalue of A and σmin be the smallest nonzero singular value of A. The con-dition number of A is κ(A) = σmax/σmin. If A ∈ RS then κ(A) = κ(A,P⊥S),where P⊥S is the orthogonal projector onto the subspace orthogonal to S.

The following lemma, which is a generalization of [24, Theorem 4.5], isthe key to the characterization of Bopt.

Lemma 4.2.5. Let A = UUT and B = V V T , where U and V are realvalued matrices of order n × m. Assume that A and B are symmetric,


positive semidefinite and null(A) = null(B). We have

Λ (A,B) = Σ2(V +U

)and

Λ (A,B) = Σ−2(U+V

).

In these expressions, Σ(·) is the set of nonzero singular values of the matrixwithin the parentheses, Σ` denotes the same singular values to the `th power,and V + denotes the Moore-Penrose pseudoinverse of V .

Proof. Both U and V have n rows, so U+ and V + have n columns.Therefore, the products V +U and U+V exist. Therefore,

Σ2(V +U

)= Λ

(V +UUT

(V +)T)

= Λ((V +)TV +UUT

)= Λ

((V V T

)+UUT

)= Λ

(B+A

).

(Λ(·) denotes the set of eigenvalues of the argument.) For the second linewe use the following observation. If X is n-by-k and Y is k-by-n, then thenonzero eigenvalues of XY and Y X are the same. The second line followsfrom this observation for X = (V +)

Tand Y = V +UUT . The third line

follows from the equality(XXT

)+= (X+)

TX+ for an order n-by-k matrix

X [19, Proposition 6.1.6].It is sufficient to show that Λ (B+A) = Λ(A,B) in order to prove the

first part of the lemma. Let λ ∈ Λ(A,B), then there exists a vector x ⊥null(B) = null(A), such that Ax = λBx. Since B is symmetric, x ∈range(B) = range(BT ). Therefore, B+Ax = λB+Bx = λx. The lastequality follows from the fact that B+B is a projector onto range(BT ) [19,Proposition 6.1.6]. Therefore, Λ (B+A) ⊇ Λ(A,B). Let λ ∈ Λ(B+A), thenthere exists a vector x, such that B+Ax = λx. Therefore, Ax = BB+Ax =λBx. The first equality follows from the fact that range(A) = range(B)and [19, Proposition 6.1.7]. Therefore, Λ (B+A) ⊆ Λ(A,B) which showsthe equality.

The second result Λ (A,B) = Σ−2 (U+V ) follows from replacing theroles of A and B in the analysis above and from the equality Λ (A,B) =Λ−1 (B,A). The reversal yields

Λ (A,B) = Λ−1 (B,A) =(Σ2(U+V

))−1= Σ−2

(U+V

).

This lemma shows that Problems 4.2.2 and 4.2.3 can be reduced to theproblem of scaling the columns of a single matrix to minimize its conditionnumber. Let A = UUT and let Z satisfy range(Z) = range(A). (If A issymmetric positive semidefinite but U is not given, we can compute such


a U from the Cholesky factorization of A or from its eigendecomposition.)According to the lemma,

Λ(A,ZDDTZT ) = Σ−2(U+ZD

).

Therefore, minimizing κ(A,ZDDTZT ) is equivalent to minimizing the con-dition number κ(U+ZD) under the constraint range(ZD) = range(Z).

The other equality in Lemma 4.2.5 does not appear to be useful for sucha reduction. According to the equality

Λ(A,ZDDTZT ) = Σ2((ZD)+ U

),

but unfortunately, there does not appear to be a way to simplify (ZD)+ Uin a way that makes D appear as a row or column scaling. (Note that ingeneral, (ZD)+ 6= D+Z+.)

The problem of scaling the columns of a matrix to minimize its conditionnumber has been investigated extensively. Although efficient algorithms forminimizing the condition number of a rectangular matrix using diagonalscaling do not exist, there is a simple approximation algorithm. It may bepossible to use Vavasis’s algorithm [152] to solve this problem too, but thisis not the concern of this work.

4.2.3. Computing the Pseudoinverse of a Factor of an ElementMatrix. Before we can find a scaling matrix D, we need to compute U+

from a given element matrix Ke = UUT and to form U+Z.We compute U+ in one of two ways. If the input to our solver is an

element matrix Ke with a known null space, we can compute U+ froman eigendecomposition of Ke. Let Ke = QeΛeQ

Te be the reduced eigende-

composition of Ke (that is, Qe is n-by-rank(Ke) and Λe is a rank(Ke)-by-

rank(Ke) nonsingular diagonal matrix). We have Ke = QeΛ1/2e

(QeΛ

1/2e

)Tso we can set U = QeΛ

1/2e , so U+ = Λ

−1/2e QT

e .Many finite-element discretization techniques actually generate the ele-

ment matrices in a factored form. If that is the case, then some symmetricfactor F of Ke = FF T is given to our solver as input. In that case, wecompute a reduced singular-value decomposition svd of F , F = QeΣeR

Te ,

where Σe is square, diagonal, and invertible, and Re is square and unitary,both of order rank(F ). Since

Ke = FF T = QeΣeRTe ReΣ

TeQ

Te = QeΣ

2eQ

Te

is an eigendecomposition of Ke, we can set U = QeΣe and we have Ke =UUT . In this case U+ = Σ−1

e QTe . This method is more accurate when Ke

is ill conditioned.Note that in both cases we do not need to explicitly form U+; both

methods provide a factorization of U+ that we can use to apply it to Z.


Once we form U+Z, our solver searches for a diagonal matrix D thatbrings the condition number of U+ZD close to the minimal condition num-ber possible. This problem is better understood when U+Z is full rank.Fortunately, in our case it always is.

Lemma 4.2.6. Let U be a full rank m-by-n matrix, m ≥ n, and let Z bean m-by-` matrix with range(Z) = range(U). Then U+Z has full row rank.

Proof. Since range(Z) = range(U), there exists an n-by-l matrix Csuch that Z = UC. By definition, rank(Z) ≤ rank(C) ≤ n. Moreover,since range(Z) = range(U) and U is full rank, we have that n = rank(Z).Therefore, rank(C) = n.

It is sufficient to show that C = U+Z to conclude the proof of thelemma. Since U is full rank and m ≥ n, the product U+U is the n-by-nidentity matrix. Therefore, U+Z = U+UC = C.

Without the assumption range(Z) = range(U), the matrix U+Z can berank deficient even if both U+and Z are full rank.

Example 4.2.7. Let

U =

2 0−1 −1−1 1

, Z =

1 11 −11 0

.

The columns of U are orthogonal, range(Z) 6= range(U). This gives

U+ =

[1/3 −1/6 −1/60 −1/2 1/2

]and

U+Z =

[0 1/20 1/2

],

which is clearly not full rank.

4.2.4. Nearly-Optimal Column Scaling. Given a matrix U+Z, wewish to find a diagonal matrix D that minimizes the condition numberof U+ZD, under the assumption range(Z) = range(U), and under theconstraint that range(ZD) = range(Z).

To keep the notation simple and consistent with the literature, in thissection we use A to denote U+Z and we use m and n to denote the numberof rows and columns in A = U+Z.

The key result that we need is due to van der Sluis [147], who showedthat choosing D such that all the columns of AD have unit 2-norm bringsAD to within a factor of

√n of the optimal scaling. Van der Sluis, extending

an earlier result of Bauer for square invertible matrices [16], analyzed thefull-rank case.


Given an m-by-n matrix A, m ≥ n, and a nonsingular diagonal D vander Sluis defined

(4.2.1) κvdS(AD) =‖AD‖2

minx 6=0 ‖ADx‖2/‖x‖2

(his original definition is not specific to the 2-norm, but this is irrelevantfor us). If A is non-singular then κvdS(AD) = κ(AD). He, like Bauer, wasinterested in finding the diagonal matrix D that minimizes (4.2.1). Thisdefinition of the problem implicitly assumes that A is full rank, otherwiseκvdS(AD) = ∞ for any nonsingular diagonal D. Also, if A is full rankthen the minimizing D must give a full-rank AD. If A has more columnsthan rows, we can use a complementary result by van der Sluis, one thatuses row scaling on a matrix with more rows than columns. Van der Sluisresult show that if the rows of DA have unit 2-norm, then κvdS(DA) iswithin a factor of

√m of the minimum possible. This gives us the result

that we need, because κvdS(AD) = κvdS(DTAT ). Shapiro showed that, in

the general case, van der Sluis’ estimate on κvdS(AD) cannot be improvedby more than a

√2 factor [135]. The following are formal statements of

the last two cited results.

Lemma 4.2.8. (Part of Theorem 3.5 in [147]) Let A be an m-by-n full-rank matrix and let D be a diagonal matrix such that in DA all rows haveequal 2-norm. Then for every diagonal matrix D we have

κvdS(DA) ≤√mκvdS(DA) .

Lemma 4.2.9. (Theorem in [135]) For every ε > 0 there exists an n-by-n nonsingular, real valued matrix A such that if:

(1) D is a diagonal matrix such that all the diagonal elements ofDATAD are equal to one, and

(2) D is a diagonal matrix such that κ(AD) is minimized,

then

κ(AD) >(√

n/2− ε)κ(AD) .

As we have shown in Lemma 4.2.6, that matrix A = U+Z whosecolumns we need to scale is full rank, so van der Sluis’s results apply to it.

We note that further progress has been made in this area for squareinvertible A’s, but it appears that this progress is not applicable to ourapplication when A = U+Z is rectangular (which is usually the case).Braatz and Morari showed that for a square invertible A, the minimumof κ(AD) over all positive diagonal matrices D can be found by solv-ing a related optimization problem, which is convex [27]. Their paperstates that this result also applies to the rectangular case [27, Remark 2.5];what they mean by that comment is that the related optimization problemminimizes ‖AD‖2‖D−1A+‖2 [26], whereas we need to minimize κ(AD) =‖AD‖2‖(AD)+‖2.


4.2.5. Nearly-Optimal Symmetric diagonally dominant Approx-imations. We now turn our attention to the matrices that arise as elementmatrices in finite-element discretizations of (4.1.1). Such a matrix Ke hasnull space that is spanned by the constant vector and by unit vectors ejfor every zero column j of Ke. The part of the null space that is spannedby the unit vectors is irrelevant, so we assume that we are dealing with a

matrix A whose null space is spanned by constant vector[1 1 · · · 1

]T.

We wish to approximate a symmetric semidefinite matrix A with thisnull space (or possibly a nonsingular matrix) by a symmetric diagonallydominant matrix B,

Bii ≥n∑j=1j 6=i

|Bij| .

To define a matrix Z such that the expression ZDDTZT can generateany symmetric diagonally dominant matrix, we define the following vectors.

Definition 4.2.10. Let 1 ≤ i, j ≤ n, i 6= j. A length-n positive edgevector, denoted 〈i,−j〉, is the vector

〈i,−j〉k =

+1 k = i−1 k = j

0 otherwise.

A negative edge vector 〈i, j〉 is the vector

〈i, j〉k =

+1 k = i+1 k = j

0 otherwise.

A vertex vector 〈i〉 is the unit vector

〈i〉k =

+1 k = i

0 otherwise.

A symmetric diagonally dominant matrix can always be expressed as asum of outer products of scaled edge and vertex vectors. Therefore, we canconservatively define Z to be the matrix whose columns are all the positiveedge vectors, all the negative edge vectors, and all the vertex vectors.

If A is singular and its null space is the constant vector, we can dobetter. Chen and Toledo provided a combinatorial characterization of thenull space of sdd matrices [38].

Lemma 4.2.11. ([38]) Let A be a symmetric diagonally dominant ma-trix whose null space is the constant vector. Then A is a sum of outerproducts of scaled positive edge vectors. Furthermore, the null space of asymmetric diagonally dominant matrix with a positive off-diagonal element(corresponding to an outer product of a scaled negative edge vector) cannot

be span[1 1 · · · 1

]T.


Therefore, if A is singular with this null space, we only need to includein the column set Z the set of positive edge vectors. If A is nonsingular,we also include in Z negative edge vectors and vertex vectors.

We can also create even sparser Z’s; they will not allow us to expressevery sdd B as B = ZDDTZT , but they will have the same null space asA. To define these sparser Z’s, we need to view the edge vector 〈i,−j〉 asan edge that connects vertex i to vertex j in a graph whose vertices are theintegers 1 through n. The null space of ZZT is the constant vector if an onlyif the columns of Z, viewed as edges of a graph, represent a connected graph.Therefore, we can build an approximation B = ZDDTZT by selecting anarbitrary connected graph on the vertices 1, . . . , n. By [38, Lemma 4.2],if A is nonsingular, we can include in Z the positive edge vectors of aconnected graph plus one arbitrary vertex vector.

If A is well conditioned (apart perhaps from one zero eigenvalue), wecan build a good approximation B = ZDDTZT even without the column-scaling technique of Lemma 4.2.5. In particular, this avoids the compu-tation of the pseudo-inverse of a factor U of A = UUT . Clearly, if A isnonsingular and well conditioned, then we can use I as an approximation:the generalized condition number κ(A, I) is κ(A). If A has rank n− 1 andthe constant vector is its null vector, then

BC(ne) =1

ne

ne − 1 −1 −1 · · · −1−1 ne − 1 −1 · · · −1−1 −1 ne − 1 · · · −1...

......

. . ....

−1 −1 · · · · · · ne − 1

yields

(4.2.2) κ(A,BC(ne)) = σmax(A)/σmin(A) ,

which we assumed is low (σmax(A) is the largest singular value of A, andσmin(A) is the smallest nonzero singular value of A). The matrix BC(ne)

is the Laplacian of the complete graph, and it is clearly sdd. The iden-tity (4.2.2) follows from the fact that BC(ne) is an orthogonal projectiononto range(A). The following lemma summarizes this discussion.

Lemma 4.2.12. Let A be a symmetric positive (semi)definite matrix.

If A is nonsingular, or if the null space of A is span[1 1 · · · 1

]T, then

there is an sdd matrix B such that κ(A,B) ≤ κ(A).

This result may seem trivial (it is), but it is nonetheless important.The global stiffness matrix K =

∑eKe is often ill conditioned, but the

individual element matrices Ke are usually well conditioned. Since we buildthe approximation L =

∑e Le element by element, this lemma is often

applicable: When Ke is well conditioned, we can set Le to be an extensionof BC(ne) into an n-by-n matrix. The rows and columns of the scaled BC(ne)

are mapped to rows and columns Ne of Le and the rest of Le is zero.


For well-conditioned A’s, we can also trade the approximation qualityfor a sparser approximation than BC(ne). The matrix

BS(ne) =1

ne

ne − 1 −1 −1 · · · −1−1 1 0 · · · 0−1 1 · · · 0...

.... . .

...−1 0 · · · 1

gives(4.2.3)

κ(A,BS(ne)) ≤ κ(A,BC(ne))κ(BC(ne), BS(ne)) = κ(A)κ(BS(ne)) =neσmax(A)

σmin(A).

For small ne, this may be a reasonable tradeoff. The bound (4.2.3) followsfrom the observation that the eigenvalues of BS(ne) are exactly 0, 1, and ne.We note that besides generating a sparser matrix this kind of approximationpreserves structural properties such as planarity. This may be importantfor certain sparsification algorithms [18, 100] and subsequent factorizationalgorithms.

When A is ill conditioned, there may or may not be an sdd matrix Bthat approximates it well. The following two examples demonstrate bothcases.

Example 4.2.13. Let

A =1

2ε

1 + ε2 −ε2 −1−ε2 ε2 0−1 0 1

for some small ε > 0. This matrix has rank 2 and null vector

[1 1 1

]T,

and it its condition number is proportional to 1/ε2. Since A is diagonallydominant, there is clearly an sdd matrix B (namely, A itself) such thatκ(A,B) = 1. This matrix is the element matrix for a linear triangularelement with nodes at (0, 0), (0, ε), and (1, 0) and a constant θ = 1. The illconditioning is caused by the high aspect ratio of the triangle, but this illconditioned matrix is still diagonally dominant. Small perturbations of thetriangle will yield element matrices that are not diagonally dominant butare close to a diagonally dominant one. If we discretize the same trianglewith the same material constant using a quadratic element, the resultingelement matrix is still approximable and ill-conditioned, but is far fromsdd; it has positive off-diagonal with magnitude proportional to the normof the element matrix [11] (the quadratic element example is due to Avron).

Example 4.2.14. Our example of a matrix that cannot be approxi-mated well by an sdd matrix is the element matrix for an isosceles trianglewith two tiny angles and one that is almost π, with nodes at (0, 0), (1, 0),


and (1/2, ε) for some small ε > 0. The element matrix is

A =1

2ε

14

+ ε2 14− ε2 −1

2

14− ε2 1

4+ ε2 −1

2

−12

−12

1

This matrix has rank 2 and null vector

[1 1 1

]T. We now show that

for any sdd matrix B with the same null space, κ(A,B) ≥ ε−2/4. Let

v =[1 −1 0

]Tand u =

[1 1 −2

]T; both are orthogonal to

[1 1 1

]T.

We have vTAv = 2ε and uTAu = 4.5ε−1. Therefore,

κ(A,B) = maxx⊥null(A)

xTAx

xTBx× max

x⊥null(A)

xTBx

xTAx

≥ uTAu

uTBu× vTBv

vTAv

=4.5

2ε2× vTBv

uTBu.

We denote the entries of B by

B =

b12 + b13 −b12 −b13

−b12 b12 + b23 −b23

−b13 −b23 b13 + b23

where the bij’s are non-negative. Furthermore, at least two of the bij’s mustbe positive, otherwise B will have rank 1 or 0, not rank 2. In particular,b13 + b23 > 0. This gives

vTBv

uTBu=

4b12 + b13 + b23

9b13 + 9b23

=4b12

9b13 + 9b23

+1

9≥ 1

9.

Therefore, κ(A,B) > ε−2/4, which can be arbitrarily large.

4.2.6. A Heuristic for Symmetric diagonally dominant Ap-proximations. In Section 4.2.5 we have shown how to find a nearly-optimal sdd approximation B to a symmetric positive (semi)definite matrix

A whose null space is spanned by[1 · · · 1

]T. In this section we show a

simple heuristic. We have found experimentally that it often works well.On the other hand, we can show that in some cases it produces approxi-mations that are arbitrarily far from the optimal one. Thus, this sectionhas two related goals: to describe a simple heuristic that often works well,but to point out that it cannot always replace the method of Section 4.2.5.


Definition 4.2.15. Let A be a symmetric positive (semi)definite ma-trix. We define A+ to be the sdd matrix defined by

(A+)ij =

aij i 6= j and aij < 0

0 i 6= j and aij ≥ 0∑k 6=j − (A+)ik i = j .

Clearly, A+ is sdd.

It turns out that in many cases, κ(A,A+) is small, making A+ a goodapproximation for A. The following lemma gives a theoretical bound forκ(A,A+).

Lemma 4.2.16. Let A be an spsd ne-by-ne matrix with null(A) =span[1 . . . 1]T . Then null(A+) = span[1 . . . 1]T , and κ(A,A+) ≤ √neκ(A).Moreover, if there exist a constant c and an index i such that ‖A‖inf ≤ cAii,then κ(A,A+) ≤ cκ(A).

Proof. We first show that null(A) = null(A+). Let A− = A+ − A.The matrix A− is symmetric and contains only nonpositive off diagonals,and

A−[1 . . . 1

]T= A+

[1 . . . 1

]T − A [1 . . . 1]T

= 0 .

Therefore, A− is an sdd matrix. Since sdd matrices are also spsd, for allx,

(4.2.4) 0 ≤ xTAx = xTA+x− xTA−x ≤ xTA+x .

Therefore, null(A+) ⊆ null(A). The equality of these linear spaces follows

from the equation A+

[1 . . . 1

]T= 0.

By equation 4.2.4, for all x /∈ null(A+), xTAx/xTA+x ≤ 1. This showsthat max Λ(A,A+) ≤ 1.

We now bound max Λ(A+, A) from above. For every vector x /∈ null(A),

xTA+x

xTAx=xTA+x/x

Tx

xTAx/xTx≤ max Λ(A+)

min Λ(A).

Therefore, it is is sufficient to show that max Λ(A+) ≤ θmax Λ(A) for somepositive θ in order to show that max Λ(A+, A) ≤ θκ(A) and κ(A,A+) ≤θκ(A).

Since A[1 . . . 1

]T= 0, A is spsd, and assuming ne > 1, for every i,

|Aii| = Aii =∑j 6=i

∣∣∣(A+)ij

∣∣∣−∑j 6=i

∣∣∣(A−)ij

∣∣∣ .Therefore, since A+

[1 . . . 1

]T= 0, for every i,


∑j

|Aij| = |Aii|+∑j 6=i

∣∣∣(A+)ij

∣∣∣+∑j 6=i

∣∣∣(A−)ij

∣∣∣= 2

∑j 6=i

∣∣∣(A+)ij

∣∣∣= (A+)ii +

∑j 6=i

∣∣∣(A+)ij

∣∣∣=

∑j

∣∣∣(A+)ij

∣∣∣ .(4.2.5)

By the definitions of the 1-norm and the inf-norm, and the fact that A+ issymmetric, ‖A+‖1 = ‖A+‖inf . Moreover, by [66, Corollary 2.3.2], ‖A+‖2 ≤√‖A+‖1 ‖A+‖inf . Therefore,

‖A+‖2 ≤ ‖A+‖inf

= maxi

∑j

∣∣∣(A+)ij

∣∣∣= max

i

∑j

|Aij|(4.2.6)

= ‖A‖inf ,

where the second equality is due to equation 4.2.5. Therefore, by [66,Equation 2.3.11]

‖A+‖2 ≤ ‖A‖inf ≤√ne ‖A‖2 .

Since A+ and A are both symmetric, max Λ(A+) = ‖A+‖2 and max Λ(A) =‖A‖2. Therefore, max Λ(A+) ≤ √ne max Λ(A). This shows that κ(A,A+) ≤√neκ(A) and concludes the proof of the first part of the lemma.

We now assume that there exist a constant c and an index i, suchthat ‖A‖inf ≤ cAii. By equation 4.2.6, we have that ‖A+‖2 ≤ cAii.Since for every i, Aii ≤ ‖A‖2, we have that ‖A+‖2 ≤ c ‖A‖2. Therefore,max Λ(A+) ≤ cmax Λ(A). This shows that in this case κ(A,A+) ≤ cκ(A)and concludes the proof of the lemma.

This means that if A is well conditioned and small, then A+ is alwaysa fairly good approximation. The matrix A+ is also sparser than A, andsimilarly to BS(n) its mesh will be planar if the mesh of A was planar.

But when A is ill conditioned, A+ can be an arbitrarily bad approxi-mation even though A is approximable by some other sdd matrix.

4.3. Scaling and Assembling Element Approximations 117

Example 4.2.17. Let 0 < ε 1, and let M ≥ 4ε,

A =

1 +M −1 0 −M−1 1 +M −M 00 −M M 0−M 0 0 M

−

0 0 0 00 0 0 00 0 1− ε −1 + ε0 0 −1 + ε 1− ε

.

This matrix is symmetric semidefinite with rank 3 and null vector[1 1 1 1

]T.

For small ε, A is ill conditioned, with condition number larger than 8ε−2.The matrix A+ is a poor approximation of A,

κ(A,A+) >

(1− 1− ε

2M + 1

)ε−1 ≈ ε−1 .

Nevertheless, the matrix

B =

ε+M −ε 0 −M−ε ε+M −M 00 −M ε+M −ε−M 0 −ε ε+M

is a good approximation for A, with κ(A,B) < 9 for a small enough ε. Theproofs for this example are due to Avron, and can be found in [11].

4.3. Scaling and Assembling Element Approximations

Given a set of approximations Le to a set of element matrices Ke,our solver scales the Le’s so that their assembly L =

∑e αeLe is a good

approximation of K =∑

eKe. We can scale them in one of two equivalentways. The next lemma shows that under these scalings, if every Le is agood approximation of Ke, then L is a good approximation of K. Bothscaling rules require the computation of one extreme eigenvalue for eachpair (Ke, Le).

Lemma 4.3.1. Let Kee∈E and Lee∈E be sets of symmetric positive(semi)definite matrices with null(Ke) = null(Le). Let αe = min Λ(Ke, Le)and let βe = max Λ(Ke, Le). Then

κ

(∑e∈E

Ke,∑e∈E

αeLe

)≤ maxκ (Ke, Le)

κ

(∑e∈E

Ke,∑e∈E

βeLe

)≤ maxκ (Ke, Le)

Proof. Scaling Le by αe transforms the smallest generalized eigenvalueof Λ(Ke, αeLe) to 1, since

Λ(Ke, αeLe) =1

αeΛ(Ke, Le) .

Scaling Le clearly does not change the generalized condition number, somax Λ(Ke, αeLe) = κ(Ke, Le).

4.4. Sparsification of the Assembled SDD Approximation 118

By the splitting lemma [18],

min Λ

(∑e∈E

Ke,∑e∈E

αeLe

)≥ min min Λ(Ke, αeLe)e∈E

= min 1e∈E= 1 .

Also by the splitting lemma,

max Λ

(∑e∈E

Ke,∑e∈E

αeLe

)≤ max max Λ(Ke, αeLe)e∈E

= max κ (Ke, Le)e∈E .

Combining these two inequalities gives the result. The proof for scaling byβe is the same.

If we use inexact estimates αe for the minimum of Λ(Ke, Le), the boundbecomes

κ

(∑e∈E

Ke,∑e∈E

αeLe

)≤ maxe (αe/αe)κ (Ke, Le)

mine (αe/αe),

and similarly for estimates of βe. This shows that κ(∑

e∈EKe,∑

e∈E αeLe)

depends on how much the estimates vary. In particular, if the relativeestimation errors are close to 1, scaling by the estimates is almost as goodas scaling by the exact eigenvalues.

4.4. Sparsification of the Assembled SDD Approximation

Once we obtain an sdd approximation L =∑

e αeLe of K, we can use acombinatorial graph algorithm to construct an easier-to-factor sdd approx-imation M of L. Because M is spectrally close to L and L is spectrally closeto K, M is also spectrally close to K. By applying [24, Proposition 3.6] toboth ends of the generalized spectra, we obtain the following lemma.

Lemma 4.4.1. Let K, L, and M be symmetric (semi)definite matriceswith the same dimensions and the same null space. Then

κ(K,M) ≤ κ(K,L)κ(L,M) .

There are several algorithms that can build M from L. All of them viewan exactly (but not strictly) sdd matrix L as a weighted undirected graphGL, to which they build an approximation graph GM . The approximationGM is then interpreted as an sdd matrix M . If L is strictly diagonaldominant, the approximation starts from an exactly sdd matrix L − D,where D is diagonal with nonnegative elements. From L−D the algorithmbuilds an approximation M ; If M is a good approximation to L−D, thenM +D is a good approximation to L.

4.4. Sparsification of the Assembled SDD Approximation 119

Lemma 4.4.2. Let A and B be symmetric positive (semi)definite matri-ces with the same null space S, and let C be a positive symmetric (semi)definitewith a null space that is a subspace, possibly empty, of S. Then

Λ(A+ C,B + C) ⊆ [min Λ(A,B) ∪ 1,max Λ(A,B) ∪ 1] .

Proof. The result is a simple application of the splitting lemma [18],since

max Λ(A+ C,B + C) ≤ max max Λ(A,B),max Λ(C,C)= max max Λ(A,B), 1= max Λ(A,B) ∪ 1 ,

and similarly for the smallest generalized eigenvalue.

This lemma is helpful, since most of the algorithms that construct ap-proximations of sdd matrices provide theoretical bounds on Λ(L−D, M)that have 1 as either an upper bound or a lower bound. When this is thecase, adding D to L − D and to M preserves the theoretical condition-number bound.

The earliest algorithm for this subproblem is Vaidya’s algorithm [18,37, 145]. This algorithm finds a maximum spanning tree in GM and aug-ments it with suitable edges. The quantity of extra edges that are addedto the tree is a parameter in this algorithm. When few or no edges areadded, κ(L,M) is high (but bounded by nm/2, where m is the number ofoff-diagonal elements in L), but L is cheap to factor (can be factored inO(n) operations when no edges are added to the tree). When many edgesare added, κ(L,M) shrinks but the cost of factoring L rises. When GM isplanar or nearly planar then the algorithm is very effective, both theoret-ically and experimentally. For more general classes of graphs, even with abounded degree, the algorithm is less effective.

A heuristic for adding edges to the maximum spanning tree was pro-posed by Frangioni and Gentile [60]. They designed their algorithm forsdd linear systems that arise in the interior-point solution of network-flowproblems. There are no theoretical convergence bounds for this algorithm.

Several algorithms are based on so-called low-stretch spanning trees.Boman and Hendrickson realized that a low-stretch spanning tree GM

of GL leads to a better convergence-rate bound than maximum spanningtrees [23]. Elkin et al. showed simpler and lower-stretch constructions forlow-stretch trees [53]. Spielman and Teng proposed algorithms that createdenser graphs GM (which are still based on low-stretch trees) [141, 142,143]. The algorithms of Spielman and Teng lead to nearly-linear workbound for solving Lx = b.

There are other classes of combinatorial preconditioners, for exam-ple [70, 69, 108]. It is not clear whether they can be used effectivelyin our framework.

4.5. Dealing with Inapproximable Elements 120

4.5. Dealing with Inapproximable Elements

When some of the element matrices cannot be approximated well byan sdd matrix, we split the global stiffness matrix K into K = K≤t +K>t, where K≤t =

∑e∈E(t) Ke is a sum of the element matrices for which

we found an sdd approximation Le that satisfies κ(Ke, Le) ≤ t for somethreshold t > 0, and K>t =

∑e6∈E(t) Ke is a sum of element matrices for

which our approximation Le gives κ(Ke, Le) > t.We then scale the approximations in E(t) and assemble them to form

L≤t =∑

e∈E(t) αeKe. We now apply one of the combinatorial graph al-gorithms discussed in Section 4.4 to construct an approximation M≤t toL≤t. Once we have M≤t, we add it to K>t to obtain a preconditionerM1 = M≤t +K>t.

This construction gives a bound on κ(K,M1), but it is a heuristic inthe sense that M1 may be expensive to factor. The analysis of κ(K,M1)is essentially the same as the analysis of strictly dominant matrices in Sec-tion 4.4: by Lemma 4.4.2, a theoretical bound Λ(K≤t,M≤t) ⊆ [α, β] impliesΛ(K,M1) ⊆ [minα, 1,maxβ, 1].

The scaling technique of Lemma 4.3.1 ensures that either α, β ≤ 1 or1 ≤ α, β. But the interval [α, β] may be quite far from 1. If the intervalis far from 1, the bound on κ(K,M1) can be considerably larger then thebound on κ(K≤t,M≤t). We observed this behavior in practice (experimentsnot reported here). To avoid this danger, we scale M≤t before adding itto K>t; that is, we use a preconditioner Mγ = γM≤t + K>t. We chooseγ as follows. We find some vector v that is orthogonal to null(M≤t) andcompute its generalized Raleigh quotient

γ =vTK≤tv

vTM≤tv.

The null space of M≤t is determined by the connected components of itsgraph, so it is easy to quickly find such a v (we use a random v in thissubspace). This definition ensures that γ ∈ [α, β]. Since Λ(K≤t, γM≤t) ⊆[α/γ, β/γ], we have 1 ∈ [α/γ, β/γ].

Lemma 4.5.1. Under this definition of Mγ, κ(K,Mγ) ≤ β/α, where theinterval [α, β] bounds Λ(K≤t,M≤t). In particular, if we take α and β to bethe extremal generalized eigenvalues of (K≤t,M≤t), we obtain

κ(K,Mγ) ≤ κ(K≤t,M≤t) .

We expect that this overall heuristic will be effective when E \ E(t)contains only few elements. If only a few elements cannot be approximated,then K>t is very sparse, so the sparsity pattern of Mγ = γM≤t + K>t issimilar to that of M≤t. Since M≤t was constructed so as to ensure that itssparsity pattern allow it to be factored without much fill, we can expectthe same to hold for Mγ. If E \E(t) contains many elements, there is littlereason to hope that the triangular factor of Mγ will be particularly sparse.

4.6. Asymptotic Complexity Issues 121

If E \E(t) contains very few elements a different strategy can be used.Instead of introducing the ill-conditioned elements into Mγ they can beignored. Each ignored element can be considered as an ne − 1 rank per-turbation. Results in [10] suggest that this will increase the number ofiterations by at most ne − 1. Therefore, if the number of inapproximableelements is small only a few more iterations will be needed. If many inap-proximable elements are dropped convergence can be slow.

4.6. Asymptotic Complexity Issues

In this section we explain the asymptotic complexity of the differentparts of our solver. We do not give a single asymptotic expression thatbounds the work that the solver performs, but comment on the cost andasymptotics of each phase of the solver. The cost of some of the phases ishard to fully analyze, especially when E(t) ( E. The next section presentsexperimental results that complement the discussion here.

The first action of the solver is to approximate each element matrixKe by an sdd matrix αeLe. For a given element type, this phase scaleslinearly with the number of elements and it parallelizes perfectly. The per-element cost of this phase depends on the approximation method and onthe number ne of degrees of freedom per element. Asymptotically, all theapproximation methods require n3

e operations per element, but the uniform-clique is the fastest method. This phase also gives us κ(Ke, αeLe) whichwe use to decide which elements belong to E(t) and which do not.

The next phase of the solver assembles the scaled sdd approximationsin E(t). The cost of this step is bounded by the cost to assemble K =∑

eKe, which most finite-elements solvers (including ours), perform. Theassemblies of K and L performs O(

∑e n

2e) operations: fewer than the first

phase, but harder to parallelize.The cost and the asymptotic complexity of the sparsification of L de-

pends on the algorithm that is used. For Vaidya’s sparsification algorithm,which our code uses, the amount of work is O(n log n+

∑e n

2e). For the al-

gorithm of Spielman and Teng [141, 142, 143], the work is O(m logO(1)m)where m =

∑e n

2e.

Next, the algorithm assembles the element matrices Ke that are not inE(t) into M≤t. The cost of this phase is also dominated by the cost ofassembling K.

The cost of computing the Cholesky factorization of M is hard to char-acterize theoretically, because the cost depends on the nonzero pattern ofM in a complex way. The nonzero pattern of M depends on how many andwhich elements are not in E(t), and on how much we sparsified L≤t. Thenumber of operations in a phase of the solver is not the only determinant ofrunning time, but also the computational rate. The Cholesky factorizationof M usually achieves high computational rates.

The cost of the iterative solver depends on the number of iterationsand on the per-iteration cost. The number of iterations is proportional to

4.7. Experimental Results 122√κ(K,Mγ) ≤ t

√κ(L,M≤t). The amount of work per iteration is propor-

tional to the number of nonzeros in K plus the number of nonzeros in theCholesky factor of M . The sparsification algorithms of Vaidya and Spiel-man and Teng control the number of iterations, and if E(t) = E than theyalso control the density of the Cholesky factor.

4.7. Experimental Results

This section presents experimental results that explore the effectivenessof our solver.

4.7.1. Setup. Our solver currently runs under Matlab [111], but it isimplemented almost entirely in C. The C code is called from Matlab usingMatlab’s cmex interface. The element-by-element approximations arecomputed by C code that calls lapack [6]. The assembly of the element-by-element approximations (and possibly the inapproximable elements) is alsodone in C. The construction of Vaidya’s preconditioners for sdd matricesis done by C code [37]. The Cholesky factorization of the preconditioneris computed by Matlab’s sparse chol function, which in Matlab 7.2calls cholmod 1.0 by Tim Davis. We always order matrices using metisversion 4.0 [95] prior to factoring them. The iterative Krylov-space solverthat we use is a preconditioned Conjugate Gradients written in C andbased on Matlab’s pcg.; within this iterative solver, both matrix-vectormultiplications and solution of triangular linear systems are performed byC code.

In most experiments we compare our solver to an algebraic multigridsolver, BoomerAMG [79]. We use the version of BoomerAMG that ispackaged as part of hypre 1.2. We compiled it using gcc version 3.3.5,with options -O (this option is hypre’s default compilation option). Wenote that BoomerAMG is purely algebraic and does not exploit the element-by-element information. There exist variations of algebraic multigrid thatdo exploit the element structure [30, 35]. We have not experimented withthese variants. Our comparison with BoomerAMG is meant mainly toestablish a credible baseline for the results and not to explore the behaviorof algebraic multigrid solvers.

In some experiments we compare our solver to solvers that are based onincomplete Cholesky preconditioners [110, 113, 130, 151]. To computethese preconditioners, we use Matlab’s built-in cholinc routine. Heretoo, the matrices are preordered using metis.

Since many of our test problems are ill conditioned, we iterate until therelative residual is at most 10−14, close to εmachine, in order to achieveacceptable accuracy.

We use two mesh generators to partition the three-dimensional problemdomains into finite elements. We usually use tetgen version 1.4. [139]. Ina few experiments we use distmesh [122], which can generate both two-and three-dimensional meshes.

4.7. Experimental Results 123

Table 1. Notation for the test problems.

The Domain ΩC A 3-dimensional cubeB A 3-dimensional box with aspect ratio 1-by-1-by-10000CH A 1-by-1-by-1 cube with a 1-by-0.1-by-0.79 hole in the

middleSC A 10-by-10-by-10 cube containing a spherical shell of inner

radius 3 and thickness 0.1.The Mesh (the parameter indicates the number n of mesh points)G 3-dimensional, generated by tetgenD 3-dimensional, generated by distmeshThe Conductivity θ(x)U uniform and isotropic, θ = I everywhereJ jump between subdomains but uniform and isotropic

within subdomain (e.g., θ = 104I in the spherical shell ofdomain SC and Θ = I elsewhere); the parameter indicatesthe magnitude of the jump

A anisotropic within a subdomain (e.g. the spherical shell inSC) and θ = I elsewhere; θ is always 1 in the x and ydirections and the parameter indicates the conductivity inthe z direction.

The element typeL Linear tetrahedral element, 4-by-4 element matrixQ Quadratic tetrahedral element, 10-by-10 element matrix

Running times were measured on a 1.6 GHz AMD Opteron 242 com-puter with 8 GB of main memory, running Linux 2.6. This computer has2 processors, but our solver only uses one. We used a 64-bit version ofMatlab 7.2. This version of Matlab uses the vendor’s blas, a librarycalled acml. The measured running times are wall-clock times that weremeasured using the ftime Linux system call.

4.7.2. Test Problems. We evaluated our solver on several three-dimensionalproblems. We used both linear and quadratic tetrahedral elements. Table 1summarizes the problems that we used in the experiments. The boundaryconditions are always pure Neumann ∂u/∂n = 0, and we removed the sin-gularity by fixing the solution at a fixed unknown (algebraically, we removethe row and column of K corresponding to that unknown). We generatethe right-hand side b of the linear system Kx = b by generating a randomsolution vector x and multiplying it by K to form b.

In all the experiments reported below, except for a single experiment,our solver produced acceptable forward errors. The computed solution xsatisfied

‖x− x‖2

‖x‖2

≤ 10−4 .


100

105

10100

0.2

0.4

0.6

0.8

1

Condition Number

Per

cent

CH_D8954_U_L

κ(K

e)

0 2 4 6 8 10x 10

5

101

102

103

104

NNZ in the Cholesky factor

Itera

tions

CH_D8954_U_L

t = 101

t = 102

t = 103

t = 104

t = 1015

100

105

10100

0.2

0.4

0.6

0.8

1

Condition Number

Per

cent

CH_G32867_U_L

κ(K

e)

0 2 4 6 8x 10

6

101

102

103

104


Itera

tions

CH_G32867_U_L

t = 101

t = 102

t = 103

t = 104

t = 1015

Figure 4.7.1. The distribution of element condition num-bers (left graphs) and the number of iterations for several val-ues of t and several sparsification levels (right graphs). Thetop row shows results on a mesh generated by distmesh,and the bottom row on a mesh generated by tetgen.

In one of the experiments with well-conditioned elements and jumping co-efficients with ratio 108 (Section 4.7.5), when running Vaidya’s precondi-tioner with the goal 0.6, the forward error was 1.24 · 10−3.

4.7.3. Choosing the Parameter t. We begin with simple problemsthat are designed to help us choose t, the approximation threshold. Thebehavior of our solver depends on two parameters, t and the aggressivenessof the combinatorial sparsification algorithm. These parameters interact incomplex ways, because both influence the sparsity of the Cholesky factorof M and the number of iterations in the Krylov-space solver. It is hard tovisualize and understand the performance of a solver in a two-dimensional(or higher) parameter space. Therefore, we begin with experiments whosegoal is to establish a reasonable value for t, a value that we use in all of thesubsequent experiments.

Figure 4.7.1 shows the results of these experiment, which were carriedout on two meshes, one generated by distmesh and the other by tetgen.The elements are all linear tetrahedral, and their approximations Le arebuilt using our nearly-optimal approximation algorithm. The graphs on


the left show the distributions of κ(Ke). With distmesh, we see thatthe elements belong to two main groups, a large group of elements withgeneralized condition numbers smaller than about 100, and a small set ofso-called slivers with much higher condition numbers, ranging from 200to 108. From results not shown here, it appears that for the non-slivers,κ(Ke, Le) is smaller than κ(Ke) by roughly a constant factor. For theslivers, κ(Ke, Le) is close to κ(Ke). With tetgen, there are no highlyill-conditioned elements, and the distributions of κ(Ke) and κ(Ke, Le) aresmoother.

The graphs on the right show the number of iterations that the Conju-gate Gradients algorithm performs for several values of t and various levelsof sparsification. In all the graphs in the chapter whose horizontal axis isfill in the Cholesky factor, the horizontal axis ranges from 0 to the numberof nonzeros in the Cholesky factor of K. When t is small, K>t is relativelydense, so the sparsification algorithm cannot be very effective. Even whenwe instruct Vaidya’s algorithm to sparsify L≤t as much as possible, theCholesky factor of M remains fairly similar to the Cholesky factor of K.On the other hand, a small t leads to faster convergence. With a large t wecan construct M ’s with very sparse factors, but convergence is very slow.If all the elements are relatively well-conditioned then there is little depen-dence on t, as can be seen in the bottom right figure. A value of t near1000 gives a good balance between high iteration counts caused by usingLe’s with fairly high κ(Ke, Le) and the inability to construct a sparse pre-conditioner caused by a dense K>t. We use the fixed value t = 1000 in theremaining experiments in order to clarify the role of the other parameterin the algorithm.

We stress that the selection of t in practice should not be static; it shouldbe based on the actual distribution of the generalized condition numbersof the approximation and on analysis similar to the one described in thissection.

4.7.4. Baseline Tests. The next set of experiments shows the per-formance of our solver relative to other solvers on the same problems andthe same meshes, for a few relatively easy problems. The graphs in Fig-ure 4.7.2 compare the running time of our solver to that of an incompleteCholesky preconditioner, BoomerAMG, and a state-of-the-art direct solver,cholmod. In these graphs the vertical axis represents wall-clock time forall the phases of the solution and the horizontal axis represents the numberof nonzeros in the triangular factors of the preconditioner. The rightmost(largest) horizontal coordinate in the graphs always corresponds to thenumber of nonzeros in a complete sparse Cholesky factor of the coefficientmatrix. When the complete factorization runs out of space, we still usethis scaling of the horizontal axis, and we estimate the running time ofthe complete factorization based on the assumptions that it runs at 109

floating-point operations per second. The direct solver and BoomerAMG


0 2 4 6 8 10x 10

5

0

10

20

30

40

50

60


Tim

e (s

ec)

CH_D8954_U_L

DirectAMGNOC+Vaidyacholinc

0 2 4 6 8x 10

6

0

10

20

30

40

50


Tim

e (s

ec)

CH_G32867_U_L


Figure 4.7.2. Running times for our solver, for incompleteCholesky, for BoomerAMG, and for a direct solver on simple3-dimensional problems. The graph on the left uses a meshgenerated by distmesh, and the one on the right a mesh gen-erated by tetgen. See the first paragraph of Section 4.7.4for a complete explanation of the graphs.

only give us one data point for each problem; their running times are rep-resented in the graphs by horizontal lines. We ran each preconditionedsolver with several values of the parameter that controls the sparsity of thefactor (drop tolerance in incomplete Cholesky and the sparsification param-eter in Vaidya’s preconditioner). Therefore, for each preconditioned solverwe have several data points that are represented by markers connected bylines. Missing markers and broken lines indicate failures to converge withina reasonable amount of time. Our algorithm is labeled as “NOC+Vaidya”.NOC stands for “Nearly-Optimal Clique” because our algorithm uses aclique topology for the approximation. Most of the remaining graphs inthis section share the same design.

The graphs in Figure 4.7.2 compare the running time of the solvers oneasy problems with a relatively simple domain and uniform coefficients.The mesh produced by tetgen leads to a linear system that is easy forall the iterative solvers. With a good drop-tolerance parameter, incompleteCholesky is the fastest, with little fill. Our solver is slower than all the rest,even with the best sparsity parameter. The mesh produced by distmeshcauses problems to BoomerAMG, but incomplete Cholesky is faster thanour solver.

Although the performance of incomplete Cholesky appears to be good inthe experiments reported in Figure 4.7.2, it sometimes performs poorly evenon fairly simple problems. Figure 4.7.3 shows that on a high-aspect-ratio3-dimensional structure with uniform coefficients, incomplete Cholesky per-forms poorly: the sparser the preconditioner, the slower the solver. Oursolver, on the other hand, performs reasonably well even when its factor


0 1 2 3x 10

6

0

200

400

600

800

1000

1200


Tim

e (s

ec)

B_G196053_U_L


0 0.5 1 1.5 2 2.5x 10

8

0

200

400

600

800

1000


Tim

e (s

ec)

C_G326017_U_Q

Estimated DirectAMGNOC+Vaidyacholinc

Figure 4.7.3. Experimental results on two additional prob-lems with uniform coefficients; these problems are muchlarger than those analyzed in Figure 4.7.2.

0 0.5 1 1.5 2 2.5x 10

8

0

200

400

600

800

1000

1200

1400


Tim

e (s

ec)

SC_G395700_A1e1_Q

(1)(1)+..+(3)(1)+..+(5)(1)+..+(7)

Notation for phases of the solver:(1) Approximate Ke by Le

(2) Assembly L≤t =∑

E(t) Le

(3) Sparsify L≤t to obtain M≤t

(4) Assembly of M = M≤t +K>t

(5) Order, permute, and factor M(6) Assembly of K =

∑E Ke

(7) Permute K and iterate

Figure 4.7.4. A breakdown of the running time of oursolver, on a particular problem. The graph shows the timeconsumed by the different phases of the solver. Assemblyphases are not separately shown because their running timeis negligible.

is much sparser than the complete factor. On the high-aspect-ratio prob-lem, as well as on any problem of small to moderate size, the direct solverperforms well. But as the problem size grows the direct solver becomesslow and tends to run out of memory. The rightmost graph in Figure 4.7.3shows a typical example.

Figure 4.7.4 shows a breakdown of the running time of our solver forone particular problem. The data shows that as the preconditioner getssparse, the time to factor the preconditioner decreases. The running timeof the iterative part of the solver also initially decreases, because the pre-conditioner gets sparser. This more than offsets the growth in the numberof iterations. But when the preconditioner becomes very sparse, it becomesless effective, and the number of iterations rises quickly.


0 0.5 1 1.5 2 2.5x 10

8

0

200

400

600

800

1000

1200

1400


Tim

e (s

ec)

SC_G395700_J1e4_Q


0 0.5 1 1.5 2 2.5x 10

8

0

200

400

600

800

1000

1200

1400


Tim

e (s

ec)

SC_G395700_J1e8_Q


Figure 4.7.5. Running times for problems with jumping co-efficients. The average of the ratio κ(Ke)/κ(Le, Ke) in bothexperiments is around 3.6 and is nearly constant. The con-dition numbers of the Ke’s are always small.

4.7.5. Well-Conditioned Elements and Jumping Coefficients.The next set of experiments explores problems with a large jump in theconductivity θ. We instructed the mesh generators to align the jump withelement boundaries, so within each element, there is no jump. This leadsto a large κ(K), but the conditioning of individual element matrices isdetermined by their geometry, not by θ. The results, shown in Figure 4.7.5,show that the jump in θ does not influence any of the four solvers in asignificant way.

4.7.6. Ill-Conditioned Elements: Anisotropy. Some of the exper-iments shown in Section 4.7.3 included ill-conditioned elements. The ill-conditioning of those elements resulted from their geometrical shape. Othermesh generators may be able to avoid such element shapes. Indeed, tet-gen did not produce such elements, only distmesh did. But in problemsthat contain anisotropic materials in complex geometries, ill-conditionedelements are hard to avoid.

Figure 4.7.6 compares the performance of our solver with that of othersolvers on a problem in which the conductivity θ is anisotropic in onepart of the domain. The results clearly show that anisotropy leads toill-conditioned element matrices. As the anisotropy increases, Boomer-AMG becomes slower and incomplete Cholesky becomes less reliable. Theanisotropy does not have a significant influence on our solver. In experi-ments not reported here, our solver behaved well even with anisotropy of108. The incomplete-factorization solver becomes not only slow, but alsoerratic, as the graphs in Figure 4.7.7 show. The convergence of our precon-ditioner is always steady, monotonically decreasing, and the convergencerate is monotonic in the density of the preconditioner. The convergenceof incomplete Cholesky is erratic, not always monotonic, sometimes veryslow. Furthermore, sometimes one incomplete factor leads to much faster


100

102

104

106

1080

0.2

0.4

0.6

0.8

1

Condition Number

Per

cent

Condition numbers

θ = 101

θ = 102

θ = 103

0 0.5 1 1.5 2 2.5x 10

8

0

1000

2000

3000

4000


Tim

e (s

ec)

SC_G395700_A1e1_Q


0 0.5 1 1.5 2 2.5x 10

8

0

1000

2000

3000

4000


Tim

e (s

ec)

SC_G395700_A1e2_Q


0 0.5 1 1.5 2 2.5x 10

8

0

1000

2000

3000

4000

5000

6000


Tim

e (s

ec)

SC_G395700_A1e3_Q


Figure 4.7.6. The behavior of our solver and other solverson a solid 3-dimensional problem that contains a thin spher-ical shell with anisotropic material. The mesh was gener-ated by tetgen. We report here three experiments withanisotropy rates 10, 102, and 103 (θ in the legend). Thetop left graph shows the distribution of the elemenat ma-trix condition numbers in the different experiments. Theratio κ(Ke)/κ(Le, Ke) is nearly constant within every exper-iment. This ratio for the isotropic elements is similar in allthe experiments. For the anisotropic elements, the maximalratio grows from about 19 for anisotropy 10 to about 806 foranisotropy 103.

convergence than a much denser incomplete factor. We acknowledge thatin some cases, when targeting larger relative residuals (like 10−2 or 10−5),incomplete factorization and multigrid preconditioners are more effectivethan combinatorial preconditioners. This is evident in other combinato-rial preconditioners (e.g., [37] and Section 2.9). This is clearly shown inFigure 4.7.7.

These results show that the ability of our solver to detect inapprox-imable elements and to treat them separately allows it to solve problemsthat cause difficulty to other iterative solvers. When there are few such ele-ments, as there are here because the anisotropic shell is thin, these elements


0 500 1000 1500 2000 2500 300010

−15

10−10

10−5

100

Time (sec)

Rel

ativ

e R

esid

ual

SC_G395700_A1e2_Q

NOC+Vaidya, NNZ = 1.1e6NOC+Vaidya, NNZ = 1.4e6NOC+Vaidya, NNZ = 6.9e6NOC+Vaidya, NNZ = 47.0e6NOC+Vaidya, NNZ = 202.1e6AMG

0 500 1000 1500 2000 2500 300010

−15

10−10

10−5

100

Time (sec)

Rel

ativ

e R

esid

ual

SC_G395700_A1e2_Q

Cholinc, NNZ = 0.4e6Cholinc, NNZ = 1.4e6Cholinc, NNZ = 6.1e6Cholinc, NNZ = 16.8e6Cholinc, NNZ = 41.9e6AMG

Figure 4.7.7. The relative norm of the residual as a func-tion of the running time (and implicitly, of the iterationcount) on one anisotropic problem, for our solver (left) andfor incomplete Cholesky (right). The horizontal coordinatein which individual plots start indicates the time to constructand factor the preconditioner.

do not significantly increase the density of the factor of M . In problems inwhich most of the elements are inapproximable by sdd matrices, M wouldbe similar to K and the characteristics of our solver would be similar tothe characteristics of a direct solver.

This is perhaps the most important insight about our solver. As prob-lems get harder (in the sense that more elements become inapproximable),its behavior becomes closer to that of a direct solver. As problems getharder we lose the ability to effectively sparsify the preconditioner prior toto factoring it. But unlike other solvers, our solver does not exhibit slowor failed convergence on these difficult problems.

4.7.7. Comparisons of Different Element-by-Element Approxi-mations. We now explore additional heuristics for approximating Ke. Theapproximation methods that we compare are:

Nearly Optimal Clique (NOC): Le = ZDDTZ, where the columnsof Z is the full set of edge vectors and D scales the columns ofU+Z to unit 2-norm. This method gives the strongest theoreticalbound of the metods we tested on κ(Ke, Le): it is at most n2

e/2times larger than the best possible for an sdd approximation ofKe. Here and in the next four methods, we set the scaling factorαe to be max Λ(Ke, Le).

Nearly Optimal Star (NOS): Le = ZDDTZ, where the columnsof Z are edge vectors that form a star, 〈1,−2〉 , 〈1,−3〉 , . . . , 〈1,−ne〉and D scales the columns of U+Z to unit 2-norm. Sparser thanthe first but usually a worse approximation.


Uniform Clique (UC): Le is the extension of the ne-by-ne matrixBC(ne) to an n-by-n matrix. Computing Le is cheap, but the ap-proximation is only guaranteed to be good when Ke is very wellconditioned. The low cost of this method stems from the factsthat (1) BC(ne) is a fixed matrix, and (2) αn = max Λ(Ke), so wedo not need to estimate an extreme generalized eigenvalue, only asingle-matrix eigenvalue.

Uniform Star (US): Le is the extension of the ne-by-ne matrixBS(ne) to an n-by-n matrix. Sparser than the uniform clique, butmore expensive, since we compute an extreme generalized eigen-value to set αe.

Positive Part (PP): Le = (Ke)+, defined in Section 4.2.6.Boman et al. (BHV): αeLe is the Boman-Hendrickson-Vavasis ap-

proximation of Ke [25]. In their method, Le is a uniform star, andthe scaling factor αe is computed from quantities associated withthe finite-element discretization that produces Ke.

The results are shown in Figure 4.7.8. When element matrices arefairly well conditioned (bottom graphs), different approximation methodsexhibit similar qualitative behaviors. But when there are ill-conditionedelements, naive approximation methods perform poorly. The results alsoshow that the nearly-optimal approximation (which we used in all the otherexperiments) performs well relative to other approximation methods, butis usually not the best.

The comparision between PP and NOC in the top experiments in Fig-ure 4.7.8 is interesting. The top left graph indicates that PP is a betterapproximation in that experiment. This was consistent in additional pa-rameters that we explored and are not reported here: abosulte conditionnumber, number of elements that were inapproximable (beyond t = 1000),the distribution of the ratio κ(Ke, Le)/κ(Ke), sparsity of the preconditioner,and time to compute the approximation.

Nevertheless, the number of iterations with the NOC+Vaidya precondi-tioner is substantially smaller (about half) of the number of iterations withthe PP+Vaidya preconditioner. The bottom line is that NOC+Vaidya per-forms better than PP+Vaidya. We do not have a good explanation for this;it may be a good subject for future research.

4.7.8. Running Times for Growing Problem Sizes. The resultsin Figure 4.7.9 present the growth in running time of our solver as theproblem size grows. For very dense and very sparse preconditioners, thegrowth is highly nonlinear. This is consistent with the theory of sparsedirect solvers on one side and with the theory of Vaidya’s preconditionerson the other. For intermediate levels of fill, running times grow more slowly,but they still seem to grow superlinearly.

4.8. Open Problems 132

10−2

100

102

104

1060

0.2

0.4

0.6

0.8

1

(Generalized) Condition Number

Per

cent

SC_G93202_A1e2_Q

USUCNOSNOCPPTheortical Bound

0 0.5 1 1.5 2 2.5x 10

7

0

50

100

150

200


Tim

e (s

ec)

SC_G93202_A1e2_Q

USUCNOSNOCPP

10−2

100

102

104

1060

0.2

0.4

0.6

0.8

1

(Generalized) Condition Number

Per

cent

SC_G395700_J1e4_Q

USUCNOSNOCPPBHVTheortical Bound

0 0.5 1 1.5 2 2.5x 10

8

0

200

400

600

800

1000

1200

1400


Tim

e (s

ec)

SC_G395700_J1e4_Q

USUCNOSNOCPPBHV

Figure 4.7.8. Different element-approximation methods.The graph on the left shows the distribution of κ(Ke, Le)for each approximation method. The method of Boman etal. is only applicable to well-conditioned elements, so it isnot included in the graphs in the top row. The theoreticalbound in the left figure is calculated using the fact that thespectral distance of NOC is within n2

e/2 of the best possible.In the bottom left figure, the BHV (diamond markers) plotoccludes the US (star markers) plot.

4.8. Open Problems

This chapter raises several interesting questions and challenges for fur-ther research. We mention three. One challenge is to extend the optimal-scaling method of Braatz and Morari [27] to rank-deficient and rectangularmatrices. It may be possible to use the reduction in [152] to solve this prob-lem, but we have not explored this. It is not even clear whether van derSluis’s nearly-optimal scaling for rectangular matrices [147] is also nearly-optimal for rank-deficient matrices. Another interesting question is to finda reliable and cheap-to-compute estimate of the spectral distance betweena given symmetric positive (semi)definite matrix A and the closest sddmatrix to A. Our method can be used to estimate this distance, but forlarge matrices the method is expensive and its estimates are loose. We havealso shown that κ(A) is an upper bound on that distance, but this bound

4.8. Open Problems 133

0 5 10 15x 10

4

0

50

100

150

200

250

300

350

n

Tim

e (s

ec)

scaling − Vaidya

Goal = 0.00Goal = 0.40Goal = 0.60Goal = 0.80Goal = 0.95Goal = 1.00

0 5 10 15x 10

4

0

20

40

60

80

100

120

n

Tim

e (s

ec)

scaling − all (best)

DirectAMGVaidya, Goal = 0.60Cholinc, Droptol = 0.10

Figure 4.7.9. Solution times as a function of mesh size, onthe same physical problem (a cube with uniform coefficients,discretized using linear tetrahedral elements). The graph onthe left compares the running times of our solver with dif-ferent levels of fill, and the graph on the right compares oursolver (with the best-case fill level) with BoomerAMG andthe direct solver. The fill in our solver is controlled by aparameter called goal in these graphs. A goal of 1 does notsparsify the approximation M≤t, and a goal of 0 sparsifiesit as much as possible, resulting in a tree or forest graphstructure for L≤t.

can be arbitrarily loose. The third and probably most important challengeis to find better ways to exploit the splitting K = K≤t + K>t. There maybe several ways to exploit it. For example, it is probably possible to buildbetter preconditioners by sparsifying L≤t with the objective to reduce fillin the Cholesky factor of M≤t + K>t; the algorithm that we used for thesparsification phase ignores K>t and only tries to reduce fill in the factorof M≤t.

CHAPTER 5

Finite-Element Formulation for SurfaceReconstruction

5.1. Introduction

This chapter1 presents a finite-element formulation for surface recon-struction. As 3D scanners are becoming commonplace, surface reconstruc-tion is becoming a common step in modeling workflows. Thus, there is anemerging need to allow users to create high-quality surfaces from scans,including imperfect scans. This need brought much attention to surfacereconstruction techniques. Recent research efforts have led to significantprogress in all aspects of the problem [103]. However, the reconstruction ofa complete watertight model2 that is faithful to the original physical objectis still a difficult process.

One of the major difficulties is the coverage of the scanned model: Asa result of physical inaccessibility, poor visibility and material properties,the coverage is often imperfect and significant portions of the surface areeither under-sampled or completely missing. The problem is more acutefor complex shapes with deep cavities and bifurcations (e.g., Figure 5.1.1).While it is reasonable to assume that scanning hardware will advance, fu-ture reconstruction systems will still need to employ algorithms that re-construct under-sampled areas. Systems that reconstruct poorly-sampledareas merely based on priors, without any user intervention, fail in manycases to faithfully reconstruct the expected shape. At the other extreme,systems that rely only on explicit manual surface editing are too tedious.

Without prior assumptions and user constraints, the reconstructionproblem is ill posed; an infinite number of surfaces pass through or near thedata points. Smoothness and watertight constraints usually regularize theproblem and remove the ill posedness. Nevertheless, even if the problemis successfully transformed into a well conditioned one, the reconstructed

1The results in this chapter are contained in a paper published in SIGGRAPH 2007, co-authored by Andrei Sharf, Thomas Lewiner, Sivan Toledo and Daniel Cohen-Or [136].My contribution to this work is the formulation of the mesh-free optimization goal, theformulation of the smoothness penalty as an energy integral, its finite-element formula-tion, and its implementation. In particular, section 5.3 is my direct contribution. Theother components of the interactive surface reconstruction method are due to AndreiSharf, Thomas Lewiner, and Daniel Cohen-Or. They are described in detail in [136].2A closed-surface model that has all gaps and holes filled to clearly define an inside andoutside.

134


Figure 5.1.1. Interactive reconstruction of the riding monk(left). Our finite-element field formulation (center-left) incor-porates user’s interactive scribbles at automatically-detectedunstable regions (center-right) to obtain the expected shape(right).

object is not necessarily the expected one. Our method is based on the ob-servation that it is often possible to detect the ill conditioning and to askthe user for inside/outside constraints to locally resolve them and achievethe expected shape.

In this chapter we present a topology-aware reconstruction techniquethat requires minimal user input to make correct decisions at critical re-gions, where the topology of the shape cannot be induced automaticallywith a reasonable degree of confidence. Our method uses priors to re-construct the surface, but it also allows the user to influence the priordistribution. Two aspects of the prior distribution are fixed: we assumethat the surface is smooth almost everywhere, and that it should be wa-tertight. Other aspects of the prior distribution are controlled by the userwho specifies constraint points that should be inside or outside the surface.

To reconstruct a watertight surface given raw scans without normals,and possibly the user’s inside/outside constraints, we first construct a con-tinuous function over a three-dimensional domain. The zero level-set ofthis function approximates the data points. We construct this function byminimizing a penalty that measures its non-smoothness, the deviation ofits zero level-set from the data points, and its deviations from prescribedpositive/negative values at the inside/outside constraints. Our functionoptimization problem is formulated in a mesh-independent manner, andmapped onto a specific mesh using the finite-element method. Computa-tionally, the function is constructed by solving a large sparse linear system.However, at complex under-sampled regions these constraints might be in-sufficient. Therefore, we analyze the local topological stability of the zerolevel-set to detect weak regions of the surface. These regions are suggestedto the user for adding local inside/outside constraints by merely scribblingover a 2D tablet corresponding to a cross section of the field (see Fig-ure 5.1.2). The new user input augments the linear system with additional

5.2. Our Approach 136

Figure 5.1.2. Reconstruction pipeline for a scanned ele-phant model. An initial coarse function is automatically com-puted from loose, automatic inside/outside (blue/red balls)constraints (left). The topological analysis of this functionselects weak regions (center-left), where the user can makelocal decisions by scribbling over 2D tablets at a coarse res-olution (center-right). Further iterations at finer resolutionslead to a complete reconstruction of the model (right).

constrains, improving the reconstruction. The stability analysis is then re-peated. If the surface is still topologically unstable, the user is promptedfor additional constraints. This incremental process refines the surface untilit is topologically stable.

The problem of reconstructing a surface from scans has been researchedextensively for almost two decades. See [136] for a detailed analysis of therelated work.

5.2. Our Approach

Our surface reconstruction method initially computes a smooth three-dimensional implicit function whose zero level-set is a coarse approximationof the data points. The method then explicitly extracts the reconstructedsurface from this function as its zero level-set. We formulate this implicitfunction as the solution of a physical minimization problem, which we solveusing the finite-element method (FEM). As Kazhdan et al. [98] showed,defining implicit functions as a solution of a linear system guarantees aglobal coherent solution that is watertight. Moreover, it leads to a wellconditioned sparse system whose size is proportional to the reconstructedsurface. Nevertheless, our system is designed to be interactive and to in-corporate the user’s constraints.

The main challenge, shared by any method which fits implicit func-tions [33], is to retrieve zero values only at the surface. Necessarily, thesemethods require additional information, like the points’ orientations (i.e.,the normal) which is typically less precise than the points’ location. Fur-thermore in under-sampled regions, the orientation of the data points is notreliable. This may cause significant misinterpretation of the data, leadingto an erroneous surface reconstruction.

Therefore, we argue that user assistance is necessary to correctly inter-pret the data, especially in under-sampled regions. The key point in ourwork is to analyze the implicit function and to identify topologically weak

5.3. Constrained FEM Reconstruction 137

regions, in the sense that small perturbations of the field lead to differenttopological interpretations of the data. Our system indicates such weakregions, allowing the user to merely draw few scribbles over a correspond-ing 2D tablet to specify the correct local sign of the implicit function. Asthe user draws such scribbles, the solution in that region is correctly con-strained and the local topological stability increases. The mechanism ofidentification of weak regions is described in detail in [136].

A common source of errors is the use of a fixed hierarchical data struc-ture, since coarse resolution close to the surface would generate erroneousreconstructions. The volumetric nature and the required accuracy of thesolution necessitate the use of an adaptive domain, which ideally is coarseaway from the surface and fine close to it. However, since the locus ofthe surface is unknown a priori, the definition of a correct adaptive domainturns into a chicken/egg problem. Therefore, we use a dynamically adaptedhierarchical structure, generated by triangulating the dual of an octree (seeFigure 5.3.3 (right)). Our reconstruction process performs in a coarse-to-fine, incremental manner. At each iteration, the hierarchy is locally refinedclose to the zero level-set of the current implicit function. Next, the func-tion is updated and analyzed, suggesting finer user interactions at weakregions.

We illustrate the process in Figure 5.1.2. Initially, a set of automaticconstraints is computed from the data points (Figure 5.1.2 (left)), definingapproximate inside/outside constraints. Next, the FEM system is solvedand the topology of the resulting implicit function is analyzed. At weakregions (Figure 5.1.2 (center-left)), the user is guided to add scribbles on2D tablets for regularizing the local topology (Figure 5.1.2 (center-right)).Starting at this coarse resolution, the process is iteratively refined by locallyincreasing the resolution until achieving the expected shape (Figure 5.1.2(right)).

The remainder of the chapter is organized as follows. Section 5.3 intro-duces our formulation of the reconstruction problem. In Section 5.4 we givemore technical details. We conclude after showing reconstruction results ofstructured light scans in Section 5.5.

This chapter does not contain a detailed presentation of the topologicalanalysis of the implicit function, and the definition of weak regions. Bothissues are described in [136, Section 5] and are the contributions of AndreiSharf, Thomas Lewiner, and Daniel Cohen-Or.

5.3. Constrained FEM Reconstruction

In this section, we describe the formulation of our implicit functionoptimization. Let P s be the data points’ set, and P in/P out be sets of in-side/outside constraint points. Our goal is to construct a smooth watertightsurface Z that is close to P s, and separates P in and P out.

The problem defines three criteria, one of which is binary (disconnected-ness of P in and P out) and two of which are continuous and high dimensional


Figure 5.3.1. A 2D illustration of a hummingbird recon-struction. The reconstructed shape is extracted from the zerolevel-set (in yellow) of an implicit function, computed usinga finite-element formulation (left). Our underlying hierarchyadapts to the data points and to the automatic constraints(right).

(smoothness of Z, closeness of P s to Z). We look for a solution that discon-nects P in and P out while balancing the smoothness of Z and its closenessto P s.

5.3.1. Continuous optimization on implicit representation. Wefirst project the problem into a higher dimension; instead of searching forZ directly, we construct a continuous implicit function u(p), p ∈ Ω definedover some domain Ω ⊂ R3, and define Z = u−1(0) to be its zero level-set(see Figure 5.3.1 (left)). We translate the objectives defined with respectto Z onto the function u, by requiring it to be smooth (generically leadingto a smooth Z) and to satisfy :

u(p) ≈ 0 for p ∈ P s(5.3.1)

u(p) > 0 for p ∈ P in(5.3.2)

u(p) < 0 for p ∈ P out(5.3.3)

Unless the gradient of u near its zero level-set is small, the points of P s areclose to Z. In our interactive system, the user can easily add points to P in

and P out to stabilize Z (see Figure 5.3.2). In the next section, we explainhow to detect areas where more constraints are needed.

The method replaces the constraints (5.3.2) and (5.3.3) by specifyingtarget values at P in and P out to remove the linear inequality constraints,and hence amenable to optimization methods. Thus, the three criteria onu can be combined into a single continuous score function Ψ, which assignsa penalty to each candidate. The score Ψ(u) is a sum of two scores, onethat penalizes u for not being smooth and one that penalizes it for beingfar from zero, positive and negative target values at P s, P in and P out.


Figure 5.3.2. dding scribbles to constrain weak regionsachieves the expected topology in the Hummingbird head:from the data points and initial automatic constraints (left),a scribble is drawn over the initial FEM field (center) to cor-rect the local topology (right).

Figure 5.3.3. A 2D example of the FEM solution on twodifferent underlying grids: the smoothness penalty is inde-pendent of the mesh, adapting automatically to different el-ement sizes and aspect ratio.

5.3.2. Penalty functions. We define the non-smoothness penalty forany function u that is differentiable almost everywhere in Ω by:

(5.3.4) Ψsmoothness(u) =1

2

˚Ω

[(∂u∂x

)2 + (∂u∂y

)2 + (∂u∂z

)2]dx dy dz .

This penalty has a physical interpretation [144, Sec. 3.3]: it is the amountof work that is used to maintain state u in a uniform isotropic diffusiveproblem (heat conduction, electrostatics) with no forcing terms. If we addproper forcing terms on the boundary of Ω, the steady-state solution of thisphysical problem minimizes this penalty function and satisfies the Laplaceequation.

The penalty function Ψpoint constraints, penalizes u for being far from thepoint constraints. The target values t(p) are defined by t(p) = 0 for p ∈ P s

and heuristically setting t(p) = −d(p) (resp. t(p) = +d(p)) for p ∈ P in (resp.p ∈ P out), where d(p) is the distance to the nearest data point :

Ψpoint constraints(u) =∑p∈P

(u(p)− t(p))2 ,

where P = P s ∪ P in ∪ P out.


5.3.3. Finite-Element Method. The set of admissible functions uis defined by interpolation using a finite element mesh M that partitionsΩ [86, Chap. III]. We denote the values of u at the vertices of M by thevector uM .

We use the finite-element method to construct a matrix K such that:

(5.3.5) Ψsmoothness(u) = uTMKuM .

Thus, in our method, the mesh defines only the family of admissible func-tions u. The non-smoothness penalty Ψsmoothness(u) of a particular functionis completely independent of the mesh (see Figure 5.3.3), since it is intrin-sically defined using the mesh-free expression (5.3.4).

In this context, the finite-element method is essentially a recipe forconstructing a matrix K given an integral expression such as (5.3.4) and afinite-dimensional set of admissible functions, such that K satisfies (5.3.5).The recipe is based on computing the integral (5.3.4) on a set of basisfunctions that span the space of admissible functions. Here, we use atetrahedral mesh and define the admissible functions as piecewise tri-linear,interpolated from vertices’ values on each tetrahedron of the mesh. Thematrix K is sparse and easy to compute tetrahedron by tetrahedron.

The construction of K is a standard finite-elements one, based on [86]but simplified to our particular case and made completely explicit. Let(xi, yi, zi)ni=1 be the coordinates of the vertices of the mesh and let

(n(1)j , n

(2)j , n

(3)j , n

(4)j )mj=1

be the vertices’ indices of the tetrahedrons in the mesh. For each tetrahe-dron (linear tetrahedral element) we compute the 4-by-4 matrix:

Kj =| det(Jj)|

6E∗jEj , where

Jj =

xn

(1)j− x

n(4)j

xn

(2)j− x

n(4)j

xn

(3)j− x

n(4)j

yn

(1)j− y

n(4)j

yn

(2)j− y

n(4)j

yn

(3)j− y

n(4)j

zn

(1)j− z

n(4)j

zn

(2)j− z

n(4)j

zn

(3)j− z

n(4)j

,

and Ej is the 3-by-4 matrix solution of the following linear system:

J∗jEj =

1 0 0 −10 1 0 −10 0 1 −1

.

We define Kj to be an n-by-n symmetric matrix which is zero except for

rows and columns n(1)j , n

(2)j , n

(3)j , n

(4)j , where Kj(n

(a)j , n

(b)j ) = Kj(a, b). Fi-

nally, we sum the Kj to generate K =∑m

j=1 Kj.The literature contains many methods, often referred to as Laplacian

approximations, for constructing similar matrices from 2D meshes. Themain advantage of our FEM-based approach is that the smoothness penalty

5.4. Technical Details 141

formulation is independent of the mesh, since the left-hand side of Equa-tion (5.3.5) is mesh free. Therefore it extends naturally to 3D meshes.Our method automatically adapts to functions interpolated on meshes withtetrahedrons of widely different size and aspect ratio, common in our datastructure (see Figure 5.3.3).

We sum Ψsmoothness and Ψpoint constraints to form a single least squaresoptimization problem. For each constraint point p ∈ P , we define cp suchthat cp uM = u(p). The row cp represents a linear interpolation operatoron M . It has at most four non-zero values for a tetrahedral mesh. Theconstraint for point p now writes cp uM = t(p), and we weight this constraintwith weight ωp. The smoothness constraint can be incorporated to the leastsquares formulation using any matrix E such that ETE = K as follows:

minuM

∥∥∥∥∥∥∥∥∥∥

E

ω1 c1

ω2 c2...

ω|P | c|P |

uM −

0ω1 t(p1)ω2 t(p2)

...ω|P | t(p|P |)

∥∥∥∥∥∥∥∥∥∥

2

2

,

The matrix E is never computed explicitly. Instead, we solve the least-squares problem using its normal equation:

(5.3.6)

(K +

∑p∈P

ω2pcTp cp

)uM =

∑p∈P

t(p) ω2pcTp .

We construct the coefficient matrix of this linear system of equations byconstructing K using the finite-element method, and then adding to it thesparse matrices cTi ci. We solve using a fast sparse Cholesky factorization(cholmod [48]).

5.3.4. Adding and removing constraints. One aspect of this nu-merical approach allows real-time interaction in our surface reconstructionapplication. The structure of Equation (5.3.6) allows the method to in-crementally update the linear system factorization when the user adds orremoves inside/outside constraints. We compute the additional rows cp andtarget values t(p) for each new point p, and add the new cTp cp matrices and

cTp t(p) vectors to left and right hand side of Equation (5.3.6). To updatethe sparse Cholesky factorization we use [48]. Thus, the system factorizesthe initial matrix only once. In most cases, updating this factorization isfaster than factoring again.

5.4. Technical Details

The scanned data is given by a set of range images, where each datapoint is associated with the scanner position. The range images are reg-istered [103] to form a point cloud, which is structured by a hierarchicaldomain that tessellates the 3D space.

5.4. Technical Details 142

Figure 5.3.4. Reconstruction pipeline for a scanned ele-phant model. An initial coarse function is automatically com-puted from loose, automatic inside/outside (blue/red balls)constraints (left). The topological analysis of this functionselects weak regions (center-left), where the user can makelocal decisions by scribbling over 2D tablets at a coarse res-olution (center-right). Further iterations at finer resolutionslead to a complete reconstruction of the model (right).

We use a tetrahedral mesh, which is generated from the dual of a dy-namically adapted octree. This dual allows a better representation of con-tinuous functions, as required by the FEM and the topological analysis.Each cell of the dual is triangulated with a static pattern of 6-tetrahedronscube decomposition. To save memory, we keep only the edge graph of thetetrahedrons. A 2D illustration of this data structure is shown in Fig-ure 5.3.1 (right). The initial octree is adapted just to separate the datapoints, restricting its maximum depth to five. Next, following user vali-dation of weak regions, the octree is iteratively refined close to the zerolevel-set of the implicit function until reaching a visually-validated localfeature size.

To initiate the reconstruction process, we use few, automatically gener-ated, loose inside/outside constraints. The location and the sign of theseinitial automatic constraints can be computed heuristically. In this workwe simply compute an unsigned distance transform from the data points byfast marching over the hierarchy. We use local maxima, detected by exam-ining field gradients in the neighborhood of an octree cell, as automatic con-strains (in Figure 5.3.1 (left) automatic constraints were extracted from the2D medial-axis for illustration purposes). We automatically classify con-straints as inside/outside applying a simplified space carving method [43].

The full paper [136] contains additional technical details on the initialreconstruction, including a comparison to other automatic reconstructiontechniques. It also contains a detailed description of the user interaction,and the details of the final surface reconstruction.


5.5. Results

To demonstrate the effectiveness of the proposed method, we focusedon complex objects, such as the riding monk (Figure 5.1.1), which we ac-quired with few structured light scans. This relatively inexpensive technol-ogy has the advantage of being fast: each model was scanned from 6 to12 shots in less than five minutes. Our experiments exhibit the difficultiesthat are common to various scanning techniques: uncoverable areas as theelephant’s trunk in Figure 5.1.2 and the camel legs in Figure 5.6.3; high-lights in the sitting woman in Figure 5.6.4; object’s material as the ridingmonk’s black areas in Figure 5.1.1; and shadows in the tiger’s head in Fig-ure 5.6.2. For illustration purposes, we rendered the unoriented raw scansusing normals computed from our reconstruction. For a demonstration ofthe interactivity of our system, we refer to the accompanying video.

To further analyze the specific characteristics of our technique, we con-sider feature-specific models, generated by an openGL-based virtual scan-ner. As we can see in the triple Mobius band (Figure 5.5.1), our FEMformulation yields a smooth field everywhere, even for thin shapes. Fur-thermore, our topological analysis detects all the weak regions to allowa complete reconstruction, even on models as complex as the knot (Fig-ure 5.3.4). We also demonstrate the effectiveness of the interaction onvirtual scans of real models such as the hip (Figure 5.6.1).

Our method can handle input data with very large missing parts andoutliers. The smoothness constraint of the FEM distinguishes gracefullybetween frontiers of under-sampled regions and outliers as in the tiger’shead (Figure 5.6.2). In such cases, the smoothness criteria of the FEM,smoothly completes the missing parts and discards outliers, resulting ina watertight manifold surface. In similar noisy situations, the topologicalanalysis accurately detects weak regions such as the arm-leg contacts ofthe woman in Figure 5.6.4. Furthermore, user’s scribble constraints cangenerate coherent geometries in under-sampled regions such as the camel’slegs (Figure 5.6.3), while guaranteeing watertightness and avoiding spuriousconnected components.

As a limitation, this work focuses only on the topological aspects ofthe reconstruction and not specifically on its geometry. The constraintswe associate to the scribbles correct the local topology of the shape. Ourscribbles merely indicate whether a region is inside or outside the shape.Therefore, this mechanism does not precisely define the geometry of theresulting shape. In particular, it completes smoothly large missing parts(such as the woman’s shoulder in Figure 5.6.4). Thus, the method does notdeal with important issues such as the reconstruction of sharp features.

5.6. Conclusions

To conclude, we presented an interactive tool for surface reconstruc-tion, where the user assists the interpretation of data at automatically


Figure 5.5.1. The smoothness penalty of the FEM formu-lation yields a field that is smooth everywhere, even for thinshapes.

Figure 5.6.1. The addition of scribbles in the weak regionsallows a faithful reconstruction: the original scan (left), ourFEM reconstruction without scribble (center-left) and withfour scribbles to reinforce and correct the initial reconstruc-tion (right).

detected topologically unstable regions. The use of inside/outside scribblesat these regions allows achieving the expected topology of the object. Thereconstructed surface is faithful to the data points and guaranteed to bewatertight even with noisy under-sampled data . Our experiments showthat from only a dozen of structured light shots, we are able to reconstructcoherent models. In the future, we want to be able to guide the geometryof the reconstructed surface in the missing areas. This can be done in thesame framework, by adding other semantics to scribbles, such as specifyingadditional data points, associating sharp features or smoothness.


Figure 5.6.2. The smoothness penalty discards outliersand completes smoothly missing parts, as can be seen in thetiger’s head. From left to right: the original statue, the rawdata points (the missing parts are in black), the FEM fieldwith two close-ups on outliers and large missing parts in thehead, and our interactive reconstruction of the model with12 scribbles.

Figure 5.6.3. Due to the statue’s base (left), the camel’slegs cannot be covered (center-left). Scribble constraintsdrawn on automatically generated 2D tablets (center-right)can be used to generate a coherent geometry (right).

Figure 5.6.4. The topological analysis detects weak regions(center-right) in the presence of noise. Weak regions are typ-ically located where two parts of the shape are close and thetopology is ambiguous, such as the arm-leg contacts in thesitting woman. Missing parts with unambiguous topology,such as her right shoulder, are automatically smoothly com-pleted.

CHAPTER 6

Partitioned Triangular Tridiagonalization

6.1. Introduction

This chapter1 presents a partitioned algorithm for reducing a squaresymmetric matrix to a tridiagonal form. This algorithms is essentiallya partitioned formulation of the algorithm of Aasen [1] combined with anupdate step from the algorithm of Parlett and Reid [121]. This formulationeffectively exploits modern computer architecture through the use of thelevel-3 blas [50].

The Parlett-Reid and Aasen algorithms factor a symmetric matrix Ainto a product of a unit lower triangular matrix L, a symmetric tridiagonalmatrix T , and the transpose of L:

A = LTLT .

When partial pivoting is employed, these algorithms compute a factoriza-tion PAP T = LTLT where P is a permutation matrix. Asymptotically,the Parlett-Reid algorithm performs 2n3/3 +O(n2) arithmetic operations.Aasen’s method performs only n3/3 +O(n2) arithmetic operations, so it ismuch more efficient.

This factorization is one of the two main methods for factoring densesymmetric matrices. The other method, due to Bunch and Kaufman,factors the matrix into a product A = LBLT where L is unit lower tri-angular and B is block diagonal with 1-by-1 and 2-by-2 blocks. Themethod of Bunch and Kaufman also performs n3/3 + O(n2) operations.The performance of straightforward implementations of Bunch and Kauf-man’s method is similar to that of straightforward implementations ofAasen (see [15] for old experimental evidence; there is no reason that theresults should be different on a more modern machine).

Aasen’s algorithm is component-wise backward stable [81]. The Bunch-Kaufman method is only norm-wise backward stable. It can be shown ([8])that the values of its triangular factors can grow and lead to accuracyproblems. A variant of this method called Bounded Bunch-Kaufman was

1The results in this chapter are contained in a paper that was submitted for publicationin ACM Transactions on Mathematical Software, co-authored by Miroslav Rozloznıkand Sivan Toledo [131].The journal paper establishes the numerical stability of our partitioned triangular tridi-agonalization method. This result is due to Miroslav Rozloznık and does not appear inthis chapter. The rest of the results are mine.

146

6.2. A Variant of Aasen’s Method 147

developed to solve this problem. This variant generates bounded triangularfactors.

The performance of dense matrix algorithms improves dramaticallywhen the ordering of operations is altered so as to cluster together op-erations on small blocks of the matrix. This is usually called partitioningor blocking (e.g., see [20] and [54]). The blocked Bunch-Kaufman algorithmis described in [51]. A study by Anderson and Dongarra compared the per-formance of blocked versions of the Bunch-Kaufman and Aasen methodson the Cray 2 and found that Aasen’s method did not benefit from blockingas much as the Bunch-Kaufman method [7]. This finding appears to bethe main reason that the Bunch-Kaufman method is used in the symmetriclinear solver in lapack [6] rather than Aasen’s method. The details of theblocked algorithms are not given in the report by Anderson and Dongarra;the code appears to have been lost [8].

The implementation of our partitioned algorithm achieve similar perfor-mance to that of the blocked Bunch-Kaufman code in lapack. Our algo-rithm performs partial pivoting and its storage requirements are similar tothose of lapack’s implementation of Bunch-Kaufman. Our findings showthat there is no performance-related reason to prefer the Bunch-Kaufmanmethod over that of Aasen. We cannot determine whether the opposite con-clusion of Anderson and Dongarra resulted from the particular way theyblocked the algorithm or whether it is specific to the Cray 2 architecture.

In [131] we present stability analysis of our new algorithm and solu-tion routines. We show that our partitioned algorithm is as stable as thestraightforward column-by-column original algorithm, with only a slightchange in the backward stability constant. We also show that our solutionroutine, which is based on QR factorization of T is backward stable.

This chapter is organized as follows. Section 6.2 presents a variant ofAasen’s algorithm. Section 6.3 shows our partitioned formulation. Sec-tion 6.4 discuss how to efficiently implement our partitioned formulation.Section 6.5 presents experimental results. We present our conclusions inSection 6.6.

6.2. A Variant of Aasen’s Method

We start by describing a variant of Aasen’s method. The variant wederive here is slightly different from the original algorithm in [1] and thealgorithm described in [81, 82]. This variant is a building block for ourpartitioned algorithm.

Let A be an n-by-n real matrix. Let

PAP T = LTLT ,

be its factorization, where L is a unit lower triangular matrix, T is a sym-metric tridiagonal matrix, and P is a permutation matrix.

Let H = LT . Clearly, H is a lower-Hessenberg matrix and PAP T =HLT . See Figure 6.2.1 for an illustration of these matrices. The matrix H

6.2. A Variant of Aasen’s Method 148

Figure 6.2.1. An illustration of the factorization. Filledgray circles represent known values and empty circles repre-sent values that are computed during the factorization. Thelines show the partitioning of the matrices into blocks.

has a central role in the formulation of Aasen’s method. In our variant wealso emit H as the output of the factorization. This makes it inefficient interms of space, but this will not be a problem in the specific way we use itin our partitioned algorithm.

For clarity of the description, we first ignore pivoting, and later showhow to introduce it into the computation. The factorization advances col-umn by column. At step i we have already the first i columns of L, thefirst (i− 1) columns of T , and the first (i− 1) columns of H. The outputof this step is the ith column of H and T , and the (i+ 1)th column of L.

Assume we have completed step (i−1), we now show how to advance ina single step. First we recover Hi:n,i. We examine coordinates i to n in theith column in the equation A = HLT , and get Ai:n,i = Hi:n,1:nL

Ti,1:n. Since

L is unit triangular, we have Ai:n,i = Hi:n,1:(i−1)LTi,1:(i−1) +Hi:n,i. Therefore,

we can set Hi:n,i = Ai:n,i − Hi:n,1:(i−1)LTi,1:(i−1) (or simply Hi:n,i = Ai:n,i if

i = 1).Next we recover Ti−1,i and Hi−1,i. If i > 1, and since T is symmetric

we can set Ti−1,i = Ti,i−1. If i > 1, using the coordinate (i − 1, i) in theequation H = LT , and since Li−1,i−1 = 1, we get Hi−1,i = Ti−1,i. We nowuse the same equation H = LT to compute the rest of T:,i and L:,i+1. Wetake a look at coordinates i to n in the ith column of this equation. We get

Hi:n,i = Li:n,i−1Ti−1,i + Li:n,i:nTi:n,i .

We let u = Li:n,i:nTi:n,i, and get

u = Hi:n,i − Li:n,i−1Ti−1,i .

The vector u can be computed since all the parts of the last equation areknown (if i = 1, u is simply Hi:n,i). Moreover, since Li,i = 1, Ti,i = u1.In order to complete this step, if i < n, we need to compute Ti+1,i, andL(i+1):n,i+1. They can both be extracted from the known u. We focus on

6.3. Our Partitioned Method 149

u2:end. By definition,

u2:end = L(i+1)::n,i:nTi:n,i

= L(i+1):n,iTi,i + L(i+1):n,i+1Ti+1,i .

We define v = L(i+1):n,i+1Ti+1,i. By the last equation, v = u2:end −

L(i+1):n,iTi,i, and can be fully calculated. Ignoring pivoting for a moment,we can set Ti+1,i = v1, since Li+1,i+1 = 1. Now, L(i+2):n,i+1 = v

2:end/v1.This completes the formulation without pivoting.

Without pivoting, this construction may fail in one specific point. Whencomputing L(i+2):n,i+1, v1 might be zero. Moreover, a very small v1 mightcause numerical problems. This can be solved easily, by permuting v,such that v1 is its element with the highest magnitude. (If v is all ze-ros, L(i+2):n,i+1 can be set to zero.) If such a permutation is performed, theappropriate preceding rows of L and H should be permuted accordingly,and the trailing submatrix of A should also be symmetrically permuted.Algorithm 1 summarizes our variant of Aasen’s method.

6.3. Our Partitioned Method

We now derive our partitioned algorithm. We again start our descrip-tion with pivoting ignored. Let 1 ≤ k < n. We assume that the first kcolumns of H, T and the first (k+ 1) columns of L are computed using ourAasen variant from the previous section. The core idea is that instead ofadvancing with the factorization in the way that is described in the previ-ous section, we stop, update the trailing submatrix of A and continue towork on the trailing submatrix as-if it was a new matrix. This is similar tothe concept of the Parlett-Reid algorithm [121]; at each step, the trailingsubmatrix is symmetric and ready to be factored as if it was a new matrix.Moreover, when setting a partition size of 1, our algorithm becomes exactlythe Parlett-Reid algorithm.

For convenience of presentation, we mark

A[11] = A1:k,1:k ,

A[12] = A1:k,(k+1):n ,

A[21] = A(k+1):n,1:k , and

A[22] = A(k+1):n,(k+1):n .

(and similarly for L, T , and H). We examine the trailing submatrix partin the equation A = HLT ; we have

A[22] = H(k+1):n,1:nLT(k+1):n,1:n

= H [21] L[21] T +H [22] L[22] T .

Figure 6.3.1 illustrates this equation. We have already computed A[22],

H [21], L[21]. Therefore, we can compute the product H [22] L[22] T . We now


Algorithm 1 Aasen factorization of the symmetric matrix A. This routinecan also receive the first column of L as an input, and also the number kof columns to handle. To get the standard factorization PAP T = LTLT itshould be activated with Aasen(A,e1,n), where n is the order of A and e1

is a unit vector of order n. These extra parameters are useful when usedas a subroutine for the partitioned factorization.The routine also emits the permutation matrix P in two ways. As a matrixP and as a list p representing (up-to) n switches.

[L,T ,P ,H,p] ←Aasen(A,`,k)

n← order of A.L← In . n-by-n identity matrixH ← On . n-by-n zero matrix, stored as a whole for clarity onlyT ← On . n-by-n zero matrix, stored as a whole for clarity onlyp← n. Empty array of order nL1:n,1 ← `

For i = 1 . . . k doHi:n,i ← Ai:n,i −Hi:n,1:(i−1)L

Ti,1:(i−1)

u← Hi:n,i

If (i > 1)u← u− Li:n,(i−1)T(i−1),i

T(i−1),i ← Ti,(i−1)

H(i−1),i ← Ti,(i−1) .This value is not really needed in practiceEnd IfTi,i ← u1

If (i < n)v ← u2:end − Ti,iL(i+1):n,i

If (max(|v|) 6= 0) and (argmax(|v|) 6= 1)Switch rows and columns (i+ 1) and (i+ argmax(|v|)) in A(i+1):n,(i+1):n

Switch rows (i+ 1) and (i+ argmax(|v|)) in H:,1:i and L:,1:i

Switch values in indices 1 and argmax(|v|) in v

p(i+ 1)← i+ argmax(|v|)End IfT(i+1),i ← v1

If (v1 6= 0)L(i+1):n,(i+1) ← v/T(i+1),i

End IfEnd If

End For. Creating an explicit PP ← In . n-by-n identity matrixFor i = 1 . . . n

Switch rows p(i) and i in P

End For


Figure 6.3.1. Expansion of A[k,22] in the partitioned algorithm.

explore its structure; we first focus on H [22]. We use the equation H = LT ,and get

H [22] = L(k+1):n,1:nT1:n,(k+1):n

= L[21]T [12] + L[22]T [22] .

We multiply it with L[22] T , and get

H [22] L[22] T = L[21]T [12] L[22] T + L[22]T [22] L[22] T

= L[21]:,k T

[12]k,: L[22] T + L[22]T [22] L[22] T

= L[21]:,k T

[12]k,:1 L

[22]1,:

T+ L[22]T [22] L[22] T .

The transition from the first to the second line is based on an expansionof L[21]T [12] into a sum of column-times-row outer products, all of whichexcept the first are zero. The transition from the second to the third line is

based on a similar trick: L[21]:,k T

[12]k,: is a square matrix in which only the first

column, L[21]:,k T

[12]k,1 , is not zero. Figure 6.3.2 illustrates these steps. Now, we

can compute L[21]:,k T

[12]k,: L[22] T simply by multiplying L

[21]:,k T

[12]k,1 by the first

row of L[22] T . The first row of L[22] T is exactly column (k + 1) in L whichwe already have. Therefore, we can compute the following expression

L[22]T [22] L[22] T = H [22] L[22] T − L[21]:,k T

[12]k,:1 L

[22]1,:

T

= A[22] −H [21] L[21] T − L[21]:,k T

[12]k,:1 L

[22]1,:

T.


Figure 6.3.2. Expansion of H [22] L[22] T in the partitioned algorithm.

This computation comprises of a rank-k and a rank-1 update. The matrix

L[22]T [22] L[22] T is an (n− k)-by-(n− k) symmetric matrix, it can be tridi-agonalized with the first column set to the (already computed) L(k+1):n,k+1.The resulting factors are the required (n− k) columns of T and (n− k− 1)columns of L.

This reduction step can now applied k columns at a time until nocolumns are left. At each step we use our Aasen variant to generate thenext k columns. Then, we use them to update the trailing submatrix, andcontinue to work on the trailing submatrix.

6.4. Implementation 153

We now address the pivoting issue. The output of the Aasen factoriza-tion at each step is not only the next k columns of H, T and L, but also apermutation matrix P representing the row and column switches that wereperformed. We perform the exact row and column switches on A beforeapplying the update. This ensures that the update is consistent and we cancontinue. We also store those updates to generate the final matrix P andupdate the previously computed columns of L. Algorithm 2 summarizesthe entire partitioned process with pivoting.

6.4. Implementation

In the algorithm description so far, we have updated the entire trailingsubmatrix. This is redundant and leads to an inefficient implementation.In practice, there is no need to update the entire trailing submatrix. Thisis because the trailing submatrix is always symmetric, and that Aasen’salgorithm only uses the lower triangular part of its input matrix A. Withonly half a matrix represented, the row and column interchanges shouldcarefully performed. The key idea is that line j in a symmetric matrixA with only lower half stored, is stored in two parts: Aj,1:j and AT(j+1):n,j.Using this idea, our implementation only stores and updates half a matrix.

In order to perform the majority of the update with BLAS-3, we per-form the following changes in the algorithm that we described so far. First,

we fuse the rank-k update with H [k,21] L[k,21] T with the rank-1 update

L[k,21]:,k T

[k,12]k,:1 L

[k,22]1,:

T, to create a single rank-(k + 1) update. Let UV T be

the fused update to the trailing m-by-m matrix B, where U is m-by-(k+1)and V is m-by-(k + 1). The second change, is the specific way we up-date only half a matrix. We update the trailing submatrix in panels ofwidth k. At the first step we subtract UV T

1:k,1:(k+1) from B1:m,1:k, at the

second step we subtract U (k+1):m,1:(k+1)VT

(k+1):2k,1:(k+1) from B(k+1)::m,(k+1):2k

and so on. This means that we update redundant k-by-k upper triangularblocks. Nevertheless, it keeps the most of the update process computationin BLAS-3.

In our implementation, in order to avoid some memory copies, the up-date process is not all in BLAS-3. We perform the following slight change.We update the first column of B using a BLAS-2 update UV T

1,1:(k+1), andthen update the rest of B using panels of width k. This is due to someimplementation choices and is not inherent in our algorithm.

We are now ready to compute the time complexity of our algorithm.The Aasen factorization of k columns of order n takes O(nk2) operations sothe total n/k such calls cost O(n2k) operations. There are at most O(n) rowand column exchange operations per column that is factored, so the totalis O(n2). Therefore, not including the update there are O(n2k) operations.The update of a trailing submatrix of size m costs (m2 + mk)(k + 1) +2m(k+1), out of which 2m(k+1) is outside BLAS-3. Careful computation

6.4. Implementation 154

Algorithm 2 Our partitioned algorithm top-level routine. k is the pre-scribed partition size.

[L,T,P] ← PartitionedTridiagonalReduction(A,k)

n← order of AL← In . n-by-n identity matrixT ← On . n-by-n zero matrix, stored as a whole for clarity onlyp← [1, . . . , n] . Array of order n, used to keep track of interchanges`← L:,1 . The prescribed next column of Li← 0 .Keeps the number of columns of A that were already processedWhile (i < n)k ← min(order of A, k) .Block size is k except (possibly) the last block[L(k),T (k),P (k),H(k), q] ← Aasen(A,`,k) . Aasen on the first k columns..Place the results into L and T amL(i+1):n,(i+1):(i+k) ← L

(k):,1:k

T(i+1):(i+k),(i+1):(i+k) ← T(k)1:k,1:k

If ((i+ k) < n)

`← L(k)

(k+1):end,(k+1)

T(i+k),(i+k+1) ← T(k)(k+1),k

T(i+k+1),(i+k) ← T(i+k),(i+k+1)

End If.Make the appropriate row and column exchanges dictated by the AasenFor j ← 2 : min(k + 1, order of A)

If (j 6= q(j))Switch rows and columns j and q(j) in A

Switch rows j and q(j) in L:,1:i

Switch array cells (j + i) and (q(j) + i) in p

End IfEnd Fori← i+ k

If ((i+ 1) ≤ n) . Update the trailing submatrixA(k+1):end,(k+1):end ← A(k+1):end,(k+1):end

−H(k)

(k+1):end,1:kL

(k)

(k+1):end,1:k

T

−L(k)

(k+1):end,kT(i+1),i L(k)

(k+1):end,(k+1)

T

End IfA← A(k+1):end,(k+1):end . Replace A with its updated trailing submatrix

End While. Form the global PP ← On

For i = 1 . . . nP (i, p(i))← 1

End For


reveals that the total running time is

1

3

(1 +

1

k

)n3 +O(n2k) ,

out of which only O(n2k) are outside BLAS-3.The next aspect of the implementation that we need to discuss is the

storage requirements. In order to perform the Aasen factorization for kcolumns, and to store the part of H that is needed for the update, we storean extra buffer of size kn. The rest of the computation can be performedin-place. The computed columns of L can be stored in the place of A, thediagonal of L is known, therefore we do not have to save it. We do notstore T , but factor it into its QR factors using Givens rotations, during thefactorization. The factors are stored in unused diagonals in the upper partof A. We also use 2 additional temporary arrays of size n. The total extrastorage needed is therefore (k + 3)n which is similar to the working bufferneeded for the blocked Bunch-Kaufman routine in lapack.

Our choice of QR and the fact that we fused the factorization of Tinto the global factorization are not mandatory. We could have also usedLU with partial pivoting; both preserve the sparsity of a tridiagonal matrixalmost completely (in both cases the upper triangular matrix has 3 nonzerodiagonals; in LU the unit lower triangular matrix has only two nonzerodiagonals, and in QR we need to store n − 1 rotations). We chose to fusethe factorization in order to demonstrate that solving with T , which isslightly harder than solving with the block diagonal D in Bunch-Kaufmanfactorization, does not have a real effect on the performance.

We finish this section with a comment about row exchanges in L. Thereis no real need to exchange rows in preceding columns of L after a batchof new k columns of L is created. We do that in Algorithm 2 for clarity,in order to generate the complete L. In practice, as long as the solvingroutine is written accordingly, there is no need for that exchange.

6.5. Experimental Results

We have compared the performance of our partitioned tridiagonalizationmethod to the performance of the partitioned Bunch-Kaufman algorithmin lapack version 3.1.1 [6]. The results that we report below show that theperformance of the partitioned tridiagonalization method is similar to thatof the Bunch-Kaufman algorithm. The performance of the solver phases isalso similar.

We linked all the codes (ours and lapack) to the Goto blas ver-sion 1.12 [67]. The codes were compiled using gcc version 4.1.2 on anIntel x86 64 machine running Linux. The computer on which we ran theexperiments had an Intel Core 2 model 6400 running at 2.13 GHz. Werestricted the blas to use only one core of the CPU using an environmentvariable. The computer had 4 GB of main memory; we did not detect anysignificant paging activity during the experiments.


0 200 400 600 800 10000

1000

2000

3000

4000

5000Factorization Performance for small Matrix Size

MF

LOP

/s

N

LAPACK−3.1.1Our Method

0 5000 10000 150004000

4500

5000

5500

6000Factorization Performance for large Matrix Size

MF

LOP

/s

N

LAPACK−3.1.1Our Method

Figure 6.5.1. The performance of the two methods. Thevertical axis shows the effective computational rate.

We used a block size k = 64 both in our method and in lapack. Thisvalue is the default of lapack 3.1.1. In experiments not reported here, weverified that this value is indeed the optimal block size for this machine(and for all the methods).

The experiments were conducted using random symmetric matriceswhose elements are uniformly distributed in (−1, 1). The relative resid-uals were always below 10−12 and. The residuals produced by the twomethods were of similar magnitude in all the matrices.

Figure 6.5.1 compares the performance of the two methods. The ver-tical axis shows effective computational rates in millions of floating-pointoperations per second,

1

3

n3

T× 10−6 ,

where T is the running time in seconds. Note that this metric comparesactual running times and ignores differences in operation counts. There-fore, a higher value always indicates a faster factorization. We chose topresent computational rates and not running times in this experiment toadditionally demonstrate that we are gradually making better utilizationof the processor. This is similar to other BLAS-3 dominant computations(see [90, Figure 4], for example).

On matrices of order up to about 5000, the tridiagonalization method isslightly faster or run at the same speed as the Bunch-Kaufman algorithm.On larger matrices, the Bunch-Kaufman algorithm is sometimes faster. Ineither case, the relative performance differences are not large. We do notknow why the performance of the tridiagonalization algorithm drops atn = 8000.

The graphs in Figure 6.5.2 show that the performance of the solve phaseis essentially the same for our new methods and for lapack’s implemen-tation of the Bunch-Kaufman algorithm.

6.6. Concluding Remarks 157

0 1000 2000 3000 40000

50

100

150

200

Number of Right−Hand−Sides

Tim

e (S

econ

ds)

Solve Time, Fixed Matrix Orders

LAPACK−3.1.1 n=1000LAPACK−3.1.1 n=2000LAPACK−3.1.1 n=4000Our Method n=1000Our Method n=2000Our Method n=4000

0 2000 4000 6000 8000 100000

100

200

300

400

500

600

n

Tim

e (S

econ

ds)

Solve Time, fixed Numbers of Right−Hand−Sides (NRHS)

LAPACK−3.1.1 NRHS=100LAPACK−3.1.1 NRHS=500LAPACK−3.1.1 NRHS=2000Our Method NRHS=100Our Method NRHS=500Our Method NRHS=2000

Figure 6.5.2. The performance of the solve phase. Thegraphs of our method coincide with the graphs of the Bunch-Kaufman method in lapack.

6.6. Concluding Remarks

We have presented a symmetric tridiagonalization algorithm. The algo-rithm is partitioned, so it effectively exploits high-performance level-3 blasroutines. The algorithm performs almost the same number of operationsasymptotically as the column-by-column Aasen algorithm[1]. We have an-alyzed this issue theoretically and verified the results experimentally.

In [131] we also performed round-off analysis. We showed that ourfactorization is component-wise backward stable. Provided a conventionalBLAS is used, the only noticeable difference from the analysis of Aasen’soriginal algorithm is that we have an extra rank-1 update whenever wereduce a partition. Therefore, in this sense, the analysis is quite similar;numerically, our algorithm performs almost the same as the original one.

Our implementation of the algorithm performs similarly to lapack’sBunch-Kaufman, in both the factor and the solve phases. This showsthat the findings of Anderson and Dongarra, who found that a partitionedvariant of Aasen’s algorithm performed worse than a partitioned Bunch-Kaufman [7], are not valid for our new algorithm. Our experiments wereconducted on a very different machine than the one used by Anderson andDongarra, but we do not see why our algorithm would perform poorer thanlapack even on a machine like the Cray 2; our methods and lapack’simplementation of Bunch-Kaufman perform essentially the same sequenceof level-3 blas calls.

More generally, our results show that symmetric tridiagonalization meth-ods can be as efficient as symmetric block-diagonalization (when both uti-lize non-unitary transformations). The extra cost of solving tridiagonalmatrices is negligible. The triangular factors that our methods produce al-ways have elements bounded in magnitude by 1, whereas Bunch-Kaufmanmethods sometimes produce triangular factors with large elements [8].

Bibliography

[1] Aasen, J. O. On the reduction of a symmetric matrix to tridiagonal form. BIT11 (1971), 233–242.

[2] Ajiz, M. A., and Jennings, A. A robust incomplete Choleski-conjugate gradientalgorithm. International Journal for Numerical Methods in Engineering 20 (1984),949–966.

[3] Alon, N., and Milman, V. D. λ1, isoperimetric inequalities for graphs, andsuperconcentrators. J. Comb. Theory Series B 38 (1985), 73–88.

[4] Amestoy, P. R., Duff, I. S., Koster, J., and L’Excellent, J. A fully asyn-chronous multifrontal solver using distributed dynamic scheduling. SIAM Journalon Matrix Analysis and Applications 23 (2001), 15–41.

[5] Amestoy, P. R., Duff, I. S., L’Excellent, J., Koster, J., and Tuma,M. MUltifrontal Massively Parallel Solver (MUMPS version 4.1), specificationsheets. Available online from http://www.enseeiht.fr/lima/apo/MUMPS/doc.html, Mar. 2000.

[6] Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Don-garra, J., Croz, J. D., Greenbaum, A., Hammarling, S., McKenney,A., and Sorensen, D. LAPACK User’s Guide, 3rd ed. SIAM, Philadelphia, PA,1999. Also available online from http://www.netlib.org.

[7] Anderson, E., and Dongarra, J. Evaluating block algorithm variants in LA-PACK. In Proceedings of the 4th Conference on Parallel Processing for ScientificComputing (1989), J. Dongarra, P. Messina, D. C. Sorensen, and R. G. Voigt,Eds., SIAM, pp. 3–8.

[8] Ashcraft, C., Grimes, R. G., and Lewis, J. G. Accurate symmetric indefinitelinear equation solvers. SIAM J. Matrix Anal. Appl. 20, 2 (1998), 513–561.

[9] Avron, H., Chen, D., Shklarski, G., and Toledo, S. Combinatorial precon-ditioners for scalar elliptic finite-elements problems. To appear in SIAM Journalon Scientific Computing, Nov. 2006.

[10] Avron, H., Ng, E., and Toledo, S. Using perturbed QR factorizations to solvelinear least-squares problems. Submitted to SIAM Journal on Matrix Analysis andComputations, July 2007.

[11] Avron, H., Shklarski, G., and Toledo, S. On element SDD approximability.Tech. rep., Tel-Aviv University, Israel, Apr. 2008.

[12] Avron, H., Shklarski, G., and Toledo, S. Parallel unsymmetric-pattern mul-tifrontal sparse LU with column preordering. ACM Transactions on MathematicalSoftware 34, 2 (Mar. 2008), 8:1–8:31.

[13] Axelsson, O. A generalized SSOR. BIT 13 (1972), 443–467.[14] Axelsson, O. Conjugate gradient type methods for unsymmetric and inconsistent

systems of linear equations. Tech. Rep. 74-10, CERN, Geneva, 10 1974.[15] Barwell, V., and George, A. A comparison of algorithms for solving symmet-

ric indefinite systems of linear equations. ACM Trans. Math. Software 2 (1976),242–251.

[16] Bauer, F. L. Optimally scaled matrices. Numerische Mathematik 5 (1963), 73–87.

158

http://www.enseeiht.fr/lima/apo/MUMPS/doc.html

http://www.enseeiht.fr/lima/apo/MUMPS/doc.html

Bibiliography 159

[17] Benzi, M., Meyer, C. D., and Tuma, M. A sparse approximate inverse precon-ditioner for the conjugate gradient method. SIAM Journal on Scientific Computing17 (1996), 1135–1149.

[18] Bern, M., Gilbert, J. R., Hendrickson, B., Nguyen, N., and Toledo,S. Support-graph preconditioners. SIAM Journal on Matrix Analysis and Appli-cations 27 (2006), 930–951.

[19] Bernstein, D. S. Matrix Mathematics: Theory, Facts, and Formulas with Ap-plications to Linear Systems Theory. Princeton University Press, 2005.

[20] Bientinesi, P., Gunnels, J. A., Myers, M. E., Quintana-Orti, E., andvan de Geijn, R. The science of deriving dense linear algebra algorithms. ACMTransactions on Mathematical Software 31 (2005), 1–26.

[21] Blair, J. R. S., and Peyton, B. An introduction to chordal graphs and cliquetrees. In Graph Theory and Sparse Matrix Computation, A. George, J. R. Gilbert,and J. W. H. Liu, Eds. Springer-Verlag, 1993, pp. 1–29.

[22] Boman, E. G., Chen, D., Hendrickson, B., and Toledo, S. Maximum-weight-basis preconditioners. Numerical Linear Algebra with Applications 11(2004), 695–721.

[23] Boman, E. G., and Hendrickson, B. On spanning tree preconditioners. Un-published manuscript, Sandia National Laboratories, 2001.

[24] Boman, E. G., and Hendrickson, B. Support theory for preconditioning.SIAM Journal on Matrix Analysis and Applications 25, 3 (2004), 694–717.

[25] Boman, E. G., Hendrickson, B., and Vavasis, S. A. Solving elliptic fi-nite element systems in near-linear time with support preconditioners. CoRRcs.NA/0407022 (2004).

[26] Braatz, R. D. Response to Chen and Toledo on “minimizing the Euclideancondition number”. Private communication, Sept. 2005.

[27] Braatz, R. D., and Morari, M. Minimizing the Euclidean condition number.SIAM Journal on Control and Optimization 32 (1994), 1763–1768.

[28] Brainman, I., and Toledo, S. Nested-dissection orderings for sparse LU withpartial pivoting. SIAM Journal on Matrix Analysis and Applications 23 (2002),998–112.

[29] Brandt, A. Multilevel computations: review and recent development. In Multi-grid Methods, S. F. McCormick, Ed. Marcel Dekker, 1988, pp. 35–62.

[30] Brezina, M., Cleary, A. J., Falgout, R. D., Henson, V. E., Jones, J. E.,Manteuffel, T. A., McCormick, S. F., and Ruge, J. W. Algebraic multi-grid based on element interpolation (amge). SIAM Journal on Scientific Comput-ing 22, 5 (2000), 1570–1592.

[31] Bridson, R., and Tang, W.-P. Ordering, unisotropy, and factored sparse ap-proximate inverses. SIAM Journal on Scientific Computing 21, 3 (1999), 867–882.

[32] C. A. Felippa, K. C. P., and Filho, M. R. J. The construction of free-freeflexibility matrices as generalized stiffness inverses. Computers and Structures 68,4 (1998), 411–418.

[33] Carr, J., Beatson, R., Cherrie, J., Mitchell, T. J., Fright, W. R.,McCallum, B. C., and Evans, T. R. Reconstruction and representation of 3Dobjects with radial basis functions. In Siggraph (2001), ACM, pp. 67–76.

[34] Chan, T. F., and van der Vorst, H. A. Approximate and incomplete factor-izations. Tech. Rep. Preprint 871., Department of Mathematics, Utrecht Univer-sity, 1994.

[35] Chartier, T., Falgout, R. D., Henson, V. E., Jones, J., Manteuffel, T.,McCormick, S., Ruge, J., and Vassilevski, P. S. Spectral AMGe (ρAMGe).SIAM Journal on Scientific Computing 25, 1 (Jan. 2003), 1–26.

Bibiliography 160

[36] Cheeger, J. A lower bound for the smallest eigenvalue of the laplacian. In Prob-lems in analysis, R. C. Gunning, Ed. Princeton Univ. Press, 1970, pp. 195–199.

[37] Chen, D., and Toledo, S. Vaidya’s preconditioners: Implementation and exper-imental study. Electronic Transactions on Numerical Analysis 16 (2003), 30–49.

[38] Chen, D., and Toledo, S. Combinatorial characterization of the null spaces ofsymmetric H-matrices. Linear Algebra and its Applications 392 (2004), 71–90.

[39] Chow, E., and Saad, Y. Experimental study of ILU preconditioners for indef-inite matrices. Tech. Rep. UMSI 97/97, Supercomputer Institute, University ofMinnesota, June 1997.

[40] Coleman, and Pothen. The null space problem I. complexity. SIJADM: SIAMJournal on Algebraic and Discrete Methods 7 (1986).

[41] Concus, P., Golub, G. H., and O’Leary, D. P. A generalized conjugategradient method for the numerical solution of elliptic partial differential equations.In Sparse Matrix Computations, J. R. Bunch and D. J. Rose, Eds. Academic Press,New York, 1976, pp. 309–332.

[42] Cosgrove, J., Diaz, J., and Grienwank, A. Approximate inverse precondi-tioning for sparse linear systems. International Journal of Computational Mathe-matics 44 (1992), 91–110.

[43] Curless, B., and Levoy, M. A volumetric method for building complex modelsfrom range images. In Siggraph (1996), ACM, pp. 303–312.

[44] D. M. Day, M. K. Bhardwaj, G. M. R., and Peery, J. S. Mechanism freedomain decomposition. Computer Methods in Applied Mechanics and Engineering192 (2003), 763–776.

[45] Daitch, S. I., and Spielman, D. A. Support-graph preconditioners for 2-dimensional trusses. CoRR abs/cs/0703119 (2007). informal publication.

[46] Davis, T. A. Algorithm 832: UMFPACK V4.3 - an unsymmetric-pattern multi-frontal method. ACM Trans. Math. Softw 30, 2 (2004), 196–199.

[47] Davis, T. A. A column pre-ordering strategy for the unsymmetric-pattern mul-tifrontal method. ACM Trans. Math. Softw. 30, 2 (2004), 165–195.

[48] Davis, T. A., and Hager, W. W. Row modifications of a sparse Choleskyfactorization. SIAM Journal on Matrix Analysis and Applications 26, 3 (2005),621–639.

[49] Demmel, J. W., Gilbert, J. R., and Li, X. S. An asynchronous parallelsupernodal algorithm for sparse Gaussian elimination. SIAM Journal on MatrixAnalysis and Applications 20 (1999), 915–952.

[50] Dongarra, J. J., Cruz, J. D., Hammarling, S., and Duff, I. A set of level 3basic linear algebra subprograms. ACM Transactions on Mathematical Software16, 1 (1990), 1–17.

[51] Dongarra, J. J., Duff, I. S., Sorensen, D. C., and van der Vorst, H. A.Numerical Linear Algebra for High-Performance Computers. Society for Industrialand Applied Mathematics, Philadelphia, PA, USA, 1998.

[52] Druinsky, A., and Toledo, S. Factoring matrices with a tree-structured spar-sity pattern. Submitted to Electronic Transactions on Numerical Analysis, May2007.

[53] Elkin, M., Emek, Y., Spielman, D. A., and Teng, S.-H. Lower-stretchspanning trees. In Proceedings of the 37th annual ACM symposium on Theory ofcomputing (STOC) (Baltimore, MD, 2005), ACM Press, pp. 494–503.

[54] Elmroth, E., Gustavson, F., Jonsson, I., and Kagstrom, B. Recursiveblocked algorithms and hybrid data structures for dense matrix library software.SIAM Review 46, 1 (Mar. 2004), 3–45.

Bibiliography 161

[55] Farhat, C., and Gerardin, M. On the general solution by a direct methodof a large scale singular system of linear equations: application to the analysis offloating structures. Internat. J. Numer. Methods Engrg. 41 (1997), 675–696.

[56] Farhat, C., Pierson, K. H., and Lesoinne, M. The second generation ofFETI methods and their application to the parallel solution of large-scale linearand geometrically nonlinear structural analysis problems. Computer Methods inApplied Mechanics and Engineering 184 (2000), 333–374.

[57] Farhat, C., and Roux, F.-X. A method of Finite Element Tearing and Inter-connecting and its parallel solution algorithm. Int. J. Numer. Meth. Engrg. 32(1991), 1205–1227.

[58] Felippa, C. A., and Park, K. C. A direct flexibility method. Computer Methodsin Applied Mechanics and Engineering 149 (1997), 319–337.

[59] Fiedler, M. Algebraic connectivity of graphs. Czech. Math. Journal 23, 98(1973), 298–305.

[60] Frangioni, A., and Gentile, C. New preconditioners for KKT systems ofnetwork flow problems. SIAM Journal on Optimization 14 (2004), 894–913.

[61] George, A., and Ng, E. On the complexity of sparse QR and LU factorizationon finite-element matrices. SIAM Journal on Scientific and Statistical Computa-tion 9 (1988), 849–861.

[62] Geradin, M., and Rixen, D. Mechanical Vibrations: Theory and Applicationto Structural Dynamics. Wiley and Sons, 1997.

[63] Gilbert, and Heath. Computing a sparse basis for the null space. SIJADM:SIAM Journal on Algebraic and Discrete Methods 8 (1987).

[64] Gilbert, J. R., and Tarjan, R. E. The analysis of a nested dissection algo-rithm. Numerische Mathematik 50 (1987), 377–404.

[65] Gilbert, J. R., and Toledo, S. An assessment of incomplete-lu preconditionersfor nonsymmetric linear systems. Informatica 24 (2000), 409–425.

[66] Golub, G. H., and Loan, C. F. V. Matrix Computations, 2nd ed. Johns Hop-kins University Press, 1989.

[67] Goto, K., and van de Geijn, R. High-performance implementation of the level-3 BLAS. Tech. Rep. CS-TR-06-23, The University of Texas at Austin, Departmentof Computer Sciences, May 5 2006.

[68] Gotsman, C., and Toledo, S. On the computation of null spaces of sparserectangular matrices. To appear in SIAM Journal on Matrix Analysis and Appli-cations, Aug. 2005.

[69] Gremban, K., Miller, G., and Zagha, M. Performance evaluation of a par-allel preconditioner. In 9th International Parallel Processing Symposium (SantaBarbara, April 1995), IEEE, pp. 65–69.

[70] Gremban, K. D. Combinatorial Preconditioners for Sparse, Symmetric, Diago-nally Dominant Linear Systems. PhD thesis, School of Computer Science, CarnegieMellon University, Oct. 1996. Technical Report CMU-CS-96-123.

[71] Grote, M. J., and Huckle, T. Parallel preconditioning with sparse approxi-mate inverses. SIAM Journal on Scientific Computing 18, 3 (1997), 838–853.

[72] Guattery, S. Graph embedding techniques for bounding condition numbers ofincomplete factor preconditioners. Tech. Rep. ICASE Report 97-47, NASA LangleyResearch Center, 1997.

[73] Gupta, A., Karypis, G., and Kumar, V. Highly scalable parallel algorithmsfor sparse matrix factorization. IEEE Transactions on Parallel and DistributedSystems 8, 5 (1997), 502–520.

[74] Gupta, R. Support graph preconditioners for linear systems. Master’s thesis,Texas A&M University, December 2004.

Bibiliography 162

[75] Gustafsson, I. An incomplete factorization preconditioning method for FEM-discretized partial differential equations based on modification of element matrices.BIT 36 (1996), 86–100.

[76] H., M. R. NASTRAN Theoretical Manual. NASA, 1970.[77] Haase, G., Langer, U., Reitzinger, S., and Schicho, J. Algebraic multigrid

methods based on element preconditioning. International Journal of ComputerMathematics 78, 4 (2001), 575–598.

[78] Hendrickson, B. Conditions for unique graph realizations. SIAM Journal onComputing 21 (1992), 65–84.

[79] Henson, V. E., and Yang, U. M. BoomerAMG: a parallel algebraic multigridsolver and preconditioner. Applied Numerical Mathematics 41 (2002), 155–177.

[80] Hestenes, M., and Stiefel, E. Methods of conjugate gradients for solving linearsystems. National Bureau of Standards Jounal of Research 49 (1952), 409–436.

[81] Higham, N. J. Notes on accuracy and stability of algorithms in numerical lin-ear algebra. In The Graduate Student’s Guide to Numerical Analysis ’98 (1999),M. Ainsworth, J. Levesley, and M. Marletta, Eds., Springer-Verlag , Berlin, pp. 48–82.

[82] Higham, N. J. Accuracy and Stability of Numerical Algorithms, second ed. Societyfor Industrial and Applied Mathematics, Philadelphia, PA, USA, 2002.

[83] Howle, V. E., and Vavasis, S. A. Preconditioning complex-symmetric layeredsystems arising in electrical power modeling. In Proceedings of the Copper Moun-tain Conference on Iterative Methods (Copper Mountain, Colorado, Mar. 1998). 7unnumbered pages.

[84] Hu, J., Tong, C., and Tuminaro, R. S. ML 2.0 smoothed aggregation user’sguide, June 04 2000.

[85] Hu, Y., and Scott, J. Ordering techniques for singly bordered block diagonalforms for unsymmetric parallel sparse direct solvers. Numerical linear algebra withapplications 12, 9 (2005), 877–894.

[86] Hughes, T. J. R. The Finite Element Method. Linear Static and Dynamic FiniteElement Analysis. Prentice-Hall, 1987.

[87] Hughes, T. J. R., Levit, I., and Winget, J. An element–by–element solu-tion algorithm for problems of structural and solid mechanics. Comp. Meth. Appl.Mech. Engng. 36 (1983), 241–254.

[88] Imai, H. On combinatorial structures of line drawings of polyhedra. DiscreteAppl. Math. 10 (1985), 79–92.

[89] Ipsen, I. C. F. Computing an eigenvector with inverse iteration. SIAM Review39 (1997), 254–291.

[90] Irony, D., Shklarski, G., and Toledo, S. Parallel and fully recursive mul-tifrontal supernodal sparse cholesky. Future Generation Computer Systems 20, 3(Apr. 2004), 425–440.

[91] Jacobs, D. J., and Hendrickson, B. An algorithm for two-dimensional rigiditypercolation: the pebble game. J. Comput. Phys. 137, 2 (1997), 346–365.

[92] Joshi, M., Gupta, A., Gustavson, F., Karypis, G., and Kumar, V.PSPASES: Scalable parallel direct solver library for sparse symmetric positivedefinite linear systems: User’s manual for version 1.0.3. Tech. Rep. TR 97-059,Department of Computer Science, University of Minnesota, 1997, revised 1999.

[93] Judice, J. J., Patricio, J., Portugal, L. F., Resende, M. G. C., andVeiga, G. A study of preconditioners for network interior point methods. Com-putational Optimization and Applications 24 (2003), 5–35.

[94] Kaniel, S. Estimates for some computational techniques in linear algebra. Math.Comp. 20 (1966), 369–378.

Bibiliography 163

[95] Karypis, G., and Kumar, V. A fast and high quality multilevel scheme forpartitioning irregular graphs. SIAM Journal on Scientific Computing 20 (1998),359–392.

[96] Kaveh, A., and Koohestani, K. An efficient graph-theoretical force method forthree-dimensional finite element analysis. Communications in Numerical Methodsin Engineering (2007).

[97] Kaveh, A., Koohestani, K., and Taghizadieh, N. Efficient finite elementanalysis by graph-theoretical force method. Finite Elements in Analysis and De-sign 43 (2007), 543–554.

[98] Kazhdan, M., Bolitho, M., and Hoppe, H. Poisson surface reconstruction.In Symposium on Geometry Processing (2006), ACM/Eurographics, pp. 61–70.

[99] Kolotilina, L. Y., and Yeremin, A. Y. Factorized sparse approximate inversepreconditioning: I. theory. SIAM Journal on Matrix Analysis and Applications 14(1993), 45–58.

[100] Koutis, I., and Miller, G. L. A linear work, O(n1/6) time, parallel algo-rithm for solving planar Laplacians. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2007, New Orleans, Louisiana,USA, January 7-9, 2007 (2007), N. Bansal, K. Pruhs, and C. Stein, Eds., SIAM,pp. 1002–1011.

[101] Laman, G. On graphs and rigidity of plane skeletal structures. Journal of Engi-neering mathematics 4 (1970), 331–340.

[102] Langer, U., Reitzinger, S., and Schicho, J. Symbolic methods for the el-ement preconditioning technique. Tech. rep., Johannes Kepler Universitat (JKU)Linz, Jan. 2002.

[103] Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D.,Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade,J., and Fulk, D. The digital Michelangelo project: 3D scanning of large statues.In Siggraph (2000), ACM, pp. 131–144.

[104] Li, X. S., and Demmel, J. W. SuperLU DIST: A scalable distributed mem-ory sparse direct solver for unsymmetric linear systems. ACM Transactions onMathematical Software 29 (2003), 110–140.

[105] Lipton, R. J., Rose, D. J., and Tarjan, R. E. Generalized nested dissection.SIAM Journal on Numerical Analysis 16 (1979), 346–348.

[106] Lovasz, L., and Yemini, Y. On generic rigidity in the plane. SIAM J. AlgebraicDiscrete Methods 3 (1982), 91–98.

[107] Maggs, B. M., Miller, G. L., Parekh, O., Ravi, R., and Woo, S. L. M.Solving symmetric diagonally-dominant systems by preconditioning. Unpublishedmanuscript available online at http://www.cs.cmu.edu/~bmm, 2002.

[108] Maggs, B. M., Miller, G. L., Parekh, O., Ravi, R., and Woo, S. L. M.Finding effective support-tree preconditioners. In SPAA ’05: Proceedings of theseventeenth annual ACM symposium on Parallelism in algorithms and architec-tures (2005), ACM Pres, pp. 176–185.

[109] Mandel, J. Balancing domain decomposition. Comm. Numer. Meth. Engrg. 9(1993), 233–241.

[110] Manteuffel, T. An incomplete factorization technique for positive definite linearsystems. Mathematics of Computation 34 (1980), 473–497.

[111] MathWorks, T. Matlab version 7.2. software package, Jan. 2006.[112] MathWorks, T. Matlab version 7.4. software package, Jan. 2007.[113] Meijerink, J. A., and van der Vorst, H. A. An iterative solution method

for linear systems of which the coefficient matrix is a symmetric M-matrix. Math-ematics of Computation 31 (1977), 148–162.

Bibiliography 164

[114] Meshar, O., Irony, D., and Toledo, S. An out-of-core sparse symmetricindefinite factorization method. ACM Transactions on Mathematical Software 32(2006), 445–471.

[115] Mohar, B. The laplacian spectrum of graphs. Graph Theory, Combinatorics andApplications. 2 (1991), 871–898.

[116] Munksgaard, N. Solving sparse symmetric sets of linear equations by precondi-tioned conjugate gradients. ACM Trans. Math. Softw. 6, 2 (1980), 206–219.

[117] Ng, Esmond. A scheme for handling rank-deficiency in the solution of sparselinear least squares problems. SIAM J. Sci. and Stat. Comput. 12, 5 (1991), 1173–1183.

[118] Notay, Y. Conditioning analysis of modified block incomplete factorizations. Lin-ear Algebra and its Applications 154-156 (1991), 711–722.

[119] Paige, C. C., and Saunders, M. A. Solution of sparse indefinite systems oflinear equations. SIAM Journal on Numerical Analysis 12 (1975), 617–629.

[120] Papadrakakis, M., and Fragakis, Y. An integrated geometric-algebraicmethod for solving semi-definite problems in structural mechanics. ComputerMethods in Applied Mechanics and Engineering 190 (2001), 6513–6532.

[121] Parlett, B. N., and Reid, J. K. On the solution of a system of linear equationswhose matrix is symmetric but not definite. BIT 10 (1970), 386–397.

[122] Persson, P.-O., and Strang, G. A simple mesh generator in MATLAB. SIAMReview 46 (2004), 329–345.

[123] Pierce, D. J., and Lewis, J. G. Sparse multifrontal rank revealing qr factor-ization. SIAM J. Matrix Anal. Appl. 18, 1 (1997), 159–180.

[124] Portugal, L., Bastos, F., Judice, J., Paixao, J., and Terlaky, T. Aninvestigation of interior-point algorithms for the linear transportation problem.SIAM Journal on Scientific Computing 17 (1996), 1202–1223.

[125] Portugal, L. F., Resende, M. G. C., Veiga, G., and Judice, J. J. A trun-cated primal-infeasible dual-feasible interior point network flow method. Networks35 (2000), 91–108.

[126] Pothen, A. Sparse null basis computations in structural optimization. Nu-merische Mathematik 55, 5 (Aug. 1989), 501–519.

[127] Reif, J. H. Efficient approximate solution of sparse linear systems. Computers andMathematics with Applications 36, 9 (1998), 37–58. See also an errata in volume38, page 141, 1999.

[128] Reitzinger, S. Algebraic multigrid and element preconditioning I. Tech. rep.,Johannes Kepler Universitat (JKU) Linz, Dec. 1998.

[129] Resense, M., and Veiga, G. An efficient implementation of the network interior-point method. In Network Flows and Matching: the First DIMACS Implementa-tion Challenge, D. Johnson and C. McGeoch, Eds., vol. 12 of DIMACS Series inDiscrete Mathematics and Computer Science. AMS, 1993.

[130] Robert, Y. Regular incomplete factorizations of real positive definite matrices.Linear Algebra and its Applications 48 (1982), 105–117.

[131] Rozloznık, M., Shklarski, G., and Toledo, S. Partitioned triangular tridi-agonalization. Submitted to ACM Transactions on Mathematical Software, 2008.

[132] Ruge, J. W., and Stuben, K. Algebraic multigrid (AMG). In Multigrid Meth-ods, S. F. McCormick, Ed., vol. 3 of Frontiers in Applied Mathematics. SIAM,Philadelphia, PA, 1987, pp. 73–130.

[133] Saad, Y. ILUT: a dual threshold incomplete LU factorization. Numerical linearalgebra with applications 1, 4 (1994), 387–402.

[134] Saad, Y., and Schultz, M. GMRES: A generalized minimal residual algorithmfor solving nonsymmetric linear systems. SIAM Journal of Scientific and StatisticalComputing 7 (1986), 856–869.

Bibiliography 165

[135] Shapiro, A. Upper bounds for nearly optimal diagonal scaling of matrices. Linearand Multilinear Algebra 29 (1991), 145–147.

[136] Sharf, A., Lewiner, T., Shklarski, G., Toledo, S., and Cohen-Or, D. In-teractive topology-aware surface reconstruction. ACM Trans. Graph 26, 3 (2007),43.

[137] Shklarski, G., and Toledo, S. Computing the null space of finite elementproblems. Submitted to Computer Methods in Applied Mechanics and Engineering,2008.

[138] Shklarski, G., and Toledo, S. Rigidity in finite-element matrices: Sufficientconditions for the rigidity of structures and substructures. SIAM Journal on MatrixAnalysis and Applications 30, 1 (2008), 7–40.

[139] Si, H. TetGen, A Quality Tetrahedral Mesh Generator and Three-DimensionalDelaunay Triangulator: Users’s Manual for Version 1.4, Jan. 2006. Available inhttp://tetgen.berlios.de.

[140] Smith, B. F., Bjorstad, P. E., and Gropp, W. D. Domain Decomposition:Parallel Multilevel Methods for Elliptic Partial Differential Equations. CambridgeUniversity Press, 1996.

[141] Spielman, D. A., and Teng, S.-H. Solving sparse, symmetric, diagonally-dominant linear systems in time 0(m1.31). In Proceedings of the 44th Annual IEEESymposium on Foundations of Computer Science (Oct. 2003), pp. 416–427.

[142] Spielman, D. A., and Teng, S.-H. Nearly-linear time algorithms for graphpartitioning, graph sparsification, and solving linear systems. In STOC ’04: Pro-ceedings of the thirty-sixth annual ACM symposium on Theory of computing (NewYork, NY, USA, 2004), ACM Press, pp. 81–90.

[143] Spielman, D. A., and Teng, S.-H. Nearly-linear time algorithms for precondi-tioning and solving symmetric, diagonally dominant linear systems. Unpublishedmanuscript available online at http://arxiv.org/abs/cs/0607105, 2006.

[144] Strang, G. Introduction to Applied Mathematics. Wellesley-Cambridge, 1986.[145] Vaidya, P. M. Solving linear equations with symmetric diagonally dominant

matrices by constructing good preconditioners. Unpublished manuscript. A talkbased on the manuscript was presented at the IMA Workshop on Graph Theoryand Sparse Matrix Computation, October 1991, Minneapolis.

[146] Valiant, L. G. Graph theoretic arguments in low-level complexity. In Proceedings6th Symposium on Mathematical Foundations of Computer Science (1977), vol. 53of Lecture Notes in Mathematics, Springer, pp. 162–176.

[147] van der Sluis, A. Condition numbers and equilibration of matrices. NumerischeMathematik 14 (1969), 14–23.

[148] Vanek, P. Acceleration of convergence of a two level algorithm by smoothingtransfer operators. Appl. Math. 37 (1992), 265–274.

[149] Vanek, P., Mandel, J., and Brezina, M. Algebraic multigrid based onsmoothed aggregation for second and fourth order problems. Computing 56 (1996),179–196.

[150] Varga, R. S. Factorization and normalized iterative methods. In Boundary Prob-lems in Differential Equations, R. E. Langer, Ed. University of Wisconsin Press,Madison, WI, 1960, pp. 121–142.

[151] Varga, R. S., B., S. E., and Mehrmann, V. Incomplete factorizations ofmatrices and connections with H-matrices. SIAM Journal on Numerical Analysis17 (1980), 787–793.

[152] Vavasis, S. Private communication, 2007.[153] Wang, M., and Sarin, V. Constructing m-matrix preconditioners for finite ele-

ment problems. Submitted to SIAM Journal on Scientific Computing, 2008.

http://tetgen.berlios.de

Bibiliography 166

[154] Watts, J. A conjugate gradient truncated direct method for the iterative solu-tion of the reservoir simulation pressure equation. In Proc. SPE 54th Annual FallTechnical Conference and Exhibition, Las Vegas (1979).

[155] Whiteley, W. Some matroids from discrete applied geometry. In Matroid theory,Contemporary Mathematics, vol. 197, J. O. J. Bonin and B. Servatius, Eds. Amer.Math. Soc., 1996, pp. 171–311.

[156] Young, D. P., Melvin, R. G., Johnson, F. T., Bussoletti, J., Wigton,L. B., and Samant, S. S. Application of sparse matrix solvers as effective pre-conditioners. SIAM Journal on Scientific and Statistical Computing 10 (1989),1186–1199.

[157] Zlatev, Z. Use of iterative refinement in the solution of sparse linear systems.SIAM Journal on Numerical Analysis 19, 2 (Apr. 1982), 381–399.

סופי אלמנט לבעיות קומבינטוריים קירובים

נומרית לינארית לאלגברה נוספות ותרומות

לפילוסופיה דוקטור תואר קבלת לשם חיבור

מאת

שקלרסקי גיל

של בהדרכתו נעשתה זאת עבודה

טולדו סיון פרופ׳

תל־אביב אוניברסיטת של לסנאט הוגש

2008 אוגוסט

3

תמציתקומבינטוריים (preconditioners) קירובים הינו זאת תיזה של המרכזי הנושאיכולים אלה קירובים חיובית־למחצה. מוגדרות סימטריות סופי אלמנט למטריצותהאלמנט בשיטת המתקבלות לינאריות משוואות מערכות של יעיל לפיתרון לשמשהקירובים סופי. אלמנט בעיות של (null-space) האפס מרחב ולחישוב הסופיבקירוב המתקבלות מטריצות כולל סופי, אלמנט מטריצות של רחב למגוון ישימים(למשל, ווקטוריות סקלאריות חלקיות דיפרנציאליות משוואות של סופי אלמנט

לינארית). ואלסטיות אלקטרו־סטטיות בעיותבאופן ששימשו קומבינטוריות, קירוב טכניקות כי מראה זאת תיזה עבודתלהרחבה ניתנות דומיננטי, אלכסון בעלות סימטריות מטריצות לקירוב מסורתימוגדרות סימטריות סופי אלמנט מטריצות מטריצות: של יותר רחבה למחלקהכולן זה מסוג במטריצות לטיפול קיימות קומבינטוריות שיטות חיובית־למחצה.והאלגוריתמים התאוריה מסוימת. פיתרון לשיטת או מסוימת לבעיה מוגבלותכלליים הינם ולפיכך לחלוטין כמעט קומבינטוריים־אלגבריים הינם זאת בתיזהפיתרון שיטת בפרטי או המקורית הפיזיקלית הבעיה בפרטי ישירות תלויים ואינם

מסוימת.לקירובים תאורטית תשתית פיתוח הינה זאת בתיזה המרכזית התוצאהקומבינטורי ייצוג מציגה היא בפרט, סופי. אלמנט למטריצות קומבינטורייםאלגברית־קומבינטורית תאוריה וכן הקשיחות, גרף הנקרא סופי, אלמנט למטריצותעל בהתבסס זה. מסוג מטריצות של האפס למרחב זה גרף את המקשרתמטריצות לדילול חדשנית קומבינטורית־אלגברית שיטה מציגים אנו זאת, תאוריהמרחב אותו בעלי קירובים מייצרת זאת שיטה המסורית. הרחבת הנקראת(2006) Daitchו־ Spielman את שימשו אלה קירובים המקורית. למטריצה אפס(trusses) במסבכים שמקורן למטריצות מוכח באופן יעיל לינארי לפותר כבסיסמרחב לחישוב משמשים הקירובים זאת, בתיזה ממשולשים. הבנויים דו־מימדייםמסוימים, בתנאים שוויון. אילוצי עם מודלים כולל סופי, אלמנט מודלי של האפסוהמקום הזמן כמות לפיכך, התמלאות. ללא דליל לפירוק ניתנת המסורית הרחבתכללי, באופן המודל. גודל עם לינארי באופן גדלים זה חישוב ידי על הנדרשיםשיטה זאת, למרות הנדרשים; התנאים כלל את מקיימים לא תעשייתיים מודליםמרחב לחישוב מוכרות שיטות ומנצחת מעשי באופן יעיל באופן עובדת עדיין זאתעבודה זוהי רבים. שוויון אילוצי עם מודלים כולל תעשייתיים, מודלים של האפס

טולדו. סיון עם משותפתדיפרנציאליות משוואות עבור הסופי האלמנט בשיטת המתקבלות למטריצותליצירת לנצל שניתן אלגבריות תכונות מספר קיימות סקלאריות אליפטיות חלקיותחדשני לינארי ופותר אלה תכונות של מחקר מציגה זאת תיזה יעיל. לינארי פותרמטריצות לקירוב טהורות אלגבריות בשיטות משתמש הפותר עליהן. המבוססלאחר דומיננטי. אלכסון ובעלות סימטריות שהינן אלמנט מטריצות ידי על אלמנטסטנדרטיים. קומבינטוריים קירובים ידי על מקורבת המתקבלת המטריצה מכן,הפותר קרובות, לעיתים הקורה דבר לקירוב, ניתנות האלמנט מטריצות כל כאשרמזהה הפותר לקירוב, ניתנות שלא מטריצות קיימות כאשר מוכח. באופן יעיל הינוהפותר יותר, גדול נעשה זה שחלק ככל במודל. שלהן החלק את מדלל ולא אותןעבור כי מראות ניסוייות תוצאות ישיר. כפותר הדרגתי באופן להתנהג מתחילאפקטיבי שלנו הפותר גבוה, תנאי מספר בעל הינו מהאלמנטים חלק בהן בעיותישיר. מפותר וכן incomplete factorization מפותר אלגברי, multigrid מפותר יותר

טולדו. וסיון חן דורון אברון, חיים עם משותפת עבודה זוהיהאלמנט בשיטת שמקורן למטריצות חדשני שימוש מציגה זאת תיזה בנוסף,משתמשת היא אליפטיות. סקלאריות חלקיות דיפרנציאליות משוואות עבור הסופימידע מתוך מימדי תלת משטח לשחזור ממוחשבת בגרפיקה בעיה לפתרון בהן

4

ונקודות מימדי תלת גוף של משטח על נקודות קבוצת הוא הקלט וחלקי. רועשפונקציית הממזער משטח הוא הפלט הגוף. ובתוך לגוף מחוץ הן כי שידוע נוספותוממופה ספציפית, ברשת תלוי שאינו באופן מנוסחת הבעיה ספציפית. עונשבעיית של של יעילה בנייה מציגה התיזה הסופי. האלמנט שיטת בעזרת לרשתהיא הסופי האלמנט שיטת האופטימיזציה. בעיית לפיתרון פחותים ריבועיםהמייצגת רגולרית לא דלילה רשת יצירת מאפשרת היא התהליך. ליעילות המפתחהריבועים בעיית יותר, צפופה רגולרית ברשת בשימוש אפקטיבית. עונש פונקצייתשרף, אנדרי עם משותפת עבודה זוהי לפיתרון. בהרבה קשה היתה הפחותים

כהן־אור. ודניאל טולדו סיון ,Thomas Lewinerהתיזה נומרית. לינארית לאלגברה נוספת תרומה גם מכילה זאת תיזה לבסוף,תלת לצורה סימטרית מטריצה לצמצום (partitioned) חצוי אלגוריתם מציגהPAPT = הפירוק לחישוב אלגוריתם כלומר, חלקי. pivoting עם אלכסונית,עם תחתונה משולשית מטריצה היא L פרמוטציה, מטריצת היא P כאשר ,LTLT

מבוסס האלגוריתם תלת־אלכסונית. סימטרית מטריצה היא T ו־ יחידה אלכסוןהשימוש עמודה. אחר עמודה הפועלות Aasen ושל Parlett-Reid של השיטות עלזכרונות (ובפרט, מודרניות מחשב ארכיטקטורות לנצל מאפשר חצוי באלגוריתםהאלגוריתם כי מראות ניסוייות תוצאות ביותר). יעילות blas וספריות מטמוןBunch-Kaufman ופיתרון פירוק של הביצועים לרמת דומה ביצועים רמת משיגוסיון Miroslav Rozloznık עם משותפת עבודה זוהי .lapack התוכנה בספריית

טולדו.

מתומצת רקע .15

תקציר

רבים. והנדסיים מדעיים ביישומים חשוב תפקיד משחקת נומרית לינארית אלגברה

הבעיה. של מתמטי במידול מתחיל נומרית לינארית אלגברה של טיפוסי יישוםהניתן בדיד למודל הפיכתו את כולל בפיתרון הראשון השלב רציף, המודל אםאלגוריתם לדוגמה, הבעיה; על מופעל פיתרון אלגוריתם זה בשלב במחשב. לייצוגכלשהיא. אופטימיזציה בעיית לפתור או זמן תלויית בעיה לסמלץ יכול הפיתרוןפשוטות בעיות של לרצף או לקבוצה הבעיה את הופכים רבים פיתרון אלגוריתמימערכות פיתרון לינאריות, משוואות מערכות פיתרון כגון לינארית, באלגברה יותרבעיות עבור ויציב יעיל פותר מטריצה. של ספקטרום חישוב או פחותים ריבועיםשל ומדוייק יעיל לפיתרון המפתח רבים במקרים הוא לינארית באלגברה אלה

כולה. הבעיהלינארית באלגברה ליבה בעיית הוא לינאריות משוואות מערכות פיתרוןמלאות עיקריים: סוגים לשני מקוטלגות להיות יכולות משוואות מערכות נומרית.של משמעותית כמות מכילה המקדמים מטריצת דלילות, במערכות ודלילות.לנצלם. כדאי שיהיה מכדי מידי קטנה האפסים כמות מלאות, במערכות אפסים.בדרך משתמש מודרני דליל פותר לשני; האחד קשורים ומלאים דלילים פותרים

במטריצה. מלאים קטנים בלוקים לעיבוד עזר כשגרת מלא בפותר כללומלאות: דלילות מערכות לפיתרון שיטות של עיקריות משפחות שתי קיימותמטריצת את מפרקות ישירות שיטות איטרטיביות. ושיטות ישירות שיטותשיטות שלהן. ההופכי את להפעיל שקל מטריצות של למכפלה המקדמיםאחר שלב אותו לשפר ומנסות למשוואה מקורב פיתרון עם מתחילות איטרטיביותבמקרים לשני; האחד קשורים הם גם איטרטיביים ופותרים ישירים פותרים שלב.המקדמים. מטריצת של קירוב של פירוק כולל איטרטיבי פיתרון תהליך רבים,

התקציר. של יותר מאוחר בשלב מופיע אלה שיטות של מתומצת תיאורמתאר 5 פרק אלה. קשורות שיטות מתוך רחב מגוון על עבדתי שלי במחקרתורגמה זאת בעיה וכיצד הממוחשבת הגרפיקה מתחום בעיה של מתמטי מודלהתיזה, של המרכזי הנושא את המציגים ו־4, 3 ,2 פרקים מחשב. על בדידה לבעיהלפני שביצעתי במחקר איטרטיביים. דלילים פותרים להאצת טכניקות מתאריםהשתתפתי ב־[12] ישירים. דלילים פותרים על עבדתי הדוקטורט, לימודי התחלתב־ סימטריות. שאינן למטריצות דליל ישיר פותר של ביצועים ובניתוח בייעולזה פותר חיובית. מוגדרות סימטריות למטריצות דליל מקבילי פותר פיתחתי [90]אלה תרומות המחקר. מאותו כחלק שפותח מהיר־ביותר מלא בפותר משתמשבתחום לאחרונה שביצעתי עבודה מציג 6 פרק לבסוף, זאת. בתיזה נכללות אינןשהן סימטריות למטריצות חדשני פירוק אלגוריתם המלאים: הישירים הפותרים

ביותר. מהירים ביצועים המשיג indefiniteבזכות לעמוד מוכוון הפרק התיזה. של התוצאות את סוקר זה פרק שארבתיאור ממשיך לעבודה, הרקע של פורמלית לא בסקירה מתחיל הוא עצמו;

בתיזה. פרק מסכמת בתקציר יחידה כל כאשר התוצאות,

מתומצת רקע 1

סופי אלמנט מטריצות 1.1

,Kx = f המשוואות מערכת של היעיל הפיתרון הוא התיזה של המרכזי הנושאהאלמנט שיטת של פורמלי פיתוח הסופי. האלמנט בשיטת נוצרו fו־ K כאשרלא זאת בתיזה התוצאות רוב כן, על יתר זה. תקציר להיקף מחוץ הוא הסופיתכונות מספר רק אלא הסופי, האלמנט שיטת של וספציפי מדוייק פיתוח דורשות

לפיתרון שיטות 1.26

התכונות את מתארים אנו זה בחלק המתקבלות. המטריצות של אלגבריותהללו. האלגבריות

סופי. אלמנט מטריצת לפעמים או קשיחות מטריצת נקראת K המטריצההנקראות Ke יותר דלילות מטריצות של K =

∑

e Ke לסכום שווה היא1 איור זרמים. או כוחות של וקטור כלל בדרך הוא f הווקטור אלמנט. מטריצותבעזרת לאלמנטים המחולק דו־מימדי גוף מציג האיור .fו־ K של דוגמא מציגפיזיקליות בעיות שתי לייצג יכול הגיאומטרי הגוף אותו משולשים. של רשתהמייצג בודד למשתנה משוייך ברשת צומת כל מוליך: חומר (1) ואלגבריות:המוזרמים חיצוניים זרמים מייצג f הווקטור (מתח). אלקטרוסטטי פוטנציאלהרשת בצמתי המתקבל הפוטנציאל הוא x הפיתרון וקטור הרשת. צמתי לתוךהמערכת על הנכפים החיצוניים לזרמים שיתאימו פנימיים זרמים לייצור הנדרשf הוקטור תזוזה. משתני לשני משויך ברשת צומת כל אלסטי: חומר (2) .fהפיתרון וקטור צומת). בכל כיוון רכיבי (שני הרשת צמתי על המופעל כוח מייצגההזזה היא x אחרות, במילים הכוח. ידי על הנגרם בגוף העיוות את מייצג x

.f החיצוניים הכוחות עם שיתאזנו בגוף פנימיים כוחות לייצור הנדרשתשל חסום למספר פרט לגמרי מאופסות כמעט Ke האלמנט מטריצותהמטריצות בו במקרה מתרכזים אנו זאת בתיזה אפס. שאינן ועמודות שורותבעלות Ke המטריצות כלל, בדרך למחצה. חיובית ומוגדרות סימטריות הן Ke

בהמשך. זאת תכונה נסביר אפס. למרחב תאימות קוראים אנו לה נוספת תכונהוהרכבתם אלמנטים שלושה עם פשוט מודל עבור אלמנט מטריצות מציג 2 איור

.K הקשיחות למטריצתמבנה את לנצל יכולים אנחנו ״האם היתה עצמנו את שאלנו אותה השאלהלינארי פותר ליצירת סופיים אלמנטים מטריצות של האלגבריות התכונות ואת זה

מטריצות?״. של זו למחלקה יעיל

לפיתרון שיטות 1.2

פותרים :Kx = f הלינארית למערכת פותרים של עיקריות משפחות שתי קיימותהמקדמים מטריצת של פירוק כוללים ישירים פותרים איטרטיביים. ופותרים ישיריםומוגדרת סימטרית היא K שלנו במקרה פשוטות. מטריצות של למכפלה Kתחתונה משולשית מטריצה של למכפלה Cholesky לפירוק ניתנת K לכן חיובית,ניתן אחרים במקרים .LT המשוחלפת העליונה המשולשית המטריצה עם Lמפורקת, Kש־ לאחר חלקי. pivoting עם LU או LDLT בפירוק להשתמשישירות שיטות הפיתרון. את מייצר לאחור והצבה לפנים הצבה של זול תהליךשל הדלילות תבנית בעזרת לתחזית ניתן שלהן הריצה זמן אמינות, הן דלילות

משולש כל משולשים. של רשת בעזרת לאלמנטים מחולק מימדי דו גוף :1 איורהרשת צמתי מצב .K למטריצה מתאים הגוף וכל Ke אלמנט למטריצת מתאים

.f לוקטור מתאימים והחיצים x לוקטור מתאים

קומבינטוריים. קירובים 1.37

(א) (ב)

סימטרית היא אלמנט מטריצת כל בדו־מימד. אלסטיים משולשים :2 איורשורות ב־6 אפס שאינם איברים 36 מכילה היא למחצה. חיובית ומוגדרת

רשת. נקודת לכל המתאימים המשתנים ואת המודל את מציג (א) חלק ועמודות.השורות .K סכומן ואת Ke המתאימות האלמנט מטריצות את מציג (ב) חלקהאלמנט בהם למשתנים מתאימות אלמנט מטריצת בכל אפס שאינן והעמודות

נוגע.

דלילות למטריצות אלה שיטות המיישמות תוכנה ספריות קיימות המטריצה.[47] umfpack ,[73, 92] pspases ,[4, 5] mumps ,[90] taucs ,[48] cholmod (למשל,דרישות רבים, במקרים כי היא ישירות בשיטות הבעיה .([49, 104] SuperLu ו־הבעיה. גודל עם מידי מהר גדל שלהן הריצה שזמן וכן מידי גבוהות שלהן הזיכרון

.4 פרק של הניסויי בחלק מודגמת זאת תכונהמנסה ואז (x(0) = 0 (למשל, x(0) מקורב פיתרון עם מתחיל איטרטיבי פותרx(i) כל מיוצרת, x(1), x(2), ... פתרונות סדרת אותו. לשפר איטרטיבי באופןכאשר ,s בשלב עוצר הפותר .x(i−1) הקודם הפתרון של שיפור (בשאיפה) הואConjugate Gradients הם הבסיסיים הנפוצים הפותרים .fל־ מספיק קרוב Kx(s)

Minimal Residual ו־ חיובית מוגדרות סימטריות מטריצות עבור ([41, 80] cg)Generalized Minimal Residual כמו־כן, סימטריות. מטריצות עבור ([119] minres)כללי, באופן כלליות. מטריצות עם מערכות לפיתרון משמש ([133] gmres)לא היא K אם רע. לתפקד עלולות איטרטיביות שיטות נוספים, אמצעים ללאK אם כנ״ל. בפותר האיטרציות למספר תאורטי חסם אפילו אין סימטרית,ידי על חסום האיטרציות מספר חיובית, ומוגדרת סימטרית היאהמסתיר פסימיסטי Chebyshev חסם הוא זה חסם .

√

λmax(K)/λmin(K) הערךעבור זאת, עם יחד .([94] (ראה ההתכנסות של לינארית הלא ההתנהגות אתאיטרטיבי תהליך מתקבל שבעזרתן טכניקות פותחו מטריצות של רבות מחלקותהבא. בסעיף מתוארות אלה מטכניקות מקצת ישירות. משיטות יותר ויעיל אמין


קירוב בפשטות לו ונקרא M שנסמנו ,K המטריצה של להפיכה קל קירובהאיטרטיביים. הפותרים התכנסות את דרמטי באופן להאיץ יכול ,(preconditioner )הזאת הגישה .[150, 13, 14] כגון מוקדמות בעבודות מופיע קירובים של הרעיוןקירובים פותחו השנים במהלך .[113] השבעים שנות מסוף החל תאוצה תפסה[154, (incomplete factorization) שלם לא פירוק קירובי למשל כמו רבים, מסוגים[132, multigrid שיטות ,151, 116, 110, 157, 130, 2, 156, 118, 134, 34, 75, 39, 65]למטריצה דלילים קירובים ,[140, 109] domain-decomposition קירובי ,29, 30]

.[31, 17, 99] ההפוכה המטריצה של לפירוק וקירובים [42, 17, 71] ההפוכההמשוואה לפיתרון סתום באופן בו להשתמש היא M לקירוב נפוצה שימוש דרךפותר לקבל על־מנת .Kx = f המקורית המשוואה במקום M−1Kx = M−1f

:Mמ־ דרישות שלוש קיימות יעיל,


K G(K) G(M) M

דומיננטי אלכסון עם סימטרית למטריצה קומבינטורי קירוב בניית :3 איורשלו הפורש לעץ G(K) הגרף את המדלל ,Vaidya מסוג הוא זה קירוב .(SDD)עם SDD שמטריצות כיוון נגדים, כרשת מיוצגות Mו־ K המטריצות .G(M)

ברשת צומת כל אם נגדים. רשתות לייצג יכולות לאלכסון מחוץ שליליים ערכים,xב־ הנתון פוטנציאל עם מוארקת לסוללה מחוברת היתה K של באיור הנגדים

מתקיים הדבר מהסוללות. אלה לצמתים הנכנס המכוון הזרם הוא Kx אזי.M עבור דומה באופן

יעילה. היא K מתוך M של הבנייה .1

על חסום CG באלגוריתם האיטרציות מספר קטן. הוא האיטרציות מספר .2של המנה הוא ,A של התנאי מספר ,κ(A) כאשר

√

κ(M−1K) הערך ידי.κ(A) = λmax(A)/λmin(A) הקיצוניים העצמיים הערכים

זול. הוא Mz = r המשוואה פיתרון אחרות, במילים קלה. היא M−1 הפעלת .3Cholesky בפירוק להשתמש היא המשוואות מערכת את לפתור נפוצה דרך

.M של דליל

הדרישות את מושלם באופן מספקת M = K קביעת סותרות; הנ״ל הדרישותמספקת M = I קביעת השלישית; הדרישה את סותרת אבל והשנייה, הראשונההשנייה. הדרישה את סותרת אבל והשלישית הראשונה הדרישה את מושלם באופן

הנ״ל. הדרישות בין לאזן אמור טוב קירוב לפיכך,החסם זה במקרה סינגולריות. Mו־ K בו המקרה עבור טכנית הערה נעירהמוכללים העצמיים הערכים קבוצת על לחסם מתרגם העצמיים הערכים על

.[18] Λ(K, M) = λ|∃x s.t. Kx = λMx and Mx 6= 0קומבינטורי קירוב הנקראת יחסית חדשה קירוב בטכניקת מתמקדים אנוsupport precondition-) תמיכה קירוב ולעיתים (combinatorial preconditioning )G(K) קומבינטורי ייצוג מקבלת K הקלט מטריצת קומבינטורי, בקירוב .(ing

G(K) מתוך מייצר קומבינטורי אלגוריתם אז, היפר־גרף). או גרף כלל (בדרך3 איור .G(M) מתוך מיוצר M הקירוב לבסוף, .G(M) חדש קומבינטורי ייצוג

זה. תהליך את ממחישבראשית Vaidya ידי על הוצעה קומבינטוריים קירובים של הראשונה המשפחהעם סימטריות מטריצות עבור עובד Vaidya של הקירוב .[145] התשעים שנותמחוץ שליליים ערכים עם (Symmetric Diagonally Dominant) דומיננטי אלכסוןש־ כך ממושקל; לגרף המטריצה את ממיר שלו האלגוריתם כללי, באופן לאלכסון.Vaidya של האלגוריתם .G(K) הגרף של הממושקלת הסמיכות מטריצת היא Kזאת למחלקה מוכחת יעילות בעלי קירובים בנה Vaidya זה. מגרף קשתות מסירההתכנסות קצב וניתוח לחלוטין קומבינטורית היא הבנייה שיטת מטריצות; של

סופי אלמנט במטריצות קשיחות .29

חסמי .(graph embeddings (בעזרת קומבינטורי בעיקר אבל אלגברי בחלקו הואהקלט. מטריצת של התנאי במספר תלויים לא הריצה זמן

משמעותי. באופן הורחבה Vaidya של התיאורטית העבודה האחרון, בעשורחן, ,Boman .[37] Vaidya של הקירובים של ניסויית הערכה ביצעו וטולדו חןהסימטריות המטריצות לכלל Vaidya של הקירובים את הכלילו וטולדו Hendricksonאלגברית מסגרת פתחו Hendricksonו־ Boman .[22] דומיננטי אלכסון בעלותהתמיכה תורת הנקראת [24] קומבינטוריים קירובים של תנאי מספרי לניתוח כלליתהמטריצה בהם קומבינטוריים קירובים הציעו Millerו־ Gremban .(support theory )Maggs .[70, 69] K המקורית מהמטריצה ועמודות שורות יותר בעלת M.[108] Millerו־ Gremban של לקירובים מתוחכמת גרסה לבנות כיצד הראו ושותפיואת לחסום כדי התמיכה בתורת השתמשו [18] ושותפיו Bernו־ [72] Guatteryו־ Howle שלם. לא פירוק על המבוססים קירובים מספר של התנאי מספרי.[83] מרוכבות למערכות Millerו־ Gremban של הקירובים את הכלילו VavasisSpielman .[127] רקורסיביות גרסאות כולל ,Vaidya דמויי קירובים וניתח הציע Reifקשתות הוספת ידי על בעיקר ,Vaidya של הקירובים את שיפרו [141, 142] Tengו־משוואות מערכות בעזרתם לפתור ניתן כיצד הראו הם מתוחכמים. פורשים לעציםKoutis לינארי. כמעט בזמן דומיננטי אלכסון בעלות סימטריות מטריצות עםהגרף כאשר כאלה, משוואות מערכות לפתור ניתן כיצד הראו [100] Millerו־

יעילה. בצורה מקבילי באופן וכן לינארי בזמן מישורי הוא להן המתאיםסימטריות מטריצות עבור שימשו קומבינטוריים קירובים ,2004 עד 1991 בשניםבשימושים נפוצות אינן אלה מטריצות בלבד. (SDD) דומיננטי אלכסון בעלות[74] Gupta וגם [25] Vavasisו־ Hendrickson ,Boman גם ,2004 בשנת מעשיים.שמקורן מטריצות עבור קומבינטוריות קירוב בשיטות להשתמש ניתן כי הראוסקלאריות. אליפטיות חלקיות דיפרנציאליות בעיות עבור סופיים אלמנטים בפיתוחכן, על יתר אליפטיות. סקלאריות למשוואות רק מתאימות שלהן העבודותמסוימים. סופיים אלמנטים ולפיתוחי מסוימות לבעיות ספציפיות שלהן העבודותיותר. מורכבים סופי אלמנט מודלי על ישיר באופן לפעול יכולות לא אלה שיטותהיה סופיים אלמנטים לבעיות קומבינטוריות קירוב שיטות של היישום לפיכך,

למדי. מוגבל עדייןסופי אלמנט למטריצות הקומבינטורי הקירוב שיטות את מרחיבה זאת תיזהאלגברי מידע בעזרת נבנים הקירובים וסקלאריות. ווקטוריות מבעיות הנוצרות

סופיים. אלמנטים לפיתוח ספציפית ולדרך לבעיה ספציפי במידע שימוש ללא

סופי אלמנט במטריצות קשיחות 2

קשיחות של אלגברית תיאוריה 2.1

SIAM Journal on Matrix Analysis andב־ שפורסם מאמר על המבוסס ,2 בפרקמטריצות עבור קשיחות של אלגברית תיאוריה מציגים אנו ,[138] Applications

קומבינטוריים קירובים לפיתוח כתשתית לשמש יכולה הזו התאוריה סופי. אלמנטסופי. אלמנט למטריצות

Kב־ המסומן (א) בחלק המודל סופי. אלמנט של מודלים שלוש מתאר 4 איורגרסה הוא M הקירוב קומבינטוריים, בקירובים כלל, בדרך המקורי. המודל הוא.M2ו־ M1 קירובים להיות מועמדים שני מציגים ו־(ג) (ב) חלקים .K של מדוללת

שהושמטו. אלמנטים מספר עם ,Kל־ דומים הםהמועמדים אחד האם היא עצמנו את לשאול יכולים שאנו הראשונה השאלההעצמיים הערכים על לחסם הדרישה השני. מהמועמד יותר מהותי באופן טוב.null(K) = null(M) האפס מרחבי לשיוויון דרישה גוררת λ(K, M) המוכלליםדרישה לוידוא אינפורמציה מספיק מכילה לא באיור המומחשת הגיאומטריה


K M1 M2

קירוב להיות והמועמדים K הוא המקורי המודל במישור. משולשים :4 איורמהמודל. אלמנטים מספר השמטת ידי על נוצרו המועמדים .M2ו־ M1 הם עבורו

של מודל הוא המודל אם הבעיה. של פיזיקליים במאפיינים תלוי הדבר זאת.שווקטור כך כולו, קשיר החומר M2ב־ וגם M1ב־ גם אזי אלקטרוסטטית, בעיהמודל הוא המודל עם פוטנציאלים. של הקבוע הווקטור הוא היחיד האפסרק קיימים M1 במודל שונה; הוא האפס מרחב לינארית, באלסטיות בעיה שלמסביב נוספת חופשית תנועה קיימת M2 במודל בעוד במישור, והזזות סיבוביםלבדה קשירות או לבדה, שגיאומטריה מראה זאת דוגמא במעגל. המוקף לצירמספיק מכילות לא ובוודאי האפס מרחב לניתוח אינפורמציה מספיק מכילות אינן

קירוב. לבניית אינפורמציהבמסגרת מפורש גיאומטרי או מפורש פיזיקלי במידע משתמשים לא אנוכלליים לפחות האלגוריתמים את יהפוך זה מסוג במידע שימוש שלנו. העבודהעבור היחיד הקלט ספציפי. סופיים אלמנטים בפיתוח תלויים להיות להם ויגרוםעם מתמודדים אנו .Ke האלמנט מטריצות קבוצת הוא שלנו האלגוריתמים

הבא: באופן האפס מרחבי שאלת

Ke אלמנט מטריצות בין יחס להגדיר לינארית באלגברה משתמשים אנו .1משותפות. מאפס שונות עמודות בעלות

גרף. ליצור הזה המקומי ביחס משתמשים אנו .2

אלגבריות תכונות לקבוע הגרף של קומבינטוריות בתכונות משתמשים אנו .3.K המורכבת המטריצה של

נתבונן .(rigidity relationship) הקשיחות יחס של פורמלית לא הגדרה עם נתחילאם f לאלמנט ביחס קשיח e האלמנט סופי. אלמנט במודל fו־ e באלמנטיםe של בודדת אחת קשיחה לתנועה להרחבה ניתנת לבדו e של קשיחה תנועה כל

זאת. פורמלית לא הגדרה את ממחיש 5 איור .f עם ביחדשל האפס מרחב את מנתחים אנו פורמלי, באופן זה יחס את להגדיר על־מנת

.Ke טיפוסית סופי אלמנט מטריצת

אפס שאינן העמודות של האינדקסים קבוצת Ne ⊆ 1, . . . , n תהי 2.1 הגדרהשל (trivial null space) הטריביאלי האפס מרחב עמודות. n בעלת Ke במטריצה

הוא Ke

tnull(Ke) = x|Kex = 0 and xi = 0 for all i /∈ Ne ,

הוא Ke של (essential null space) העקרוני האפס מרחב

enull(Ke) = x|xi = 0 for all i ∈ Ne .

קשיחות של אלגברית תיאוריה 2.1 11

(א) (ב)

הם ב־(א) האלמנטים שני במישור. אלסטיים משולשים של זוגות שני :5 איורביחס אחד קשיחים אינם ב־(ב) האלמנטים שני לשני. ביחס האחד קשיחים

במבנה. מימין העליון לציר מסביב חופשית תנועה שקיימת מכיוון לשני,

ושהאיחוד לשני האחד מאונכים העקרוני והמרחב הטריביאלי המרחב כי ברורבאופן להגדיר מוכנים אנו עתה, המטריצה. של האפס מרחב את פורש שלהם

הקשיחות. יחס את פורמלי

וקטור כל עבור אם Kf למטריצה ביחס קשיחה היא Ke המטריצה 2.2 הגדרה.Ne ∩Nf ב־ מזדהים yו־ xש־ כך y ∈ enull(Kf ) יחיד וקטור קיים x ∈ enull(Ke)האחת קשיחות הן אם (mutually rigid ) הדדית קשיחות הן Kf ו־ Ke המטריצות

.Ke ./ Kf בסימון מסומנת הדדית קשיחות לשנייה. ביחס

פורמלית שהגדרה לב לשים ניתן .2 בפרק זה יחס את מפורט באופן מנתחים אנואפילו או למחצה חיובית מוגדרות בהכרח לא כלליות, מטריצות עבור היא זאתחישוב בעזרת לחלוטין אלגברי באופן לבדיקה ניתנת הדדית קשיחות סימטריות.מפורטת מעשית רוטינה וכן 2.3 בסעיף תיאורטית מתואר זה תהליך בלבד. מקומי

.3.3.1 בסעיףהקשיחות. יחס על המבוסס גרף שנגדיר לפני נוסף אלגברי לכלי זקוקים אנומנותחת לכך הסיבה בהמשך. התיאורטיות התוצאות רוב עבור נחוץ זה כלי

עצמה. בתיזה

S-) Sתואמת־ היא n × n בגודל מטריצה לינארי. מרחב S ⊂ Rn יהי 2.3 הגדרה

מזדהים xו־ y) y ∈ S יחידה הרחבה קיימת x ∈ enull(K) לכל אם ,(compatible

נמצא מאופסים NK ל־ מחוץ איבריו כל כאשר y ∈ S וקטור כל ואם (NK על.enull(K)ב־

קשיחה Kל־ קוראים אנו אז ,NK = 1, . . . , n וכן Sתואמת־ היא K אםקשירה. באלקטרוסטטיות: מכניזמים, חסרת באלסטיות: למשל, .(rigid )

של הלינארי המרחב S ⊂ R10 יהי למרחב. התאימות הגדרת את ממחיש 6 איור

באיור. המופיעות במישור הנקודות 5 עם מימדית דו באלסטיות והזזות סיבוביםהוא ו־(ג) (ב) (א), המודלים של העקרוני האפס מרחבי .3 היא S של הדרגהכולם הם לפיכך, בלבד. והזזות סיבובים מכילים שלהם האפס מרחבי .3 מדרגהסיבוב ומכיל 4 מדרגה הוא (ד) מודל של העקרוני האפס מרחב .Sל־ תואמיםS במרחב לוקטור מורחב להיות יכול אינו זה סיבוב העליון. הימני הציר סביבתואם אינו (ד) מודל לפיכך, במישור. הנקודות כלל של והזזות סיבובים המכיל

.Sל־שלנו. המרכזי הקומבינטורי המבנה את להגדיר מוכנים אנו עתה

למחצה חיובית מוגדרות סימטריות מטריצות K1, K2, . . . ,Kk יהיו 2.4 הגדרהG = (rigidity graph) הקשיחות גרף .S כלשהו למרחב התואמות n × n בגודל


(א) (ב) (ג) (ד)

מטריצות במישור. אלסטיים משולשים של שונים מודלים ארבה :6 איורהסיבובים את המכיל S ⊂ R

10 עבור .10× 10 כולן הן המתאימות הקשיחותתואמי הם (א)־(ג) מודלים במישור, נקודות 5 עם מימדית דו באלסטיות והזזות

.S תואם אינו (ד) ומודל S

(א) (ב)

גרף את מראה (א) חלק במישור. משולשים של הקשיחות גרף :7 איורעבור הקשיחות גרף את מראה (ב) חלק אלסטיים, משולשים עבור הקשיחות

מוליך. מחומר העשויים משולשים

ו־ V = K1, . . . ,Kk עם מכוון הלא הגרף הוא עבורן (V,E)

E = (Ke, Kf ) |Ke ./ Kf .

משולשים המייצגות מטריצות של אוספים שני עבור הקשיחות גרף את מציג 7 איורשל ממשולשים עשוי (ב) והאוסף אלסטיים ממשולשים עשוי (א) האוסף במישור.

מוליך. חומרקשיחות. גרפי אודות המרכזיות הקומבינטוריות התוצאות הם הבאים המשפטים

המתאים. הגוף לקשיחות הגרף קשירות את משייך הראשון המשפט

תואמות מטריצות של K1, K2, . . . ,Kk האוסף של הקשיחות גרף G יהי 2.5 משפטאם .K =

∑ke=1 Ke יהי .n × n בגודל למחצה חיובית ומוגדרות סימטריות S

אזי ,⋃k

e=1NKe= 1, . . . n אם בפרט, .S תואמת הינה K אזי קשיר, G הגרף

קשיחה. היא Kו־ null(K) = S

באופן אבל הגוף, לקשיחות מספיק תנאי הינה הגרף שקשירות לב לשים ניתןזה. נושא את מדגים 2.7 סעיף הכרחי. תנאי אינה כללי

הדדית. לקשיחות גרפים תתי של קשירות משייך השני המשפט

תואמות מטריצות של K1, K2, . . . ,Kk האוסף של הקשיחות גרף G יהי 2.6 משפטשל קשירים גרפים תתי שני H2ו־ H1 יהיו למחצה. חיובית ומוגדרות סימטריות S

הם C =∑

Kf∈V (H2)Kf ו־ B =

∑

Kf∈V (H1)Kf אזי, .Ke צומת חולקים אשר G

הדדית. קשיחים

המסורית קירוב 2.213

(א) (ב) (ג)

מסורית־עץ הרחבת בניית :8 איור

המסורית קירוב 2.2

של קשיחות לבדיקת מספיקים תנאים נותנת הקודם בסעיף שהוצגה התאוריהגישה .Kל־ קירוב לבניית בה להשתמש רוצים אנו עתה, מודלים. ותתי מודליםהמקוריים. האלמנטים של קבוצה מתת המורכבת במטריצה להשתמש היתה טבעיתלקשיחות המספיקים התנאים הקשיחות. גרף מתוך צמתים לזריקת מיתרגם הדברהמזל, לרוע אפס. מרחב אותו בעל הוא המתקבל הקירוב האם לבדוק עוזרים היו

לפירוק. קלים הינם זה מסוג שקירובים להבטיח דרך מצאנו לאקוראים אנו קשתות. להסיר החלטנו הקשיחות מגרף צמתים להסיר במקוםכאן נותנים אנו .(fretsaw extensions) מסורית הרחבות המתקבלים לקירוביםמסורית־עץ הרחבת הנקרא המסורית, הרחבת של לווריאנט פורמלי לא תיאורבמישור. דו־מימדי אלסטי גוף עבור הבנייה את מדגים 8 איור .(fretsaw-tree)שלו הקשיחות גרף את מחשבים אנו האיור. של (א) בחלק מוצג המקורי המודל.G של T פורש עץ מחשבים אנו עתה, האיור. של (א) בחלק הוא גם המוצג Gאלמנטים בין הקשתות את מסירים אנו עתה, העץ. את מציג האיור של (ב) חלקחדשים משתנים הוספת ידי על זאת עושים אנו .T של קשת אין שבניהם שכניםהאלמנט, מטריצות של התוכן את משנים לא אנו ;(slack-variables עזר (משתנימשתנים בזהירות ומחליפים עזר) (משתני חדשות ועמודות שורות להן מוסיפים רקעבור המשתנים הוספת תהליך את מדגים 9 איור החדשים. במשתנים קיימיםהמשתנים. הוספת לאחר המודל את מראה 8 איור של (ג) חלק פשוט. מודלעוד. מחובר אינו החומר איפה להראות מנת על קלות כווצו המשולשים באיור,המקורית. המטריצה של F(K) קירוב לכדי החדש המבנה את מרכיבים אנו עתה,במידע שימוש ללא לחלוטין קומבינטורי־אלגברי באופן לביצוע ניתן התהליך

מפורש. גיאומטרימערכת לפתור צורך יש איטרטיבי, פיתרון בתהליך בקירוב שימוש לצורךהקירוב .(n × n (כלומר, K כמימד הוא M מימד כאשר Mz = r משוואותמרחיבים אנו בו, להשתמש מנת על .m > n כאשר ,m ממימד הוא F(K) שלנואנחנו עתה, אפסים. (n − m)ב־ מורחב r הוקטור r יהי אפסים. עם r אתnל־ z את מגבילים אנו לבסוף, .F(K)z = r המשוואות מערכת את פותריםשימוש שצורת להראות ניתן .z מקורב פיתרון ומקבלים המקוריים המשתניםכמטריצת Schur(F(K)) Schur במשלים לשימוש שקולה קירוב במטריצת כזאת[18, בתהליך) שנוספו העזר (משתני החדשים המשתנים של אלימינציה לאחר קירוב,

.70, 24]כי מציינים אנו המסורית. הרחבת של פורמלית הגדרה כאן נותנים לא אנו

המסורית קירוב 2.2 14

אנו השמאליים. האלסטיים המשולשים שני בין הקשיחות קשת ניתוק :9 איורשל עמודות ושתי שורות שתי 10× 10 בגודל המקורית למטריצה מוסיפיםאת מחליפים אנו מכן, לאחר .(x11, x12 העזר משתני את (מוסיפים אפסיםוהעמודות השורות עם השמאלי העליון המשולש של ו־2 1 ועמודות שורות

ו־12. 11 החדשות

של במקרה כמו קשירים שאינם קשיחות גרפי עבור גם מוגדרת המסורית הרחבתמוגבל אינו מסורית הרחבת של הקשיחות גרף כן, על יתר מסורית־עץ. הרחבת

מעגלים. חסר מבנה או עץ להיותהן: המסורית הרחבות עבור המרכזיות התיאורטיות התוצאות

מטריצות של אוסף הוא K1, K2, . . . ,Kk כאשר K =∑k

i=1 Ki תהי 2.7 משפטהרחבת F(K) תהי .n× n בגודל למחצה חיובית ומוגדרות סימטריות S תואמותF(K) (כלומר, F(K) ביצירת שנוספו העזר משתני מספר ` יהא .K של מסוריתn) (n + `) × n בגודל יחידה מטריצת Q תהי .((n + `) × (n + `) בגודל הואSchur(F(K)) ויהי ((n+ `)× (n+ `) בגודל יחידה במטריצת הראשונות העמודותכי נניח העזר. משתני של אלימינציה לאחר שנוצר F(K) של Schur משלים

אזי: קשיר, K1, K2, . . . ,Kk של הקשיחות גרף אם .NK = 1, . . . , n

הדדית. קשיחות F(K)ו־ QKQT .1

.null(F(K)) ⊆ null(QKQT ) .2

.null(K) = null(Schur(F(K))) אזי קיים, Schur(F(K)) אם .3

בניות עבור קשיר שאינו קשיחות גרף של למקרה להרחבה ניתן זה משפטהפורמלי הניסוח את מסירים אנו מורכבותו מפאת מסורית. הרחבת של מסוימות

מהתקציר. שלולעבור יכולה עץ על המבוססת מסורית הרחבת כי מראה הבאה התוצאה

לינארי. בזמן פירוק לעבור ולפיכך התמלאות ללא דליל פירוק

מטריצות של אוסף הוא K1, K2, . . . ,Kk כאשר K =∑k

i=1 Ki תהי 2.8 משפטהרחבת F(K) תהי .n× n בגודל למחצה חיובית ומוגדרות סימטריות S תואמותאפס שאינם האיברים שכל כך ,Γ פרמוטציה מטריצת קיימת .K של מסורית־עץi, j ∈ NΓQeKeQT

e ΓT מקיימים Γ(F(K) + I)ΓT של L Choleskyה־ בפקטור Li,j

.(jו־ iב־ תלוי e) כלשהו e עבור

את לחסום עלינו יעיל, קירוב תהליך עבור שהצגנו הדרישות את לספק על־מנתידי על λ(K,Schur(F(K))) ≥ 1 כי מראה הבאה הלמה .λ(K,Schur(F(K)))

.λ(Schur(F(K)), K)ל־ עליון חסם מתן

סופי אלמנט בעיות של האפס מרחב .315

חיובית ומוגדרות סימטריות S תואמות מטריצות K1, K2, . . . ,Kk יהיו 2.9 למה.K =

∑ki=1 Ki המטריצה של מסורית הרחבת F(K) תהי .n × n בגודל למחצה

מתקיים מוכלל עצמי ערך כל עבור אזי ,Schur(F(K)) קיימת אם

λ(Schur(F(K)), K) ≤ 1 .

בעזרת כלל בדרך מושג זה חסם .λ(K,Schur(F(K))) עבור כללי חסם לנו איןבאמצעות העצמיים הערכים את החוסמת ,[18, 45] (path lemma) מסלול״ ״למת

כללי. S עבור מסלול״ ״למת להוכיח הצלחנו לא .path embedding טכניקותואלקטרוסטטיים אלסטיים מודלים על מסורית־עץ הרחבת עם ניסויים ביצענוהאיטרטיביים הפותרים כי מראים שלנו הניסוייים מימדיים. ותלת דו־מימדייםהדבר מסורית־עץ. בקירוב משתמשים כאשר אחיד בקצב מתכנסים minresו־ cg

התוצאות .[37] SDD למטריצות Vaidya של הקירובים של ההתכנסות לאופן דומהלא Cholesky פירוק קירוב של לתוצאות דומות ולפעמים יותר טובות לפעמים הן

התמלאות. ללא שלםהם ספציפי. מקרה עבור מסלול למת Daitchו־ Spielman פיתחו [45] במאמרממשולשים העשוי מימדי דו (truss) מסבך המייצגות סופי אלמנט מטריצות חקרוהמסורית הרחבת בטכניקת השתמשו הם מסוימים. גיאומטריים אילוצים עםשלהם הפותר .[53] נוספות קשתות עם המורחבים מתוחכמים עצים עם שלנוn כאשר ,O(n5/4(log2 n log logn)3/4 log 1

ε ) בזמן ורץ מוכחת יעילות בעל הואלא עדיין בפותר. הנדרשת היחסית השגיאה הוא εו־ במסבך הצירים מספר הוא

מימדי. התלת למקרה מסלול למת קיימתלהרחבת נוספים שימושים לחפש החלטנו λ(K,Schur(F(K))) על כללי חסם ללאדורש שאינו ליישום המסורית להרחבת שימוש מראה הבא הסעיף המסורית.

.λ(K, Schur(F(K))) על חסם

סופי אלמנט בעיות של האפס מרחב 3

לינאריים אילוצים 3.1

K = המטריצה ,S ריק לא למרחב תואמות Ke האלמנט מטריצות כל כאשרכל עדיין קשיר, הקשיחות גרף אם גם סינגולרית. היא n × n בגודל

∑

e Ke

לינארית, באלסטיות למשל, .K של האפס במרחב וקטורים הם Sב־ הווקטוריםבחלל. לצוף יכול עדיין הוא קשיח, הגוף כלומר קשיר, הקשיחות גרף אם גם

.K של האפס במרחב נמצאים בחלל והזזות סיבוביםבעלת אינה Kx = f המשוואות מערכת רצויה. תכונה אינה זאת כלל, בדרךc × n בגודל C מטריצה של הוספה ידי על לרוב נפתר הדבר בודד. פיתרוןלקיים, x הפיתרון וקטור שעל לינאריים אילוצים המייצגים ,c בגודל d ווקטורהם לינארית באלסטיות למשל, שפה. תנאי ממדלת C המטריצה .Cx = d כלומרזה מסוג אילוצים וכדומה. מחליקים מכניזמים ריתוכים, צירים, להיות: יכולים

.[76] NASTRAN הנפוצה בתוכנה משמשים

האפס מרחב חישוב 3.2

Computer Methods in Appliedל־ לאחרונה שהוגש מאמר על המבוסס ,3 בפרקשל האפס מרחב לחישוב שיטה מציגים אנו ,[137] Mechanics and Engineering

מערכת לפיתרון שיטה כמו שימושית אינה השיטה .KC =[

KT CT]T

ושל Kשימושי: הוא שלה יעיל יישום אבל משוואות,

האפס מרחב חישוב 3.216

(א) (ב)

בקשתות מתואר הקשיחות גרף מסורית־יער. הרחבת לבניית דוגמא :10 איור(א) בחלק המקורי המבנה כהות. בקשתות מתואר שלו הפורש והיער בהירות

.3 בפרק מופיעים מלאים פרטים (ב). בחלק למבנה הורחב

מדרגה אינה המטריצה אם .KC של הדרגה לחישוב לשמש יכולה השיטה .1להתכנס. להיכשל או ריצה כדי תוך להיכשל עלול לינארי פותר מלאה,

למשל, ,[62] צפים גופים של ודינמי סטטי בניתוח לשמש יכולה השיטה .2אבל קונסיסטנטית משוואות מערכת לפתרון משמש האפס מרחב כאשר

סינגולרית.

[84, 148, smooth aggregationב־ אופרטורים לבניית לשמש יכולה השיטה .3domain- ושיטות [57, 55] FETI עבור שימושית להיות יכולה השיטה ;149]

.[109] קשורות decomposition

וב־[120]. ב־[56] נחקרה קשיחות מטריצות של האפס מרחב חישוב של הבעיהאנו לבעיה. ספציפי שהינו פיזיקלי או גיאומטרי במידע השתמשו המקרים, בשני

טהורה. אלגברית־קומבינטורית בדרך זה חישוב מבצעיםויוצרים המטריצה עבור מסורית הרחבת מחשבים שאנו הוא הבסיסי הרעיוןאז, מאוד. מהיר באופן לפירוק הניתנת זהה אפס מרחב עם אחרת מטריצהinverse) הופכיות איטרציות בעזרת האפס מרחב את ביעילות לחשב יכולים אנומלבנית) שהיא (שייתכן המטריצה בפירוק כרוך הדבר .[68, 89] (iterations

שלה. LUה־ פקטורי עם איטרציות מספר ובביצועהרחבת מחשבים אנו ,C ריקה) (אולי ומטריצה אלמנטים כאוסף K בהינתןאנו קשיר. לא להיות יכול K של הקשיחות גרף .K עבור F(K) מסוריתאנו הקשיחות. גרף של פורש יער על המבוססת מסורית בהרחבת משתמשיםלכל מסורית הרחבת תלוי בלתי באופן המחשב אלגוריתם בעזרת זאת עושיםfretsaw-forest) מסורית־יער הרחבת כזאת להרחבה קוראים אנו קשירות. רכיבבעמודות C את מרפדים אנו מכן, לאחר כזאת. הרחבה מציג 10 איור .(extension

של N האפס מרחב את ומחשבים (F(K) ב־ העזר למשתני (המתאימים אפסים

[

F(K)C 0

]

.

להסיר די ,KC של האפס מרחב את לחשב מנת על כי מראה הבאה הלמה.Nב־ החדשים למשתנים המתאימות השורות את

מוגדרות סימטריות מטריצות אוסף הוא Ke כאשר ,K =∑

e Ke תהי 3.1 למהבגודל מטריצה C ותהי ,S כלשהו למרחב התואמות n×n בגודל למחצה חיוביות(כלומר, (n + `) × (n + `) בגודל K של מסורית־יער הרחבת F(K) תהי .c × nמטריצה N ותהי ,(n+`)×n בגודל יחידה מטריצת Q תהי עזר), משתני ` הוספנו

ניסוייות ותוצאות ריצה זמן 3.317

span(QT N) = אזי, .null(

[

F(K)CQT

]

) עבור בסיס הן שעמודותיה (n+ `)× s בגודל

.null(KC)

מרחב מייצר ההופכיות האיטרציות תהליך נומריות, מסיבות רבים, במקריםוקטורים עם יחד משולב F(K) של האפס מרחב את מכיל אשר N לינאריבקלות, נפתרת הבעיה מדי, גדול אינו זה מרחב של הגודל עוד כל נוספים.

הבאה. בלמה כמתואר

(n+ בגודל מטריצה N תהי למעלה, המוגדרים Cו־ Q ,F(K) ,K עבור 3.2 למה

V תהי .null(

[

F(K)CQT

]

) את המכיל לינארי מרחב פורשות שעמודותיה `) × s

.span(QT NV ) = null(KC) אזי ,span(V ) = null(KCQT N) ש כך מטריצה

ניסוייות ותוצאות ריצה זמן 3.3

בזמן המסורית־יער הרחבת את שמחשב האלגוריתם את בפירוט מציג 3 פרקבעלות שורות רק מכילה Cו־ קשיר הוא Ke של הקשיחות גרף אם לינארי.עם F(K) והסימטרית הריבועית המטריצה כי מובטח אזי אפס, שאינו בודד איברהרחבת למעשה (היא התמלאות ללא לפירוק ניתנת משורשרת CQT המטריצההדבר לינארי. בזמן עובד התהליך כלל זה, במקרה כן, על יתר מסורית־עץ).

.3.4.1 בסעיף המתוארים בניסויים הודגםשל מאחד גדול מספר עם שורות מכילה C טיפוסיים תעשייתיים במודליםאין זה במקרה קשיר. תמיד אינו הקשיחות גרף כמו־כן, אפס. שאינם איבריםהשיטה של הריצה זמן ועל LU בפירוק הפקטורים גודל על תיאורטי חסם לנוLU בפירוק שהפקטורים כך על מעידה הקיימת התיאוריה זאת, למרות שלנו.המקורית. למטריצה יחסית לפירוק קלים להיות אמורים מסורית־יער הרחבת שלהפירוק שלם, תאורטי בניתוח המחסור שלמרות מעידות שלנו הניסוייות התוצאות

יקר. לא אכן הוא

[

F(K)CQT

]

של

מודלים על שלנו השיטה של ניסויים של מפורטות תוצאות מכיל 3 פרקהשיטה את משווים אנו אמיתיים. תעשייתיים מודלים ועל פשוטים סינתטיים.KC המקורית המטריצה על הופכיות איטרציות המבצעת ישירה לשיטה שלנו

הישירה. מהשיטה מובהק באופן טובה שלנו שהשיטה מראים הניסוייםחישבנו השיטות. בשתי האפס מרחב חישוב דיוק את גם בחנו לביצועים, בנוסףאת בחנו N עמודות ידי על הנפרש ריק לא מרחב עבור היחסיות; לשגיאות קירוביחד ריק שאינו מרחב התקבל וכאשר C ללא .‖KCN‖/maxi,j |KC(i, j)| הערךהישירה. לשיטה בהשוואה דיוק ספרות ל־5 2 בין מפסידה שלנו השיטה C עםזאת. זיהתה תמיד כמעט שלנו השיטה סינגולרית, לא להיות KCל־ גרם C כאשרהשיטה וגם שלנו השיטה שגם כזאת במידה ill conditioned היו בעיות מספר

האפס. ומרחב המטריצה דרגת בחישוב אמינות היו לא הישירה

אליפטיות סקלאריות בעיות עבור סופי אלמנט מודלי 4

SDD למטריצות הקירוב גישת 4.1

במטריצות מתרכזת סופי אלמנט למטריצות קירובים לבניית שלנו הבאה השיטהאת כאן נתאר סקלאריות. אליפטיות חלקיות דיפרנציאליות במשוואות הנוצרותישיר. באופן זה בניסוח משתמשת לא שלנו שהעבודה למרות הרציפה הבעיה

SDD למטריצות הקירוב גישת 4.118

PDE K L M

(א) (ב) (ג) (ד)

המשוואה את מתאר (א) חלק .SDD למטריצות הקירוב גישת :11 איורהסופי האלמנט מודל את מתאר (ב) חלק המקורית. החלקית הדיפרנציאליתהמטריצה לנעלם. מתאים צומת וכל בודד Keל־ מתאים משולש כל המתאים;נגדים של משולש כל .Kל־ L SDDה־ קירוב את מציג (ג) חלק .K =

∑

e Ke

.L עבור קירוב ,M את מציג (ד) חלק .L =∑

e Le ו־ בודד Leל־ מתאים

מצא ,Γ = Γ1 ] Γ2 שפה עם ,Rdב־ חסומה פתוחה קבוצה שהוא Ω תחום בהינתןהמקיימת u : Ω −→ R

∇ · (θ(z)∇u) = −f on Ωu = u0 on Γ1 ,

θ(z)∂u/∂n = g on Γ2 .(1)

בתחום, המשתנה d×d בגודל חיובית מוגדרת סימטרית מטריצה היא θ המוליכותשפה תנאי היא gו־ Dirichlet שפה תנאי היא u0 סקלארית, פונקציה היא f

כזה. תחום מציג 11(א) איור .Neumann

מטריצת זה. מסוג מטריצות עבור קירובים לבניית גישה ממחיש 11 איורדומיננטי אלכסון בעלת סימטרית מטריצה ידי על מקורבת K הסופי האלמנטSDD מטריצות עבור M סטנדרטי קומבינטורי קירוב מכן, לאחר .L (SDD)לשימושית. זאת גישה ההופכות אלגבריות אבחנות שתי קיימות .L לקירוב משמש

,[24] מתוך הבא השיוויון אי היא הראשונה

λmax(K, M)

λmin(K, M)≤ λmax(K, L)

λmin(K, L)· λmax(L, M)

λmin(L, M).

טוב קירוב היא Mו־ K עבור טוב קירוב היא L אם כי מראה זה שיוון איקירובים קיימים ,1.3 בסעיף כמתואר .K עבור טוב קירוב היא M אז ,L עבורלקירוב ניתנות SDD מטריצות כן, על יתר .SDD למטריצות טובים קומבינטורייםלמצוא רק נותר בסיסי, באופן .[132] אלגבריות multigrid שיטות בעזרת יעיליעיל. פותר לקבל על־מנת K עבור טוב קירוב וגם SDD שהיא L מטריצההפיצול למת זה. את בדיוק לעשות לנו מאפשר השנייה האלגברית האבחנהאלמנט־אחר־אלמנט. להיבנות יכול L הקירוב כי מראה [18] (splitting lemma)

הבאים: החסמים את נותנת הלמה ,L =∑

e∈E Leו־ K =∑

e∈E Ke עבור

λmax(K, L) ≤ maxe∈E

λmax(Ke, Le)

λmin(K, L) ≥ mine∈E

λmin(Ke, Le) . (2)

שלנו העבודה 4.219

למספר כבסיס שימשו זה, בניסוח בהכרח שלא למרות אלה, אבחנותהשתמשה [77, 128, 102] ושותפיו Reitzinger של הראשונה העבודות קבוצת עבודות.הפותר כן, על יתר .Le ידי על Ke כל לקרב יקרים ודי מורכבים באלגוריתמיםשל השנייה העבודות קבוצת בודד. אלמנט בקירוב לקשיים רגיש היה שלהם[153] Sarinו־ Wang ושל [74] Gupta של ,[25] Vavasisו־ Hendrickson ,Bomanאלמנטים פיתוח תלוי או פיזיקלי, גיאומטרי, מידע בעזרת Le המטריצות את בונהלבעיות מוגבלות אלה עבודות לפיכך, הנתונות). Keל־ (בנוסף מסוים סופייםשנערכו המוגבלים בניסויים לעין נראית זאת מגבלה בלבד. מסוימים ולפיתוחיםמסוימים ולפיתוחים לאלמנטים מוגבלות הניסוייות (התוצאות אלה. לקירובים

בלבד).


SIAM Journal on Matrix Analysis andב־ שיופיע מאמר על המבוסס ,4 בפרקוסטודנט (אני אלה. עבודות של המגבלות את מסירים אנו [9] Applications

משמעותית תרומה תרמנו אברון, חיים תל־אביב, באוניברסיטת לדוקטורט נוסףזה.) למאמר

Ke המטריצות לקירוב לאופטימלי, קרוב חדשני, באלגוריתם משתמשים אנו .1מתאים ולכן לחלוטין אלגברי הוא שלנו האלגוריתם .Le מטריצות ידי על

אלמנט. סוג לכל

קירוב. יכולת של התיאורטית השאלה את חוקרים אנו .2

קירוב. קשיי עם להתמודד יודע שלנו הפותר .3

יסודי. השוואתי מחקר וביצענו מעשי פותר מימשנו .4

בעיית של חדשנית רדוקציה על מבוסס לאופטימלי הקרוב הקירוב אלגוריתםעל ומלבנית) (ייתכן מטריצה של שורות משקול לבעיית Le בעזרת Ke קירובבין היחס הוא התנאי מספר זה, (במקרה שלה. התנאי מספר את למזער מנתקירוב באלגוריתם משתמשים אנו המטריצה). של הקיצוניים הסינגולריים הערכיםמייצר שלנו הקירוב אלגוריתם Ke כל עבור פתרון. לקבל כדי [147] לבעיה קיים

מקיימת המטריצה לחישוב. זולה בצורה Lnearly-opte SDD מטריצה

λmax(Ke, Lnearly-opte )

λmin(Ke, Lnearly-opte )

≤ n2e2· λmax(Ke, L

opte )

λmin(Ke, Lopte )

,

מטריצת היא Lopte ו־ Keב־ אפס שאינן והעמודות השורות מספר הוא ne כאשר

באופן נוספות. קירוב שיטות וניתחנו הצגנו כמו־כן, האופטימלית. SDDה־למרות קטן. תנאי מספר בעל Ke אם רק טובים קירובים מייצרות כולן כללי,לשיטת דוגמא כאן נותנים אנו מובטח. אינו הדבר המצב, זה רבים שבמקרים

.(positive-part ) חיובי חלק קירוב קוראים אנו לה נוספת מסוימת קירוב

את נגדיר (למחצה). חיובית מוגדרת סימטרית מטריצה K תהי 4.1 הגדרההדומיננטי האלכסון עם הבאה הסימטרית המטריצה להיות K+המטריצה


(K+)ij =

kij i 6= j and kij < 0

0 i 6= j and kij ≥ 0∑

k 6=j − (Ke+)ik i = j .

טוב לקירוב K+ את ההופך דבר קטן, הוא κ(K, K+) רבים במקרים כי מסתבר.κ(K, K+) עבור תיאורטי חסם מציינת הבא הלמה .Kל־

עם ne × ne בגודל למחצה חיובית מוגדרת סימטרית מטריצה K תהי 4.2 למהκ(K, K+) ≤ וכן ,null(K+) = span[1 . . . 1]T אזי .null(K) = span[1 . . . 1]T

,‖K‖inf ≤ cKii עבורו i ואינדקס c קבוע קיים אם כן, על יתר .√

neκ(K).κ(K, K+) ≤ cκ(K) אז

לקירוב ניתנים האלמנטים כל ״האם היתה: עצמנו את ששאלנו הבאה השאלהמספר בעלת היא Ke אם למעלה, שמודגם כפי SDD?״. מטריצות ידי על טובבעלות מטריצות מציגים אנו 4 בפרק טוב. לקירוב ניתנת היא אזי קטן, תנאיבעזרת טוב לקירוב ניתנות שעדיין ,SDD ממטריצות רחוקות גדול, תנאי מספר

טוב. SDD קירוב להן שאין מטריצות מציגים אנו בנוסף, .SDD מטריצותהבסיסית שהשיטה כך על מצביע טוב קירוב להן שאין אלמנט מטריצות קיוםמסוימים. במקרים להיכשל עלולה K את לקרב כדי אלמנט־אחר־אלמנט קירוב שלאינו בודד אלמנט אם אפילו לרופפים הופכים 2 במשוואה התיאורטיים החסמיםאם גם כי, הבחנו בניסויים תיאורטית, אבחנה רק אינה זאת טוב. לקירוב ניתןהנצפה האיטרציות מספר לקירוב, ניתנים אינם האלמנטים של יחסית קטן חלק

גבוה. היה בקירוב בשימוש האיטרטיביים הפותרים שלאלמנט כל עבור .12 באיור המומחשת הפיצול, גישת הוא זאת לבעיה הפיתרוןהקרוב־ שלנו הקירוב באלגוריתם בשימוש Le בעזרת Ke את מקרבים אנו ,eהיחס אם .λmax(Ke, Le)/λmin(Ke, Le) היחס את מחשבים אנו אז, לאופטימלי.קירוב. ללא שהיא כמו Ke את משאירים אנו ,t מראש הנקבע קבוע מערך גדולסכום K =

∑

>t Keו־ קירוב שעברו המטריצות סכום L =∑

<t Le תהי עתה,M בעזרת L את מקרבים אנו שהן. כפי ונשמרו קירוב עברו שלא המטריצותכקירוב γM + K ב־ משתמשים אנו קומבינטורי. קירוב באלגוריתם בשימושהשיטה עצמו. בטקסט ומוסבר למדי טכני הינו γ המישקול גורם .K למטריצהלהיפוך קשה הקירוב המזל, לרוע המוכללים. העצמיים הערכים על חסם מבטיחה

מדי. גדול K אםמחקר וביצוע שלנו הפותר של מעשי מימוש הוא זו עבודה של האחרון החלק[111] Matlab של cmexה־ בממשק והשתמשנו Cב־ הקוד רוב את מימשנו השוואתי..[37] taucs מתוך Vaidya של בקירוב השתמשנו הביצועים. בדיקות לביצוע כממשקשל cholmod 1.0 דליל Cholesky פירוק באמצעות שלנו הקירובים את פירקנו.[95] metis מתוך התמלאות המקטין ועמודות שורות סידור עם יחד ,Tim Davisלמול ,(metis עם cholmod 1.0) ישיר פותר למול שלנו הפותר את השוונוpreconditioned drop-tolerance) זריקה סף עם שלם לא Cholesky פירוק פותרבחנו .(BoomerAMG [79]) אלגברי multigrid פותר ולמול (incomplete Choleskyלהלן אלמנטים. סוגי מגוון עם מימדיים תלת מודלים מגוון על הפותרים את

התוצאות: של מקוצר תיאור

עקביות תוצאות ייצר הוא ;(robust) איתן היה שלנו הפותר כללי באופן .1תמיד היה לא הוא משתנים. בעיה וגדלי משתנים פרמטרים עבור ורציפות

ואמין. צפוי אבל ביותר, המהיר

משטחים לשחזור סופיים אלמנטים ניסוח .521

K γL+ K γM + K

(א) (ב) (ג)

הקשיחות מטריצת את מציג (א) חלק הפיצול. גישת עם SDD קירוב :12 איורחלק של SDD מטריצות בעזרת הקירוב ,L את מציג (ב) חלק .K המקוריתלקירוב ניתן שאינו K של החלק לתוך ומסוכם γב־ מוכפל זה קירוב .Kמ־Kל־ ומסוכם ממושקל L של M הקירוב את מציג (ג) חלק .Kב־ המסומן

.K לכלל קירוב ליצירת

שלנו הניסויים מחשב לזיכרון להיכנס כדי דיין קטנות היו הבעיות כאשר .2אחרת. שיטה לכל מעבר ומהיר אמין הכי היה הישיר הפותר ,(8GB (בערך

האלמנטים כל כאשר גבוה, תנאי מספר עם כאלה אפילו גדולות, בעיות עבור .3האלגברי multigridה־ פותר אבל טוב, עבד שלנו הפותר קירוב, ברי היו

יותר. טוב עבדו שלם הלא הפירוק ופותר

גם (ולכן לקירוב ניתנים שאינם אלמנטים היו כאשר גדולות, בעיות עבור .4פותר מאוד, רע תפקד האלגברי multigridה־ פותר גבוה), תנאי מספר בעלי

ביותר. המהיר היה שלנו והפותר אמין לא היה שלם הלא הפירוק

משטחים לשחזור סופיים אלמנטים ניסוח 5

שמקורה משטחים שחזור לבעיית סופיים אלמנטים ניסוח מתארים אנו ,5 בפרקבאופן משטח לשחזור נוספות טכניקות עם יחד זה, פיתוח ממוחשבת. בגרפיקהזאת עבודה מטרת .[136] SIGGRAPH 2007 ב־ כמאמר פורסם אינטראקטיבי,ורועש. חלקי סריקה מידע מתוך מימדי תלת חלק 1watertight משטח שחזור היאThomas שארף, אנדרי ידי על ופותחו נחקרו בעבודה הממוחשבת הגרפיקה היבטיבעיה לפיתרון שלי בתרומות מתרכזים אנו זה בסעיף אור. כהן ודניאל Lewinerבעיית של יעילה ובנייה לבעיה הסופיים האלמנטים פיתוח התאמת כלומר זאת,

הפחותים. ריבועיםאו המשטח על הנמצאת PS נקודות קבוצת היא לבעיה ההתחלתי הקלטחורים בו יש מסוימים, איזורים של חסר מדגימת סובל הקלט אליו. קרובכזה משטח שחזור בעיית כן, על יתר .(outliers) שגויות דגימות שאף וייתכןוליד דרך העוברים משטחים אינסוף קיימים היטב; מוגדרת שאינה בעיה היאמיוצר המקורי, בקלט בבעיות ולטפל בהירויות אי לפתור על־מנת הקלט. נקודותהקלט המשתמש. ידי על לשליטה ניתן שגם קלט אוטומטי, באופן נוסף קלט

וחוץ פנים היטב המגדיר חורים או רווחים ללא סגור 1משטח

משטחים לשחזור סופיים אלמנטים ניסוח .522

(א) (ב) (ג)

(ד) (ה) (ו)

מימדים. בשני המשטח שחזור תהליך של המחשה :13 איור

watertightה־ המודל בתוך נקודות P In נקודות: של קבוצות משתי מורכב הנוסף

לו. מחוץ נקודות POutו־מוצגת המקורי הדבש יונק דמות מימדים. בשני התהליך את ממחיש 13 איורהחסר לדגימת לב לשים ניתן (ב). בחלק מתוארות הסריקה נקודות (א). בחלק

נקודות שונים. בצבעים POutו־ P In את מתאר (ג) חלק העליונה. ובכנף במקורבטרנספורמציית מקומי מקסימום מציאת ידי על אוטומטי באופן נוצרו אלה(ד) חלק הדגומות. הנקודות של (unsigned distance transform) מכוונת לא מרחקהעליונה בכנף העיוות בהיר. בצבע זה) במקרה (קו השחזור תוצאת את מראהטופולוגית שונה שחזור קיבלנו המקור בסביבת כן, על יתר לזיהוי. קל ובמקור

על עריכתן לאחר POutו־ P In הקבוצות את מראה (ה) חלק הרצוי. מהשחזור(ו). בחלק המוצג סביר לשחזור מובילות אלה חדשות קבוצות המשתמש. ידי

לשחזר הוא ,[98] ושותפיו Kazhdan אצל כבר המופיע בשחזור, הבסיסי הרעיוןהמשטח עצמו. המשטח את רק במקום מימדי התלת התחום בכלל u שדהטרנספורמציית d תהי השדה. של (zero isosurface) אפס הערך של כמקור יתקבלמימדית הדו בדוגמא הסריקה. נקודות מתוך המחושבת מכוונת הלא המרחקים

רוצים אנו שלנו

u(x, y) ≈ 0 for (x, y) ∈ PS

u(x, y) ≈ d(x, y) for (x, y) ∈ P In

u(x, y) ≈ −d(x, y) for (x, y) ∈ POut.

בוחרים אנו לחלקה. u הפיכת על זאת נשיג חלק. יהיה שהמשטח גם רוצים אנוחלקות לחוסר הבאה העונש פונקציית את

Ψ(u) =1

2

ˆ ˆ

[

(

∂u

∂x

)2

+

(

∂u

∂y

)2]

dx dy .

משטחים לשחזור סופיים אלמנטים ניסוח .5 23

פוטנציאל מייצג u כאשר המשתחררת האנרגיה בדיוק היא זאת עונש פונקציהתהיה זאת אבחנה שלנו. התחום בצורת מוליך אחיד בחומר אלקטרוסטטי

בהמשך. שימושיתלפיתרון הניתנת בדידה לבעיה הרציפה הבעיה את להפוך היא הבאה המטלהמימד). בתלת (טטרהדרים משולשים של (mesh) רשת M תהי מחשב. באמצעותצומת כל עבור סקלר המכיל ,uM וקטור כעת היא u שלנו הרציפה הפונקציההמקיימת K = ET E מטריצה מייצרים אנו הסופי האלמנט שיטת בעזרת ברשת.פואסון לבעיית הקשיחות מטריצת בדיוק זאת .Ψ(uM ) ≈ uT

MKuM = ‖EuM‖22צמתים בהכרח לא הן POutו־ P In ,PSב־ הנקודות שפה. תנאי ללא סקלאריתמייצגים אנו טטרהדרים). (או משולשים בתוך נמצאות הן זאת, עם יחד ברשת.4 או 3 (עם cp לינארי אינטרפולציה וקטור ידי על p בנקודה u של הערך אתאנו .uM ב־ מוכפל בהתאמה) טטרהדרים או משולשים עבור אפס שאינם ערכים

לנקודה. אילוץ כל עבור משקל ־ wp מגדירים גםהפחותים הריבועים מערכת לכדי האילוצים כלל את לשלב יכולים אנחנו עתה

הבאה

minuM

∥

∥

∥

∥

∥

∥

∥

∥

∥

∥

∥

Ew1c1w1c2...

w1c|P |

uM −

0w1t(p1)w2t(p2)

...w|P |t(p|P |)

∥

∥

∥

∥

∥

∥

∥

∥

∥

∥

∥

2

2

,

p ∈ PS או p ∈ POut ,p ∈ P In עבור 0 או −d(pi) ,d(pi) ל שווה t(pi) כאשרהנורמליות המשוואות פירוק ידי על ישירות זאת בעיה פותרים אנו בהתאמה.של (update/downdate) העדכון ברוטינות משתמשים גם אנו .cholmod בעזרת

.POut ו־ P In-ב נקודות במהירות ולהסיר להוסיף כדי cholmod של הפקטוריםהשימוש רגולריות. לא רשתות עם לעבוד לנו מאפשר הבעיה של זה ניסוחהאופטימיזציה בעית כי מבטיחות לינארית ובאינטרפולציה סופיים אלמנטים בניסוחנבנית עובדים אנו עמה הספציפית הרשת כאלה. רשתות עבור טוב באופן מוצגת

P In ,PS) הנקודות ענן מעל נבנה אדפטיבי octree בראש, בשלב שלבים. בשני

.octreeל־ דואלית (טטרהדרית) משולשית רשת בונים אנו השני, בשלב .( POutו־רחוק מאוד וגסה העניין נקודות ליד מאוד עדינה רשת לקבל לנו מאפשר הדברהפיתרון לפתור. שצריך הלינארית הבעיה את להקטין מאפשר הדבר מהן.ברשת. צמתים אלף מ־70 פחות יש כלל בדרך כאשר ישיר, פותר בעזרת מתבצעקטנות בעיות עבור ביותר והיעיל האמין הוא ישיר פותר ,1.4 בסעיף שמוסבר כפיעדינה ברשת להשתמש צריכים היינו רגולרית, ברשת משתמשים היינו אם כאלה.תהליך את והופך מאוד גדולה לינארית למערכת מוביל היה הדבר התחום. כל על

אינטראקטיבי. ללא השחזורשל התרומה והם בתיזה מכוסים שאינם נוספים היבטים בעלת זאת עבודהזיהוי (1) שלמות: עבור כאן אותם מציינים אנו .[136] של האחרים הכותביםמנגנון (2) בהם, אילוצים להוסיף המשתמש והנחיית טופולוגית חלשים איזוריםאילוצים הוספת המאפשר (Tablet scribbling mechanism) טבליות על שרבוטאחרות. שחזור לשיטות ניסויית השוואה ו־(3) מימד, בתלת אינטראקטיבי באופן

חצוי משולשי ליכסון תלת .6 24

חצוי משולשי ליכסון תלת 6

משולשי ליכסון תלת 6.1

מטריצה פירוק של התהליך הוא (triangular tridiagonalization) משולשי ליכסון תלתהבא באופן A סימטרית

PT AP = LTLT ,

מטריצה היא T יחידה, אלכסון עם תחתונה משולשית מטריצה היא L כאשרשל כזה פירוק בהינתן פרמוטציה. מטריצת היא P ו־ אלכסונית תלת סימטרית

פתרון ידי על Ax = b הלינארית המערכת את לפתור ניתן ,A

PT APPT x = PT b

LTLT PT x = PT b .

עם התחתונה, המשולשית L עם פיתרון ידי על לפיתרון ניתנת השנייה המשוואההעליונה. המשולשית LT ועם האלכסונית התלת T

הפירוק סימטריות. למטריצות משתמשים בו הנפוץ הפרוק אינו זה פירוקהוא הפירוק .Bunch-Kaufman פירוק הוא הנפוץ

PT AP = LDLT ,

עם בלוק־אלכסונית סימטרית מטריצה היא D תחתונה, משולשית היא L כאשרסימטרי בלוקים ליכסון לעיתים נקרא זה מסוג פירוק ו־2×2. 1×1 בגודל בלוקים

.(symmetric block-diagonalization)את חושף Bunch-Kaufman פירוק גישה. לכל וחסרונות יתרונות קיימיםמציאת לבעיית הבעיה את הופך רק הליכסון תלת בעוד המטריצה, של האינרציהמעט וקשה D עם לינארית מערכת לפתור קל כן, על יתר .T של האינרציהמאחוריו והתאוריה pivotingה־ תהליך שני, מצד .T עם מערכת לפתור יותרBunch- בשיטת מהמקבילה בהרבה פשוטים וכן פשוטים מאוד הם ליכסון בתלתידי על ליכסון בתלת חסום Lב־ לאלכסון מתחת האיברים גודל בנוסף, .Kaufman.[8] דיוק לבעיות ולהוביל לגדול עלולים הם Bunch-Kaufman שבשיטת בעוד ,1

מנמק [7] Andresonו־ Dongarra מאת 1989 משנת lapack של עבודה מסמךבכך lapackב־ (partitioned) החצוי Bunch-Kaufman באלגוריתם הבחירה אתLTLT פירוק לחישוב החצוי הקוד טובים. לא LTLT פירוק של שהביצועיםלמצוא אתגר מציב הדבר .[8] אבד וכנראה פורסם לא זאת לבדיקה ששימש

זו. פירוק שיטת עבור יעיל חצוי אלגוריתםלשיטת חצוי ומימוש ניסוח קיים היה לא הפרוייקט על העבודה את כשהתחלנוהדיוק בעיות עם להתמודד [8] ב שפותחה ,(Bounded) החסומה Bunch-Kaufmanלעבודה נוספת כמוטיבציה שימש הדבר המקורית. Bunch-Kaufman שיטת שלתקשורת באמצעות גילינו [131] ב לפרסום העבודה הגשת לאחר לאחרונה, זאת.Bunch- לאלגוריתם חצויה גירסה מימשו ואחרים הוא כי Nick Higham עם פרטית

חצוי. חסום Kaufman(היחסית מעורבותי במהלך הוצת מלאים סימטריים בפירוקים שלי האישי הענייןעל העבודה במהלך .[114] indefinite דליל ישיר פירוק פותר במימוש מינורית)השתמשתי בה ,Bunch-Kaufman בשיטת הנומריות לבעיות מודע נעשיתי זה פותראלה, בעיות בעקבות הדליל. הפותר בתוך מלאים בלוקים לפירוק עזר כרוטינתלעבוד החלטתי ,indefinite סימטריות מטריצות עבור חצוי מדוייק פותר והיעדרלמרות דלילים, לינאריים לפותרים קשורה לפיכך, שלי, התרומה זה. פרוייקט עלל־ שהוגש מאמר על המבוסס ,6 פרק קומבינטוריים. לקירובים קשורה שאינה

זאת. תרומה מציג ,[131] ACM Transactions on Mathematical Software

א־סימטריות למול סימטריות מצומצמות מטריצות 6.225

א־סימטריות למול סימטריות מצומצמות מטריצות 6.2

טבעי. די באופן משולשי ליכסון תלת מבצע [121] Reidו־ Parlett של האלגוריתםמשתנה בין המחליפה פרמוטציה מטריצת P2 תהי ,A סימטרית מטריצה בהינתן.A2:n,1ב־ ביותר הגדול המקדם מופיע בה השורה היא k כאשר ,k למשתנה 2

לכתוב נוכל עתה,

P2APT2 =

A1,1

[

Ak,1vT]

[

Ak,1

v

]

B

,

בגודל סימטרית מטריצה היא Bו־ (n − 2) באורך עמודה וקטור הוא v כאשרעם התחתונה המשולשית המטריצה את להגדיר נוכל כעת, .(n − 1) × (n − 1)

היחידה אלכסון

L2 =

10 10 `2 In−2

,

את נגדיר אפסים, כולו אם אפסים. כולו אינו A2:n,1 אם `2 = −v/Ak,1 כאשרכי לראות קל .(n− 2) באורך אפסים כוקטור `2

L2P2APT2 LT

2 =

A1,1

[

Ak,1 0 . . .]

Ak,1

0...

C

,

זה (reduction) צמצום תהליך עם להמשיך יכולים אנחנו סימטרית. היא C כאשרתלת סימטרית מטריצה נקבל צעדים (n− 2) ואחרי C הסימטרית המטריצה עלמשולשיות ומטריצות פרמוטציה מטריצות של השזורה המכפלה .T אלכסוניתשל בודדת מכפלה ידי על מיוצגת להיות יכולה יחידה אלכסון עם תחתונות

יחידה. אלכסון עם תחתונה משולשית ומטריצה פרמוטציה מטריצתהעמודה יצירת לאחר ;(right-looking) צופה־ימינה הוא Parlett-Reid של הפירוקמדרגה־1 עדכונים שני מבצע הוא המטריצה. שארית את מעדכן הוא Lב־ הבאההעדכונים את להפעיל ניתן סימטרית, היא המטריצה ששארית כיוון צעד. בכלפעולות. 2

3n3 +O(n2) המבצע תהליך מתקבל דבר של בסופו בלבד. מחציתה על

לא מטריצות עבור גאוס בתהליך הפעולות למספר דומה זה פעולות מספר(והיעיל) החצוי באלגוריתם כשימושית תתברר זאת גישה זאת, למרות סימטריות.

שלנו.בתקציר .6 בפרק בפירוט מתואר הוא יותר. מורכב [1] Aasen של האלגוריתםעובד Aasen של האלגוריתם זה. אלגוריתם של תכונות מספר רק נזכיר זההתחתונה Hessenbergה־ מטריצת היא H כאשר ,PT AP = HLT המשוואה עלמעודכנת עמודה ;(left-looking) שמאלה הצופה בשיטה עובד האלגוריתם .LTלפרט מבלי (מחושבת). מפורקת שהיא לפני רגע הקודמות העמודות באמצעותבעלות התלת־ליכסון תהליך את לבצע Aasenל־ מאפשר זה ניסוח כי נציין כיצד,

פעולות. 13n

3 +O(n2) שלנסמן נוחות, לשם

A[11] = A1:k,1:k ,

A[12] = A1:k,(k+1):n ,

A[21] = A(k+1):n,1:k ,

A[22] = A(k+1):n,(k+1):n .

חצוי ניסוח 6.326

.Hו־ T ,L עבור דומה ובאופןשהמטריצה היא Aasen של השיטה עבור חצוי אלגוריתם בפיתוח האתגרL[22]T [22] L[22]

Tהמצומצמת המטריצה כמו סימטרית אינה H [22] L[22]

Tהמצומצמת

לעבוד טריביאלי לא שזה היא הדבר משמעות .Reidו־ Parlett של באלגוריתםk את ביעילות ״חשב רבים: חצויים לאלגוריתמים המתאים רקורסיבי ניסוח עםשארית את לעדכן מנת על הראשונות העמודות kב־ השתמש הראשונות, העמודותהמטריצה שארית על רקורסיבי באופן לעבוד המשך עתה ,A(k+1):n,(k+1):n המטריצההמצומצמת) (המטריצה המעודכנת המטריצה שארית ,Aasen של במקרה המעודכנת״.

סימטרית. אינה

חצוי ניסוח 6.3

לבין Aasen של בתהליך המצומצמת המטריצה בין פשוט קשר קיים למזלנו,קשר מנסחים רק אנו כאן, .Reidו־ Parlett של בתהליך המצומצמת המטריצה

:6 בפרק מפורט באופן מוסבר הוא , זה

L[22]T [22] L[22]T

= H [22] L[22]T − L

[21]:,k T

[12]k,:1 L

[22]1,:

T

= A[22] −H [21] L[21]T − L

[21]:,k T

[12]k,:1 L

[22]1,:

T.

שונה H [22] L[22]TAasen של המצומצמת המטריצה כיצד מראה הראשונה השורה

מדרגה בעדכון רק L[22]T [22] L[22]TReidו־ Parlett של המצומצמת מהמטריצה

.1אנחנו שלב בכל השנייה. השורה על מבוסס שלנו החצוי האלגוריתמי הצעדאנחנו שמאלה. הצופה Aasen אלגוריתם בעזרת עמודות k של קבוצה מעבדיםמדרגה עדכון בעזרת A[22] המטריצה שארית של ימינה צופה עדכון מבצעים

מטריצה מקבלים אנחנו .L[21]:,k T[12]k,:1 L

[22]1,:

T1 מדרגה ועדכון H [21] L[21]

Tk

את להמשיך נוכל עתה .Reidו־ Parlett כמו L[22]T [22] L[22]T

סימטרית מצומצמתל־ הקשורים עדינים נוספים פרטים קיימים המצומצמת. המטריצה על התהליךהמצומצמת המטריצה מחצית את רק לעדכן מעוניינים שאנחנו ולעובדה pivotingכאן. שתואר כפי בדיוק היא האלגוריתם של המהות זאת, עם יחד הסימטרית.

ידי על שנתרמה שלנו החצוי האלגוריתם של נומרית ליציבות הוכחה לנו ישלאחור יציב הוא שלנו החצוי האלגוריתם .[131] למאמר Miroslav Rozloznıkהמקורי לחסם מאד דומה חסם עם (component wise backward stable) איבר־איבר

.[81] Aasen של האלגוריתם על

ביצועים 6.4

משווים כאשר עמודות k כל עבור 1 מדרגה נוסף עדכון מבצעת שלנו השיטהלפיכך הוא הכללי הריצה זמן המקורי. Aasen לאלגוריתם אותה

1

3

(

1 +1

k

)

n3 +O(n2k) .

מדרגה והעדכון 1 מדרגה העדכון את איחדנו כאשר שלנו, השיטה את מימשנומשמעות .BLAS3 רוטינות באמצעות המתבצע (k + 1) מדרגה בודד לעדכון kשלנו הפירוק .BLAS3 לרוטינות מחוץ מתרחשות פעולות O(n2k) שרק היא הדברהפירוק במהלך Givens סיבובי באמצעות T של הפירוק את גם זול באופן מחשב

ביצועים 6.427

ב־ Bunch-Kaufman ופיתרון פירוק למימוש שלנו המימוש את השוונו .A שלוגם שלנו הפירוק רוטינת גם הניסויים. של מפורט תיאור מכיל 6 פרק .lapack

.lapackב־ המתאימות לרוטינות למדי דומה בצורה מתפקדות הפיתרון רוטינתיעילות להיות יכולות סימטרי לכסון תלת שיטות כי מראות שלנו התוצאותמערכת פתרון של הנוספת העלות כן, על יתר סימטרי. בלוקים ליכסון שיטות כמולהעדיף בביצועים התלויה סיבה אין כי מראה הדבר זניחה. היא אלכסונית תלת

סימטרי. ליכסון תלת שיטות פני על סימטרי בלוקים ליכסון שיטות

combinatorial preconditioners for finite element...

Documents