Digital Object Identifier (DOI) 10.1007/s00211-004-0568-zNumer. Math. (2004) Numerische

Mathematik

Stopping criteria for iterations in finite elementmethods

M. Arioli1, D. Loghin2, A. J. Wathen3

1 Atlas Centre, Rutherford Appleton Laboratory, Oxon OX11 0QX, UK;e-mail: m.arioli@rl.ac.uk

2 CERFACS, 42 ave G. Coriolis, Toulouse, 31057, France;e-mail: loghin@cerfas.fr

3 Oxford University Computing Laboratory, Parks Road, Oxford, OX1 3QD, UK;e-mail: wathen@comlab.ox.ac.uk

Received March 21, 2003 / Revised version received February 25, 2004Published online November 26, 2004 – c© Springer-Verlag 2004

Summary. This work extends the results of Arioli [1], [2] on stopping cri-teria for iterative solution methods for linear finite element problems to thecase of nonsymmetric positive-definite problems. We show that the residualmeasured in the norm induced by the symmetric part of the inverse of thesystem matrix is relevant to convergence in a finite element context. We thenuse Krylov solvers to provide alternative ways of calculating or estimatingthis quantity and present numerical experiments which validate our criteria.

Mathematics Subject Classification (2000): 65N30, 65F10, 65F35

1 Introduction

Iterative methods of Krylov subspace type form a well-known and well-researched area in the context of solution methods for large sparse linearsystems. In some cases, convergence can be described, in others not. Invari-ably however, the theoretical and practical convergence criterion is chosento be the Euclidean norm of the residual, with the ubiquitous exception ofthe Conjugate Gradient method, where the ‘energy norm’ lends itself quitenaturally to analysis. On the other hand, finite element methods which are animportant source of large, sparse linear systems provide a natural norm forconvergence. While this fact is well-known and has been noted particularlyin the case of symmetric positive-definite problems [10], [11], [16], [19] only

Correspondence to: M. Arioli

M. Arioli et al.

recently have there been attempts to relate convergence in the ‘energy’ normto the finite element context [7], [8], [9], [18], [1], [2].

In this work we consider the choice of stopping criteria for nonsymmetricpositive-definite problems. The immediate difficulty encountered is that ofdefining a suitable norm in which to measure convergence. In the case ofsymmetric positive-definite problems, the energy or A-norm of the error isequal to the dual norm or A−1-norm of the residual, which is the quantity thatis estimated. In the nonsymmetric case, we show that a useful definition ofdual norm is the norm induced by the symmetric part of A−1. We show thatone can also work with the norm induced by the inverse of the symmetricpart of A for problems which are not too non-normal.

The paper is structured as follows. In Section 2 we describe the problemsetting. In Section 3 we derive general stopping criteria while in Sections 4and 5 we present ways of approximating the criteria introduced in the case ofGMRES; we also consider the effect of preconditioning and derive the corre-sponding modified bounds. Finally, in Section 6 we investigate the stoppingcriteria by performing experiments on various discretizations of convection-diffusion problems.

2 Problem description

2.1 Abstract formulation

Consider the weak formulation

Find u ∈ H such that for all v ∈ H

a(u, v) = f (v),(1)

where H is a Hilbert space of functions u defined on a closed subset � of Rd ,

with dual H′ and inner-product norm ‖ · ‖H, while a(·, ·) is a nonsymmetric,positive-definite bilinear form on H×H and f (·) ∈ H′ is a continuous linearform on H. Existence and uniqueness of solutions to problems of type (1) isguaranteed provided the following conditions hold for all u, v ∈ H

a(w, v) ≤ C1‖w‖H‖v‖H(2a)

a(v, v) ≥ C2‖v‖2H,(2b)

with constants C1, C2 independent of discretization. In the following, we takeC1, C2 to be the smallest and, respectively largest, such constants.

Stopping criteria for iterations in finite element methods

Condition (2b) is often used to replace – and implies – the weaker andsufficient conditions of Babuska ([3])

supv∈H\{0}

a(w, v)

‖v‖H≥ C2‖w‖H,(3a)

supw∈H\{0}

a(w, v)

‖w‖H≥ C2‖v‖H;(3b)

this is due to the fact that the weak formulation (1) with a(·, ·) replaced byits symmetric part is often stable in the sense of Babuska (i.e., satisfies (3)),leading to (2b).Finally, we note that the continuity condition (2a) implies that the bilinearform defines a continuous operator A : H → H′ with norm ‖A‖H→H′ ≤ C1.

2.2 Finite element approximation

An approximation to problem (1) is sought through projection onto a finite-dimensional space Hh ⊂ H; the resulting formulation reads

Find uh ∈ Hh such that for all vh ∈ Hh

a(uh, vh) = f (vh).(4)

Finite element methods choose Hh to be a space of functions vh defined on asubdivision �h of � into simplices T of diameter hT ; h denotes a piecewiseconstant function defined on �h via h|T = hT .

Since Hh ⊂ H, (2) are satisfied with constants independent of h; thus,there exists a unique finite element approximation uh. Moreover subtracting(4) from (1) yields the standard orthogonality condition for all vh ∈ Hh

a(u − uh, vh) = 0,(5)

which can be used (together with conditions (2)) to derive standard errorestimates of the form

‖u − uh‖H ≤ C1

C2inf

vh∈Hh

‖u − vh‖H.(6)

Remark 1 Replacing vh with the interpolant of u and using interpolationerror estimates leads to a priori bounds of the form

‖u − uh‖H ≤ C(h)C(u)

where C(u) is typically a constant depending only on u and its derivatives.

M. Arioli et al.

Conditions (2) can also be used in determining a posteriori error bounds. Inparticular, if we define the functional residual as a linear functional via

〈R(uh), v〉 := f (v) − a(uh, v) = a(u − uh, v) ∀v ∈ H

then dividing by ‖v‖H and using (3a) and (2a) leads, respectively, to thefollowing upper and lower bounds on the error

1

C1‖R(uh)‖H′ ≤ ‖u − uh‖H ≤ 1

C2‖R(uh)‖H′(7)

where

‖R(uh)‖H′ := supv∈H\{0}

|〈R(uh), v〉|‖v‖H

= supv∈H\{0}

|a(u − uh, v)|‖v‖H

Alternatively, noting that ‖A‖H→H′ ≤ C1 (cf. (2a)) we can rewrite (7) as

BE ≤ ‖u − uh‖H‖uh‖H

=: FE ≤ C1

C2BE(8)

where

BE := ‖R(uh)‖H′

‖uh‖H‖A‖H→H′(9)

Definition 1 The quantities FE,BE in (8),(9) are the functional forwardand backward error respectively [2].

Remark 2 The dual norm of the functional residual, ‖R(uh)‖H′ , is not easyto compute and most a posteriori error bounds are derived as approximationsof this quantity. However, in general it is known that ‖R(uh)‖H′ and thus BEare (polynomial) functions of the discretization parameter h and thus far frombeing close to machine precision. This is our main motivation for seekingnew, improved stopping criteria. However, we will not be concerned herewith the derivation of any error bounds but we will assume the followinggeneric bound on the relative error

‖u − uh‖H‖uh‖H

≤ C(h)(10)

where C(h) is available via an a priori or a posteriori error analysis.

It is evident from the above description that approximate solutions anderrors on one hand and residuals and the right hand side data on the otherbelong to spaces with different topologies: the trial space H and its dual.Moreover, the operator A has a domain different from its range. We chooseto preserve these essential features in our discrete formulation of the problem.To this aim, we introduce the following notation and results.

Stopping criteria for iterations in finite element methods

We first define a matrix norm ‖ · ‖H1,H2 : Rn×n via

‖M‖H1,H2 := maxx∈Rn\{0}

‖Mx‖H2

‖x‖H1

where M ∈ Rn×n and Hi ∈ R

n×n, i = 1, 2, 3 are symmetric and positive-definite matrices. This choice of norm will allow us to consider matrices asoperators for which domain and range are equipped with different norms. Wealso note here that

‖M‖H1,H−12

= ‖H−1/22 MH

−1/21 ‖,(11)

where ‖ · ‖ denotes the standard Euclidean norm.We now state a result which can be found in [4].

Theorem 1 Let M ∈ Rn×n be nonsingular and let H ∈ R

n×n be a symmetricand positive-definite matrix. Then

‖M‖H,H−1 = maxw∈Rn\{0}

maxv∈Rn\{0}

wT Mv‖w‖H‖v‖H

,

‖M−1‖−1H−1,H

= minw∈Rn\{0}

maxv∈Rn\{0}

wT Mv‖w‖H‖v‖H

.

The above result justifies the following definition.

Definition 2 The H -condition number of a matrix M is

κH (M) := ‖M‖H,H−1‖M−1‖H−1,H .

We now turn to the discrete setting for the framework described above.Expanding uh in a basis of Hh, we can derive a linear system of equationsinvolving the coefficients (u)i, i = 1, . . . , n of uh in our choice of basis ofHh

Au = f(12)

where n = dim Hh and A ∈ Rn×n is a non-singular, generally nonsymmet-

ric, matrix. In fact there is an isomorphism �h between Rn and Hh which

associates to every vector v ∈ Rn a function vh ∈ Hh via

�hv =n∑

i=1

viφi,

where {φi, i = 1, . . . , n} form a basis for Hh. Henceforth, given a vectorv ∈ R

n we will denote its functional counterpart �hv by vh. Note also thatthe above choice of basis defines a norm-matrix H via

Hij = ((φi, φj ))

M. Arioli et al.

where ((·, ·)) is the H-inner product. Hence

‖vh‖H = ‖v‖H = (vT Hv)1/2.

With this notation, the stability conditions (2) become

maxw∈Rn\{0}

maxv∈Rn\{0}

wT Av‖w‖H‖v‖H

≤ C1(13a)

minv∈Rn\{0}

vT Av

‖v‖2H

≥ C2(13b)

It is easy to see that there also exists a constant C3 ≤ C1 such that

maxv∈Rn\{0}

vT Av

‖v‖2H

≤ C3.(13c)

Remark 3 In many situations of interest one can have C3 = C2. Moreover,if the symmetric part of A is H , then C2 = C3 = 1.

Finally, we derive the discrete versions of (8), (9) for the case where Hand its dual are replaced by Hh and its dual; moreover, we assume that weseek an approximation u ∈ Hh to the solution u of the linear system (12).Given the basis {φi, i = 1, . . . , n} for Hh, the discrete dual, H′

h, is spanned

by a dual basis{φ′

i , i = 1, . . . , n}, defined via

⟨φi, φ

′j

⟩= δij . As before,

there exists an isomorphism �′h between R

n and H′h defined similarly via

�′hv′ = ∑n

i=1 v′iφ

′i; moreover,

⟨v, v′⟩ = vT v′. Thus, one can define the resid-

ual R(u) = �′h(b − Au), as a linear functional from H′

h into Hh with norm

‖R(u)‖H′h

:= supv∈Hh\{0}

|〈R(u), v〉|‖v‖Hh

= maxv∈Rn\{0}

∣∣(b − Au)T v∣∣

‖v‖H

= ‖b−Au‖H−1 .

The above formalism motivates the following definitions: given an approx-imation u to the solution u of the linear system Au = f we define discreteforward and backward errors via (cf. (8), (9))

FE := ‖u − u‖H

‖u‖H

, BE := ‖f − Au‖H−1

‖u‖H‖A‖H,H−1.(14)

In the following section (cf. Thm 2), we show that, as in the continuous case,a similar upper bound holds on the forward error FE ≤ C1

C2BE. This is an

expected result since (cf. (13), Thm 1)

‖A‖H,H−1 ≤ C1, ‖A−1‖−1H−1,H

≥ C2,(15)

and hence for all n

κH (A) ≤ C1

C2.(16)

Stopping criteria for iterations in finite element methods

In other words, we recover the well-known result of linear algebra [12]

forward error ≤ condition number × backward error

but with respect to the norms inherited from the problem formulation.

3 Stopping criteria

In many large-scale computations the exact solution u of the linear system(12) is out of reach and an iterate u is used to approximate the solution. Sincewe identify u with a function uh ∈ Hh, we naturally expect a useful iterateu to satisfy an error estimate similar to (10)

‖u − uh‖H‖uh‖H

≤ C(h),

where C(h) is of the same order as C(h) in (10). Our aim is to derive asufficient and computable criterion for the above error bound to hold. First,we introduce some notation and useful results. Let M ∈ R

n×n. We denote byHM = (M +MT )/2, SM = (M −MT )/2 the symmetric and skew-symmet-ric parts of M , respectively. Moreover, if HM is positive-definite, it inducesa norm which we denote by

‖ · ‖M := ‖ · ‖HM.

We first prove the following results.

Lemma 1 Let conditions (13) hold. Then

1√C3

‖r‖A ≤ ‖r‖H ≤ 1√C2

‖r‖A

and √C2

C1C3‖r‖H−1 ≤ ‖r‖A−1 ≤ 1√

C2‖r‖H−1 .

Proof See Appendix. � Theorem 2 Let u be the solution of the weak formulation (1) and let u, uh =�hu satisfy

Au = f; ‖u − uh‖H‖uh‖H

≤ C(h).

Then uh = �hu satisfies

‖u − uh‖H‖uh‖H

≤ C(h) = O(C(h))

M. Arioli et al.

if

‖f − Au‖H−1

‖u‖H

≤ ηC(h)C2,(17)

for some η ∈ (0, 1).

Proof Let r = f − Au. We have

‖u − uh‖H‖uh‖H

≤ ‖u − uh‖H‖uh‖H

‖uh‖H‖uh‖H

+ ‖uh − uh‖H‖uh‖H

≤ C(h)

(1 + ‖uh − uh‖H

‖uh‖H

)+ ‖uh − uh‖H

‖uh‖H

and since

‖uh − uh‖H‖uh‖H

= ‖A−1r‖H

‖u‖H

≤ ‖A−1‖H−1,H‖r‖H−1

‖u‖H

≤ 1

C2

‖r‖H−1

‖u‖H

(using (15) )

we get

‖u − uh‖H‖uh‖H

≤ C(h)(1 + ηC(h)) + ηC(h) =: C(h).

� Remark 4 The stopping criterion (17) is equivalent to requiring the discretebackward error BE defined in (14) to be of the same order as the functionalbackward error BE = O(BE). This is also a sufficient condition for thediscrete forward error FE corresponding to our iterative solution to have thesame order as the functional forward error FE .

In fact, criterion (17) can be replaced with a tighter bound. By Lemma 1,

‖A−1r‖H ≤ 1√C2

‖A−1r‖A ≤ 1√C2

‖A−1‖A−1,A‖r‖A−1 = 1√C2

‖r‖A−1

and thus, we can replace the bound (17) with

‖f − Au‖A−1

‖u‖H

≤ ηC(h)√

C2.(18)

The difference between the stopping criteria (17), (18) is not significant if theH -condition number (16) is not too large. This can be seen from the equiv-alence between ‖ · ‖H−1 and ‖ · ‖A−1 provided by Lemma 1. In particular, if

Stopping criteria for iterations in finite element methods

the symmetric part of A is H , then C2 = C3 = 1 and the effective conditionnumber is C1. A large value of C1 corresponds to a ‘highly nonsymmetric’problem for which the use of criterion (18) rather than (17) may be preferable.We return to this issue in the numerics section.

3.1 One more crime

In practice, the discretization of the weak formulation (1) is generally done inan approximate fashion, very often due to the computational costs involved.This approximation has been qualified as a variational crime [20], as it leadsto a perturbed system

(A + �A)u = f .

However, it is known that if the perturbation �A is suitably small (usuallywithin the finite element error), then the approximate solution u satisfiesthe same error estimates as the exact solution u [20]. In this context, theproposed stopping criteria represent but another variational crime as the fol-lowing standard result shows (see also [2] or [17] for the case when lp normsare employed.)

Theorem 3 Let u satisfy

‖f − Au‖H−1

‖u‖H

≤ ηC(h)C2.

Then there exists �A such that

(A + �A)u = f

and‖�A‖H,H−1 ≤ ηC(h)C2

Proof See [2, Thm 1]. � Remark 5 The more general case where the right-hand side f is perturbed istreated in [2]. We do not include the results here since in most engineeringapplications bounds of type (10) are preferred.

The stopping criteria derived above pose the problem of estimating the resid-ual in the H−1- or A−1-norms. While this was possible for the symmetricand positive-definite case in a natural way (see [10], [11]), the use of a non-symmetric iterative method does not allow for the same methodology to beapplied.

In the remainder of the paper we show how this norm can be estimatedusing the information contained in the Krylov space Kk. For simplicity, we

M. Arioli et al.

will consider only the case H = (A + AT )/2 (and thus C2 = 1), i.e., thecase when H defines the so-called ‘energy norm’ for the problem. In the nextsection, we show how the norm estimation is achieved for GMRES and FOM.Finally, in the case of central preconditioning with H , these two algorithmsreduce to a three-term recurrence which computes directly ‖rk‖H−1 . This willbe the subject of Section 5.

4 Stopping criteria for GMRES and FOM

We recall here some of the basic facts and standard notation for the GMRESand FOM algorithms. The methods compute an orthonormal basis of Kk; thebasis elements are the columns of Vk ∈ R

n×k. This orthonormalization isachieved via an Arnoldi process which yields the factorizations

V Tk AVk = Hk, AVk = Vk+1Hk

where Hk ∈ Rk×k, Hk ∈ R

k+1×k are upper Hessenberg matrices, with Hk

being obtained from Hk by deleting its last row. In the case of GMRES, aQR-factorization of Hk is computed (updated at each step)

Hk = QkRk.

4.1 Estimation of ‖rk‖H−1

This can be done simply via

‖rk‖H−1 ≤ λ−1/2min (H)‖rk‖.

Depending on the application, the smallest eigenvalue of H may or may notbe estimated with sufficient accuracy. If we do not have such an estimate,we must content ourselves with estimates provided by the iterative process.In the case of GMRES and FOM this can be achieved as follows. Assumingno early termination, the method computes the following factorization of A

involving an orthonormal matrix Vn

V Tn AVn = Hn.

Thus, V Tn HVn = (Hn + HT

n )/2 =: Hsn and therefore

λmin(H) = λmin(Hsn).

Since in practice we wish to use the algorithm only for a small number ofsteps k, an estimate of λmin(H) can be taken to be λmin(H

sk ). Unfortunately,

this estimate is always an upper bound on λmin(H). In fact, we have thefollowing monotonicity result.

Stopping criteria for iterations in finite element methods

Lemma 2 Let Hsk = (Hk + HT

k )/2. Then

λmin(Hsk+1) ≤ λmin(H

sk ).

Proof

λmin(Hsk ) = min

qk∈Rk

qTk Hkqk

‖qk‖2

= minqk∈Rk

qTk V T

k AVkqk

‖qk‖2

= minr∈Kk

rT Ar

‖V Tk r‖2

≥ minr∈Kk+1

rT Ar

‖V Tk r‖2

Now, any r ∈ Kk+1 can be written as r = Vk+1qk+1 for some qk+1 ∈ Rk+1

and hence

λmin(Hsk ) ≥ min

qk+1∈Rk+1

qTk+1V

Tk+1AVk+1qk+1

‖V Tk Vk+1qk+1‖2

≥ minqk+1∈Rk+1

qTk+1Hk+1qk+1

‖ qk+1‖2

= λmin(Hsk+1).

� This result enables us to approximate the stopping criteria as follows. Sinceby the previous lemma λmin(H

sk ) ↘ λmin(H) monotonically, there exists a

k∗ and a constant C∗ = C∗(k∗) such that λmin(H) ≥ C∗λmin(Hsk ) for all

k > k∗. Hence, our stopping criterion becomes

‖rk‖ ≤ C∗λ1/2min(H

sk )ηC(h).(19)

Thus, we only have to compute λmin(Hsk ) and estimate C∗. In practice, the

constant C∗ is of order one for small values of k∗. We investigate this issuein the next section.

Remark 6 Estimating λmin(Hsk ) can be done easily in the case of the FOM

algorithm. However, in the case of GMRES this is not necessarily straight-forward, since we do not store Hk but the Rk factor of the QR-factorizationof Hk. In this case, a further approximation could be introduced

λmin(Hsk ) ≤ σmin(Hk) ≤ σmin(Hk) = σmin(Rk)

M. Arioli et al.

leading to the bound

‖rk‖ ≤ C∗σ 1/2min (Rk)ηC(h),

where C∗ is a constant which accounts for both convergence to λmin(Hsk ) and

the difference between λmin(Hsk ) and σmin(Rk), which cannot be guaranteed

to be small and is not known a priori. However, this latter bound is useful inestimating ‖rk‖A−1 .

4.2 Estimation of ‖rk‖A−1

In this case we proceed similarly

‖rk‖ ≤ ‖rk‖σ−1/2min (A).

A similar monotonicity result holds for the singular values of Hk (cf. [14,Cor. 3.1.3])

σmin(Hk) ≥ σmin(Hk+1)

and thus there exists a k∗ and a constant c∗ = c∗(k∗) such that σmin(A) ≤c∗σmin(Rk) for all k > k∗. Thus the stopping criterion (18) can be replacedwith

‖rk‖ ≤ c∗σ 1/2min (Rk)ηC(h).(20)

where, as before, c∗ is a constant (of order one) which we need to estimate.

Remark 7 We note that this criterion can be used both in the case of GMRESand FOM, since in the first case the matrix Rk is available and in the secondcase Hk is available (with σmin(Hk) = σmin(Rk)).

4.3 Restarted GMRES/FOM

There are many situations where the construction of an orthonormal basis forthe Krylov subspace is limited to a small number of vectors. This leads to therestarted versions of GMRES or FOM. From the point of view of the abovestopping criteria, this does not pose any major problems – we still need toestimate either λmin(H) or σmin(A) and this is done in a similar fashion. Thus,assuming we run the algorithms for m iterations of k steps each, we use thefollowing approximations

λmin(H) ∼ min1≤i≤m

λ(i)min(H

sk ), σmin(H) ∼ min

1≤i≤mσ

(i)min(Rk)(21)

where we denote by λ(i)(H sk ), σ (i)(Rk), the eigenvalues and singular values

of the indicated matrices constructed at the ith iteration.

Stopping criteria for iterations in finite element methods

4.4 The effect of preconditioning

In the case where a preconditioner is used, the Arnoldi algorithm constructs asimilar factorization of the preconditioned matrix. We consider here only thecase of right preconditioning for which the GMRES/FOM residual remainsunchanged. The factorization is

AP −1Vk = Vk+1Hk

and since

‖r‖A−1 ≤ σ−1/2min (A)‖r‖ ≤ σ

−1/2min (AP −1)σ

−1/2min (P )‖r‖

we can derive a stopping criterion similar to (20) using the approximationσmin(AP −1) ∼ σmin(Rk)

‖rk‖ ≤ c∗σ 1/2min (Rk)σ

1/2min (P )ηC(h).(22)

However, this requires the estimation of the smallest singular value of P

which may not be easy to achieve. We address this issue in Section 6.

5 A minimum residual algorithm

We have seen that in the case of GMRES estimation of ‖rk‖H−1 can be doneprovided the Hessenberg matrix is stored. On the other hand, the more rel-evant quantity ‖rk‖A−1 can be estimated quite naturally during the GMRESprocess. However, there is one situation where working with ‖rk‖H−1 leadsto a three-term recurrence algorithm as well as useful preconditioning. Thealgorithm solves Au = f by minimizing ‖f −Au‖H−1 over the Krylov space.This is by no means a novel result and has been previously considered byConcus and Golub [6] and Widlund [21] in the context of preconditioningnonsymmetric matrices by their symmetric part. We consider below the ver-sion of this algorithm which minimizes the H−1-norm of the residual, whereH is the symmetric and positive-definite part of A.

Consider the modified problem

Au = f(23)

where A = H−1/2AH−1/2, f = H−1/2f . Let us consider first the FOM algo-rithm applied to this system. As before, the residual is orthogonal to theKrylov space

⟨rk, q

⟩ = ⟨H−1/2(f − Auk), q

⟩ = 0, ∀q ∈ Kk

where

Kk = span{

r0, Ar0, . . . , Ak−1r0}

= H 1/2Kk(H−1A, H−1r0).

M. Arioli et al.

Thus, ∀p ∈ Kk(H−1A, H−1r0)

⟨H−1/2rk, H 1/2p

⟩ = ⟨rk, p

⟩ = ⟨H−1rk, p

⟩H

= 0.

In other words, the standard FOM algorithm for (23) is also an orthogonal pro-jection method with respect to the H -inner-product onto Kk(H

−1A, H−1r0).Moreover, the advantage of this formulation is that there exists a three-termrecurrence which solves this problem. We summarize this below.

Lemma 3 Let A have symmetric and positive-definite part H . The FOMalgorithm applied to

(H−1/2AH−1/2)(H 1/2u) = H−1/2f .

in the Euclidean inner-product is equivalent to the FOM algorithm in theH -inner product applied to

H−1Au = H−1f .

Moreover, the Arnoldi orthogonalization process applied to the normal matrixH−1/2AH−1/2 yields a factorization

V Tk AVk = Hk,

where (Hk)ij = 0 for all |i − j | > 1.

Proof See Appendix. � This idea is contained in [21], although the author constructs a different tri-diagonalization than that constructed by FOM (Arnoldi). Similarly, using theabove result one can modify the standard GMRES algorithm into a three-term recurrence which constructs the solution with the smallest residual overKk as measured in the H−1-norm. We do not include the details here, butonly present in the next section numerical results obtained with this modifiedversion of GMRES.

Remark 8 The action of the inverse of H as a preconditioner can be relaxedin practice. Indeed, solving to an accuracy of order o(C(h)) is sufficient forconvergence of the algorithm. We explore this issue in the next section.

6 Examples

In this section we are interested in establishing explicit stopping criteria forthe generic example of finite element approximation of the solution of scalarelliptic equations.

Stopping criteria for iterations in finite element methods

Let � ⊂ Rd with boundary . We will be using the following norms:

‖v(x)‖L2(�) = ‖v(x)‖0 =(∫

�

v(x)2dx)1/2

‖v(x)‖L∞(�) = ‖v(x)‖∞ = ess supx∈�

|v(x)|

‖v(x)‖Hm(�) = ‖v(x)‖m =

∑

|α|≤m

∫

�

∣∣Dαv(x)∣∣2 dx

1/2

where

Dαv(x) = ∂ |α|v(x)

∂xα11 · · · ∂x

αd

d

= ∂α1x1

· · · ∂αdxd

and α = (α1, . . . , αd) is an index of order |α| = α1 +· · ·+αd . We also needto define the space H 1

0 (�)

H 10 (�) = {

v(x) ∈ H 1(�) : v(x)| = 0}

with norm

|v(x)|H 10 (�) = |v(x)|1 =

∑

|α|=1

∫

�

∣∣Dαv(x)∣∣2 dx

1/2

.

6.1 Elliptic problems

Consider the general second-order elliptic problem

−∇ · (a(x)∇u) + b(x) · ∇u + c(x)u = f in � ⊂ Rd(24a)

u = 0 on .(24b)

where the matrix a(x) is symmetric and positive definite for all x ∈ �, i.e.,

k2(x) |ξξξ |2 ≤ ξξξT a(x)ξξξ ≤ k1(x) |ξξξ |2

for some functions k1(x), k2(x). We also assume that the coefficients arebounded, i.e., (a)ij , (b)i, c ∈ L∞(�), i, j = 1, . . . , d, and that the follow-ing condition holds

c(x) − 1

2∇ · b(x) ≥ 0 ∀x ∈ �.

The weak formulation seeks a solution u ∈ H ≡ H 10 (�) such that

a(u, v) = f (v) for all v ∈ H 10 (�)(25)

M. Arioli et al.

wherea(w, v) = (a · ∇w, ∇v) + (b · ∇w, v) + (cw, v).

It is straightforward to show that a(·, ·) satisfies the continuity and coercivityconditions (2) with respect to the H 1

0 -norm | · |1 with constants

C1 = ‖k1‖L∞(�) + ‖b‖L∞(�) + C(�)‖c‖L∞(�), C2 = minx∈�

k2(x),

where C(�) is a constant of order one which depends only on the domain.Let now Hh ⊂ H be a space of piecewise polynomials defined on a par-

tition Th of � into simplices T of diameter hT . As described in Section (2.2)the inclusion Hh ⊂ H ensures that the stability conditions (13a) and (13b)are satisfied with the constants C1, C2 defined above. Moreover, discretizing(25) as

Au = f,

the constants C2, C3 in (13) are given as follows. If we choose to monitor theerror with respect to | · |1 then

C3 = ‖k1‖L∞(�) + C(�)‖c‖L∞(�), C2 = minx∈�

k2(x).

However, if we work with the energy norm defined by

|||w||| = a(w, w),(26)

then C2 = C3 = 1.

6.2 Numerical experiments

To illustrate the ideas presented above, we chose to perform experiments ona 2D advection-diffusion problem (c = 0). In particular, we chose to studythe robustness of our stopping criteria with respect to the nonsymmetry inthe problem. Thus, we solved a test problem for constant diffusivity tensors

a(x) = νI,

where the diffusion parameter ν toggles the degree of nonsymmetry of thematrices involved. The test problem is thus

−ν∇2u + b(x, y) · ∇u = f in � ≡ (−1, 1)2(27a)

u = 0 on ,(27b)

with

b(x, y) =(

2y(1 − x2)

−2x(1 − y2)

)

Stopping criteria for iterations in finite element methods

and right-hand side f such that the solution u is

u(x, y) =(

1 − e(x−1)/√

ν + e(−x−1)/√

ν

1 + e−2/√

ν

)·(

1 + y − 2e(y−1)/ν + e(−2)/ν

1 − e−2/ν

).

This choice of solution tries to mimick the behaviour of problems whereboundary layers are present (see Fig. 1).

We consider here only the errors with respect to the H 10 -norm, as the case

where the energy norm (26) is employed yields similar results. We denoteby uI the linear interpolant of the solution at the mesh points. Our numericalresults below will display the following estimators and errors:

(i) FE: the exact relative (forward) errors |u − ukh|1/|uk

h|1;

(ii) FIE: the exact relative (forward) interpolation errors |uI − ukh|1/|uk

h|1;

(iii) HINV: the exact H−1-norm criterion (17) ηC−12 ‖rk‖H−1/‖uk‖H ;

(iv) AINV: the exact A−1-norm criterion (18) ηC2−1/2‖rk‖A−1/‖uk‖H ;

(v) HINV-est: the estimated H−1-norm criterion (19) ηC−12 ‖rk‖λ

−1/2min (H s

k )/‖uk‖H ;

(vi) AINV-est: the estimated A−1-norm criterion (20) ηC−1/22 ‖rk‖σ

−1/2min (H s

k )/‖uk‖H ;(vii) the standard 2-norm stopping criterion ‖rk‖/‖r0‖.

In all cases we chose the constants c∗ = C∗ = 1 in (19), (20). The choiceof η is related to the derivation of our criteria. Indeed, an informed choicewould be a value of η which is o(h). In our tests we used a value η = 0.15which is roughly the square root of the second coarsest mesh parameter. Wefound this choice to be robust for all parameter ranges.

−1

0

1

−1

0

1

0

1

2

(a) ν = 1/10

−1

0

1

−1

0

1

0

1

2

(b) ν = 1/100

Fig. 1. Solution of advection-diffusion problem

M. Arioli et al.

6.3 GMRES without preconditioning

We begin with the case of a uniform partition of � into squares of size h

and bilinear basis functions. The GMRES convergence curves and derivedbounds are displayed in Fig. 2.

As expected, the iterative process measured in the energy norm (theFE/FIE curve) exhibits a plateau which indicates that an approximate solutionhas been found with corresponding finite element error given by the level ofthe plateau. This level represents the functional backward error, which canonly be reduced through a better choice of finite elements (such as that ob-tained through further refinement). From a practical point of view, we wouldideally like to stop the iteration at the onset of this plateau, since no furtherimprovement is obtained with respect to this norm. This requires informationabout the order of the finite element error. This is many times available eitherasymptotically through a priori studies or numerically through a posteriorierror estimation on coarser meshes. We return to this issue at the end of thissection.

0 10 20 30 40 50 60 70 8010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(a) ν = 1

0 20 40 60 80 100 120 14010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(b) ν = 1/10

0 50 100 150 200 25010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(c) ν = 1/50

0 50 100 150 200 25010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(d) ν = 1/100

Fig. 2. Comparison of stopping criteria for GMRES; h = 1/16

Stopping criteria for iterations in finite element methods

We see that in all experiments the suggested criteria provide upper boundsfor the quantities of interest – in this case, the H 1

0 -norm of the error. Moreover,the achievable finite element error (the onset of plateau) require considerablyfewer iterations than a stopping criterion such as the relative Euclidean normof the residual being brought below 10−8 (standard threshold).

Another remarkable fact is that the criterion (17) based on the dual normof the residual is an upper bound for the interpolation error. The reason forthis is not so surprising since in standard finite element calculations the inter-polation error is usually smaller than the error in the energy or related norms(sometimes by a factor of h). Thus, our stopping criterion gives an upperbound on both errors so that the iterates have either achieved the final error orhave achieved an error bounded from above by our dual norm estimate. Wealso note here that the constants c∗, C∗ are indeed of order one for all valuesof ν. This robustness also holds with respect to the mesh parameter.

The same convergence curves for the case of restarted GMRES are dis-played in Fig. 3. They exhibit indeed the most dramatic difference between

0 20 40 60 80 10010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(a) ν = 1

0 50 100 150 200 250 300 350 40010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(b) ν = 1/10

0 200 400 600 800 1000 1200 1400 160010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(c) ν = 1/50

0 200 400 600 800 1000 1200 1400 160010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(d) ν = 1/100

Fig. 3. Comparison of stopping criteria for GMRES(50); h = 1/16

M. Arioli et al.

convergence in the H−1- or A−1-norm and the standard 2-norm criterion.Again, the estimation of the relevant convergence curves based on the approx-imation (21) works extremely well. Moreover, the difference between the twostopping criteria (17), (18) is negligible. However, this may not always be thecase. Indeed, the equivalence between the two norms described by Lemma 1deteriorates if the H -condition number of the problem deteriorates. For ourtest problem this happens when ν is small and the discretization is nonuni-form. We consider this case below.

For small values of ν, the problem becomes more nonsymmetric, withthe matrices more nonnormal. At the same time, the finite element error onuniform meshes of squares deteriorates and even becomes of order one. Oneway to avoid this is to refine the mesh suitably. Given the boundary layers inthe solution, we chose an exponential refinement of the meshes. In this case,the parameter h is not defined in (19), (20) – we chose h := ‖f‖M/‖f‖, whereM = (φi, φj ) is the Grammian (mass) matrix with respect to the L2(�)-innerproduct (·, ·). The convergence curves are displayed in Fig. 4. Again, we seethat the two norms of interest are approximated well; however, the exact con-vergence curves in the H−1- and A−1-norms are not close in the initial phaseof the iterative process, but become almost identical close to the convergencestage.

We end this section with an illustration on the use of a priori error esti-mates to guide the stopping of the iterative process in the case of uniformmeshes. In particular, following ([5]) we used in (19) C(h) = h/

√ν as stop-

ping criteria for the energy error (FE). Similarly, for the interpolation error(FIE) we chose C(h) = h2/

√ν. We chose to describe the resulting criteria

in terms of the achieved ratio of the final error after k iterations

ρ(k) = ‖u − uh‖1

‖u − ukh‖1

0 100 200 300 400 500 600 700 80010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(a) ν = 10−4

0 100 200 300 400 500 600 700 80010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVHINV−estAINV−est2−norm

(b) ν = 10−6

Fig. 4. Comparison of stopping criteria for GMRES – exponentially-stretched mesh

Stopping criteria for iterations in finite element methods

and the savings in terms of iterations using as reference the number of iter-ations K required by the standard criterion that the relative Euclidean normof the residual be brought below a tolerance of 10−8

σ (k) = K − k

K.

The results are presented in Table 1 for a range of meshes and values of ν. Wechose to compare our stopping criterion with what we call the ‘α100%-con-verged case’, i.e., the onset of the plateau for the FE curve with a guaranteedfraction α of accuracy. In particular we identified this nearly-converged caseas the iteration k for which the exact error (FE curve) satisfies ρ(k) ≥ α. Inour experiments we chose α = 0.98.

We see that in all cases the potential savings are between 40–60%, withgreater reductions possible on coarser meshes. This is, of course, natural: asthe mesh is refined, the error is reduced and the plateau occurs at levels closerand closer to that of the Euclidean residual criterion. However, in practice,for many problems of interest, the finite element error seldom reaches thesehigh levels of accuracy.

Table 1 also reveals that using a simple a priori criterion to estimate thefinite element error can work rather well for discretizations on quasi-uniformmeshes. This, again, is the case in particular on coarser meshes. With theexception ν = 1/100, for all values of ν our criterion came within 7% of thepotential savings.

Similar behaviour is noticed in the case where the interpolation error istaken as a reference level – in this case we use the notation ρ

(k)I , σ (k)

I to denotethe corresponding savings indicators where

Table 1. Full GMRES iterations (k), energy error indices (ρ(k)) and savings (σ (k)) for (i)the 98%-converged case and (ii) using stopping criterion (19) with C(h) = h/

√ν

Exact Bound

ν h k ρ(k) σ (k) k ρ(k) σ (k) K

1/16 28 0.984 0.61 29 0.988 0.59 721 1/32 64 0.983 0.55 66 0.989 0.54 144

1/64 139 0.981 0.51 146 0.994 0.48 2861/16 64 0.981 0.52 69 0.995 0.49 135

1/10 1/32 135 0.981 0.49 149 0.997 0.44 2691/64 284 0.981 0.47 316 0.998 0.40 5341/16 138 0.981 0.44 144 0.990 0.42 247

1/50 1/32 283 0.980 0.43 316 0.997 0.36 4981/64 572 0.980 0.40 646 0.999 0.30 9611/16 191 0.981 0.40 192 0.981 0.39 318

1/100 1/32 371 0.980 0.43 420 0.995 0.35 6481/64 786 0.980 0.40 905 0.998 0.31 1313

M. Arioli et al.

ρ(k)I = ‖uI − uh‖1

‖uI − ukh‖1

with uI the linear interpolant of the exact solution u. Since for our applica-tion the interpolation error tends to be one factor of h smaller, the numberof iterations required to satisfy our criterion will be greater than in the casewhere the energy was the relevant quantity. The results are displayed in Table2. While smaller, the potential savings remain important (between 25–45%)and are approximated somewhat less accurately with our criterion based ona priori error estimates (within 10% of potential savings). Moreover, we notea rather negligible (up to 0.025%) loss in robustness for larger values of ν,correspoding to the cases where the termination occurs up to 4 iterations lessthan the in the ‘optimal case’.

6.4 Preconditioned GMRES

We turn now to the case where preconditioning is employed to speed up theiteration process. As specified in Section 4, we consider only the case ofright preconditioning, which has the advantage of preserving the residual, aproperty which enabled us to derive the stopping criterion (22). However, theuse of this stopping criterion requires the estimation of the smallest singularvalue of our preconditioner P . In some cases this estimation can be performedcheaply, but in general it may be quite difficult to provide this information.The approximation we use is described below.

Table 2. Full GMRES iterations (k), interpolation error indices (ρ(k)I ) and savings (σ (k)

I )for (i) the 98%-converged case and (ii) using stopping criterion (19) with C(h) = h2/

√ν

Exact Bound

ν h k ρ(k)I σ

(k)I k ρ

(k)I σ

(k)I K

1/16 41 0.988 0.43 40 0.975 0.44 721 1/32 91 0.984 0.37 91 0.984 0.37 144

1/64 195 0.981 0.32 198 0.990 0.31 2861/16 89 0.980 0.34 88 0.979 0.35 135

1/10 1/32 194 0.980 0.28 190 0.969 0.29 2691/64 410 0.980 0.23 406 0.975 0.24 5341/16 156 0.984 0.37 178 1.001 0.28 247

1/50 1/32 35 0.980 0.32 385 1.000 0.23 4981/64 704 0.982 0.26 788 0.999 0.18 9611/16 207 0.981 0.35 237 0.999 0.26 318

1/100 1/32 445 0.980 0.31 510 0.999 0.21 6481/64 972 0.980 0.26 1093 1.000 0.17 1313

Stopping criteria for iterations in finite element methods

All preconditioning techniques require the solution of a linear systeminvolving the preconditioner matrix P :

P z = v.

Since

σmin(P ) = σ−1max(P

−1) =(

maxv∈Rn\{0}

‖P −1v‖‖v‖

)−1

we choose to approximate σmin(P ) via

σmin(P ) ∼(

maxk

‖zk‖‖vk‖

)−1

= mink

‖vk‖‖zk‖

where zk = P −1vk. In the case of GMRES, the vector vk is the vector gen-erated by the Arnoldi process, so that ‖vk‖ = 1.

The performance of GMRES with ILU preconditioning is displayed inFig. 5. While the number of iterations is greatly reduced, the convergence

0 5 10 15 20 25

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVAINV−est2−norm

(a) ν = 1

0 5 10 15 20 25 3010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVAINV−est2−norm

(b) ν = 1/10

0 5 10 15 20 2510

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVAINV−est2−norm

(c) ν = 1/50

0 5 10 15 20 2510

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINVAINV−est2−norm

(d) ν = 1/100

Fig. 5. Comparison of stopping criteria for GMRES with ILU(0) preconditioning; h =1/16

M. Arioli et al.

behaviour is similar to the unpreconditioned case. Moreover, the approxima-tion of the residual A−1-norm described above appears to work extremelywell. However, in general we expect over- or under-estimation to occur, inwhich case alternative methods for the estimation of the smallest singularvalue of P may have to be employed.

6.5 Three-term GMRES

We end this section with numerical results obtained with the minimum resid-ual algorithm based on a three-term recurrence described in Section 4. Werecall here that this is essentially the GMRES algorithm implemented in theH -norm with left-preconditioner H . The norm of the modified residual inthis method is the quantity we seek, ‖rk‖H−1 . The results are displayed inFig. 6. As before, our stopping criterion (17) provides an upper bound forthe convergence of quantities of interest, such as H 1

0 -norm of the error or theinterpolation error. More remarkable, though, is the fact that in this case thesolver yields iterates whose 2-norm residual traces closely the convergencecurves of interest. This is a phenomenon also noticed in the case of a similarGMRES implementation used for the solution of flow problems [15].

The same experiments were run with inexact implementation of the pre-conditioner H . More precisely, we solved systems with H using CG with anincomplete Cholesky preconditioner and a stopping criterion as described inArioli [1]; the tolerance was chosen to be of order h5/2, which for this problemis h1/2 less than the order of the interpolation error. The results are displayedin Fig. 7. We see indeed that our criterion is an upper bound for both the finiteelement error |u − uk

h|1 and the interpolation error |uI − ukh|1. Moreover, the

0 20 40 60 80 10010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(a) ν = 1/50

0 50 100 150 20010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(b) ν = 1/100

Fig. 6. Comparison of stopping criteria for H -norm minimum residual algorithm: exactpreconditioning

Stopping criteria for iterations in finite element methods

0 20 40 60 80 100 120 14010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(a) ν = 1/50

0 50 100 150 200 25010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(b) ν = 1/100

Fig. 7. Comparison of stopping criteria for H -norm minimum residual algorithm: inexactpreconditioning

inexact solves do not affect the convergence curve in the regime where it isrelevant.

6.6 Other iterative methods

In order to test further the relevance of the A−1- and H−1-norms of the resid-ual, we ran experiments with BICGSTAB, QMR and CGS. The results forthe case of discretization on uniform meshes are displayed in Figs 8, 9, 10.We again see that in all cases the two residuals provide upper bounds for theenergy norm of the error and interpolation error. In particular, the A−1-normprovides again the closest approximation to the these quantities, with theH−1-norm bound possibly deteriorating for more nonsymmetric problems,

0 50 100 150 200 250 300

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(a) ν = 1/50

0 100 200 300 400 500

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(b) ν = 1/100

Fig. 8. Comparison of stopping criteria for BICGSTAB: h=1/16

M. Arioli et al.

0 50 100 150 200 250 300

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(a) ν = 1/50

0 50 100 150 200 250 30010

−12

10−10

10−8

10−6

10−4

10−2

100

FEFIEHINVAINV2−norm

(b) ν = 1/100

Fig. 9. Comparison of stopping criteria for QMR: h=1/16

0 50 100 150 200 250 300 350

10−10

10−8

10−6

10−4

10−2

100

102

104

106

108

1010

FEFIEHINVAINV2−norm

(a) ν = 1/50

0 50 100 150 200 250 300 350 40010

−12

10−10

10−8

10−6

10−4

10−2

100

102

104

106

108

1010

FEFIEHINVAINV2−norm

(b) ν = 1/100

Fig. 10. Comparison of stopping criteria for CGS: h=1/16

as Fig. 9 shows. As for the 2-norm of the residual, the behaviour oscillatesbetween the smooth, relevant convergence curve of QMR to the oscillating,large residuals exhibited by CGS. However, the issue of dynamic estima-tion of the A−1- and H−1-norms is not as straightforward as in the case ofGMRES.

7 Conclusion

The message of this paper is simple: do not accurately compute the solutionof an inaccurate problem. This was highlighted already in [1] for the caseof symmetric and positive-definite problems – our contribution here was thegeneralization to the case of nonsymmetric problems. The proposed stoppingcriteria require the calculation of the residual in a norm related to the problemformulation. It also requires information about the discretization error which

Stopping criteria for iterations in finite element methods

may not always be available. In practice, a priori error estimates on quasi-uniform meshes or a posteriori error calculations may provide sufficient forthis purpose. The former estimator was successfully employed in our testcase. The latter appears also to be promising [9].

Overall, we demonstrated that the suggested criteria are relevant to con-vergence in the energy-norm (or equivalent norms) while at the same timehighlighting the fact that the standard criterion based on the Euclidean normof the residual has no relevance to the quantities of interest and is in generalwasteful. Further generalizations of these ideas include the case of indefi-nite problems and mixed finite element discretizations of systems of partialdifferential equations, where the use of mixed norms in which to measureconvergence arises quite naturally. We hope to address some of these issuesin a future paper.

Acknowledgements. We thank Serge Gratton for useful discussions and comments.

8 Appendix

Proof of Lemma 1 We need to show

1√C3

‖r‖A ≤ ‖r‖H ≤ 1√C2

‖r‖A

and √C2

C1C3‖r‖H−1 ≤ ‖r‖A−1 ≤ 1√

C2‖r‖H−1 .

The first equivalence is just a restating of the discrete stability conditions(13b), (13c). For the second we have

rT A−1rrT H−1r

≤ σ1(H1/2A−1H 1/2)

= σ−1n (H−1/2AH−1/2)

≤(

minx∈Rn\{0}

xT H−1/2AH−1/2xxT x

)−1

=(

miny∈Rn\{0}

yT AyyT Hy

)−1

≤ C−12

by using (13b). Finally, since

C2 ≤ rT ArrT Hr

= rT HArrT Hr

≤ C3,

M. Arioli et al.

we have

rT A−1rrT H−1r

= rT A−1r

rT H−1A r

· rT H−1A r

rT H−1r≥ C−1

3 minr∈Rn\{0}

rT A−1r

rT H−1A r

≥ C−13 min

y∈Rn\{0}yT A−1y

yT y,

where A = I + N, N = H−1/2A SAH

−1/2A . Since A (and thus A−1) is a normal

matrix, its field of values is the convex hull of its eigenvalues ([13, p.11]).Hence,

miny∈Rn\{0}

yT A−1yyT y

= mink

Re1

λk(A)= min

kRe

1

1 + λk(N)= 1

maxk

∣∣∣λk(A)

∣∣∣2

and since ‖H−1/2AH−1/2‖ = ‖A‖H,H−1 = C1 (cf. (11), (15)) we get

maxk

∣∣∣λk(A)

∣∣∣ ≤ ‖H−1/2A AH

−1/2A ‖ ≤ ‖H−1/2AH−1/2‖κ2(H

−1/2A H 1/2)

and the result follows. �

Proof of Lemma 3 Consider the two equivalent linear systems

Au = f, Au = f

where

A = H−1/2AH−1/2, u = H 1/2u, f = H−1/2f, A = H−1A, f = H−1f .

The first part of the Lemma follows from the equivalence of the Arnoldialgorithms below.

Arnoldi in (·, ·)v1 := r0/‖r0‖do j = 1, 2, . . . , m

hij = (Avj , vi ), 1 ≤ i ≤ j

wj = Avj −∑j

i hij vi

if hj+1,j = ‖wj‖ = 0 stopvj+1 = wj /hj+1,j

end do

Arnoldi in (·, ·)Hv1 := r0/‖r0‖H

do j = 1, 2, . . . , m

hij = (Avj , vi )H , 1 ≤ i ≤ j

wj = Avj −∑j

i hij vi

if hj+1,j = ‖wj‖H = 0 stopvj+1 = wj /hj+1,j

end do

where vi = H 1/2vi , wi = H 1/2wi , r0 = H−1/2r0, r0 = H−1r0, r0 = f −Au0 for some initial guess u0. In particular, they yield the same Hessenbergmatrices since

hij = (Avj , vi ) = (H−1/2Avj , H1/2vi ) = (H−1Avj , vi )H = hij .

Stopping criteria for iterations in finite element methods

For the second part, we work with the Arnoldi algorithm in the Euclideaninner-product and system matrix A = I + N , where N = −NT is skew-symmetric. For ease of presentation we drop the ∼’s. We now prove byinduction on j that for all i ≤ j − 2

hij = vTi (I + N)vj = 0.

We first note that (i) hij = 1 if i = j , (ii) hij = vTi Nvj if i < j and

(iii) vTi N3vi = 0, since N is skew-symmetric. Since w1 = Nv1 and w2 =

Nv2 − h12v1 we have using (i)-(iii)

h13 =vT1 Nv3 = vT

1 Nw2

h32= vT

1 N(Nv2 − h12v1)

h32= vT

1 N2v2

h32= vT

1 N3v1

h32h21=0

and the first inductive step holds. Assume now that for all i ≤ j − 2, hij =vT

i Nvj = 0. Then (iv) wj = Nvj − hj−1,j vj−1. Hence, for all i ≤ j − 2,

0 = hi,i−1vTi Nvj = vT

j NT wi−1 = vTj NT Nvi−1 = −vT

j N2vi−1

i.e., we have (v) vTj N2vi = 0 for all i ≤ j −2. We now prove that hi,j+1 = 0

for all i ≤ j − 1. We have using (iv)

hi,j+1 = vTi Nwj

hj+1,j

= vTi N(Nvj − hj−1,j vj−1)

hj+1,j

= vTi N2vj − hj−1,j vT

i Nvj−1

hj+1,j

.

If i ≤ j −3, by the inductive hypothesis vTi Nvj−1 = 0 and by (v) vT

i N2vj =0 and hence hi,j+1 = 0. If i = j −2 then hj−2,j+1 = 0 also since vT

j−2N2vj −

hj−1,j vTj−2Nvj−1 = vT

j−2N2vj + hj−1,jhj−1,j−2 = 0 because

hj−1,j = vTj−1Nvj = vT

j NT wj−2

hj−1,j−2= −vT

j N2vj−2

hj−1,j−2.

Finally, if i = j−1,hj−1,j+1 = vj−1N2vj /hj+1,j = 0 since vj−1N

2vj =0 for all j ≥ 2. This we prove again by induction. Assuming vj−1N

2vj = 0,we have using (iv), (iii)

vjN2vj+1 = vjN

2wj

hj+1,j

= vjN2(Nvj − hj−1,j vj−1)

hj+1,j

= 0.

The result follows by noting that

v1N2v2 = v1N

2w1/h21 = v1N3v1/h21 = 0.

�

M. Arioli et al.

References

1. Arioli, M.:A stopping criterion for the conjugate gradient algorithm in a finite elementmethod framework. Numer. Math. 97(1), 1–24 (2004)

2. Arioli, M., Noulard, E., Russo,A.: Stopping criteria for iterative methods: applicationsto PDEs. Calcolo 38, 97–112 (2001)

3. Babuska, I.: Error bounds for finite element methods. Numer. Math. 16, 322–333(1971)

4. Brezzi, F., Bathe, K.J.: A discourse on the stability conditions for mixed finite elementformulations. Comp. Meth. Appl. Mech. Engrg. 82, 27–57 (1990)

5. Ciarlet, P.G.: The finite element method for elliptic problems. North Holland, Amster-dam, 1978

6. Concus, P., Golub, G.H.: A generalized conjugate gradient method for nonsymmetricsystems of linear equations. In: R. Glowinski, J.L. Lions, (eds.), Proc. Second Inter-nat. Symp. on Computing Methods in Applied Sciences and Engineering, volume 134of Lecture Notes in Economics and Mathematical Systems, Berlin, 1976. SpringerVerlag

7. Deuflhard, P.: Cascadic conjugate gradient methods for elliptic partial differentialequations. Algorithm and numerical results. In: J. Xu, D. Keyes, (eds.), Proceed-ings of the 7th International Conference on Domain Decomposition Methods, AMSProvidence, 1993, pp. 29–42

8. Deuflhard, P.: Cascadic conjugate gradient methods for elliptic partial differentialequations: algorithm and numerical results. Contemporary Mathematics 180, 29–42(1994)

9. Deuflhard, P., Bornemann, F.A.: The cascadic multigrid method for elliptic problems.Numerische Mathematik 75, 135–152 (1996)

10. Golub, G.H., Meurant, G.: Matrices, moments and quadrature II; how to compute thenorm of the error in iterative methods. BIT 37(3), 687–705 (1997)

11. Golub, G.H., Strakos, Z.: Estimates in quadratic formulas. Numer. Algorithms 8, 2–4(1994)

12. Higham, N.J.: Accuracy and stability of numerical algorithms. SIAM, 200213. Horn, R.A., Johnson, C.R.: Matrix analysis. Cambridge University Press, 198514. Horn, R.A., Johnson, C.R.: Topics in matrix analysis. Cambridge University Press,

199115. Loghin, D.,Wathen,A.J.:Analysis of block preconditioners for saddle-point problems.

Technical Report 13, Oxford University Computing Laboratory, 2002, Submitted toSISC

16. Meurant, G.: Numerical experiments in computing bounds for the norm of the errorin the preconditioned conjugate gradient algorithm. Numer. Algoithms 22, 353–365(1999)

17. Rigal, J.L., Gaches, J.: On the compatibility of a given solution with the data of alinear system. J. Assoc. Comput. Mach. 14(3), 543–548 (1967)

18. Starke, G.: Field-of-values analysis of preconditioned iterative methods for non-sym-metric elliptic problems. Numer. Math. 78, 103–117 (1997)

19. Strakos, Z., Tichy, P.: On error estimation by conjugate gradient method and why itworks in finite precision computations. Electronic Transactions on Numerical Analy-sis 13, 56–80 (2002)

20. Strang, W.G., Fix, G.J.: An analysis of the finite element method. Prentice-Hall, 197321. Widlund, O.: A Lanczos method for a class of nonsymmetric systems of linear equa-

tions. SIAM J. Numer. Anal. 15(4), 1978