The algorithm that determines the start iteration of the Halley-Altman method

Octavian Cira and Cristian Mihai Cira”Aurel Vlaicu” University of Arad, Romania

Exact Science FacultyDepartment Mathematics and Computer Science

E-mail: {octavian, cristi} cira@yahoo.com

Abstract

The paper is devoted to the Ostrowski-Kantorovich typeconvergence theorem for the Halley-Altman method, withthe S-order of convergence equal to 3, for nonlinear opera-tor equations in Banach spaces. The main result of the arti-cle is the algorithm that determines the start iteration fromthe cubic convergence sphere of the Halley-Altman method.The Mathcad implementation treats the finite dimensionalcase.

1 Introduction

Let F : D ⊂ X → Y denote a nonlinear operator twiceFrechet differentiated, where D is a domain, X and Y arereal or complex Banach spaces. We consider the equationF (x) = 0. Let Γ(x) = F ′(x)−1 be the operator and denoteby NF the Newton operator

NF (x) = Γ(x)F (x) ,

where, in the finite dimensional case, X = Y = Rn orX = Y = Cn, F ′(x) is the Jacobian of F

F ′(x) =

∂f1(x)

∂x1

∂f1(x)∂x2

· · · ∂f1(x)∂xn

∂f2(x)∂x1

∂f2(x)∂x2

· · · ∂f2(x)∂xn

......

. . ....

∂fn(x)∂x1

∂fn(x)∂x2

· · · ∂fn(x)∂xn

.

Let to be the operator defined by Altman in [1]

AF (x) = Γ(x)F ′′(x)NF (x) .

In the finite dimensional case , the operator F ′′(x) is atensor, F ′′(x) ∈ Mn,n,n(K), where K = R or K = C.We denote F ′′(x) = F ′′

1 (x)&F ′′2 (x)& . . .&F ′′

n (x), where

∂2f1(x)∂x1∂x1

∂2f1(x)∂x1∂x2

∂2f1(x)∂x1∂x3

∂2f2(x)∂x1∂x1

∂2f2(x)∂x1∂x2

∂2f2(x)∂x1∂x3

∂2f3(x)∂x1∂x1

∂2f3(x)∂x1∂x2

∂2f3(x)∂x1∂x3

∂2f1(x)∂x2∂x1

∂2f1(x)∂x2∂x2

∂2f1(x)∂x2∂x3

∂2f2(x)∂x2∂x1

∂2f2(x)∂x2∂x2

∂2f2(x)∂x2∂x3

∂2f3(x)∂x2∂x1

∂2f3(x)∂x2∂x2

∂2f3(x)∂x2∂x3

∂2f1(x)∂x3∂x1

∂2f1(x)∂x3∂x2

∂2f1(x)∂x3∂x3

∂2f2(x)∂x3∂x1

∂2f2(x)∂x3∂x2

∂2f2(x)∂x3∂x3

∂2f3(x)∂x3∂x1

∂2f3(x)∂x3∂x2

∂2f3(x)∂x3∂x3

Figure 1. F ′′(x) of dimension 3× 3× 3

F ′′k ∈Mn,n(K). Then the Halley-Altman method is

x = x−

[I +

12AF (x)

(I − 1

2AF (x)

)−1]

NF (x) ,

(1.1)where x is the next iteration.

Let the Halley operator be

HF (x) =

[I +

12AF (x)

(I − 1

2AF (x)

)−1]

,

then the short form of the Halley-Altman method is

x = x−HF (x)NF (x) . (1.2)

The proof of Ostrowski-Kantorovich convergence theo-rem imposes that form (1.1) is replaced by the followingequivalent form

y = x−NF (x)

AF (x, y) = −Γ(x)F ′′(x)(y − x)

x = y − 12Γ(x)Q(x, y)F ′′(x)NF (x)2

, (1.3)

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

Ninth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

0-7695-3078-8/08 $25.00 © 2008 IEEEDOI 10.1109/SYNASC.2007.25

381

where Q(x, y) =[I − 1

2AF (x, y)]−1

.

2 Lemmas

The following lemma is a classical result in the linearoperators theory [7].

Lemma 2.1. If an operator A ∈ L(X, Y ) is invertible andthe norm of operator B ∈ L(X, Y ) satisfies the inequality‖B‖ ≤

∥∥A−1∥∥−1

, then there exists (A + B)−1 and theinequality∥∥(A + B)−1

∥∥ ≤ (∥∥A−1∥∥−1 − ‖B‖

)−1

stands.

Let D0 be a convex and open subdomain of D ⊂ X . Wepresent the representation lemma for F (x) [3], [6] [9].

Lemma 2.2. Let F : D0 ⊂ X → Y denote a nonlin-ear operator, twice Frechet differentiable, with F ′ and F ′′

continuous over D0. If there is Γ(x) and Q(x, y), then theequality

F (x) =∫ 1

0

F ′′(y + t(x− y))(1− t)dt(x− y)2

− 12

∫ 1

0

F ′′(x + t(y − x))dt(y − x)Γ(x)Q(x, y)F ′′(x)

× (y − x)2 +12Q(x, y)Γ(x)F ′′(x)(y − x)

×∫ 1

0

F ′′(x + t(y − x))(1− t)dt(y − x)2 + Q(x, y)

×∫ 1

0

[F ′′(x + t(y − x)

)(1− t)− 1

2F ′′(x)

]dt(y−x)2 .

3 Convergence conditions

Let x(0) ∈ D0 denote the start iteration. We presupposethat the nonlinear operator F is twice Frechet differentiablewith F ′ and F ′′ continuous and that we have the followingcondition fulfilled:

(a) ∃ Γ(x(0)

)and

∥∥Γ (x(0))∥∥ ≤ β and

∥∥F (x(0))∥∥ ≤ γ;

(b) ∃ M,L > 0 constants, sush that ‖F ′′(x)‖ ≤ M and‖F ′′(x)− F ′′(y)‖ ≤ L ‖x− y‖ for ∀ x, y ∈ D0;

(c) The inequality λ = κβ2γ < 1/2 stands , where

M

√3 +

2L

3M2β≤ κ .

The proof of the following lemma is trivial.

Lemma 3.1. Let κ, β and γ be three real positive numbers.Then the function

g(t) =κ

2t2 − 1

βt + γ , (3.1)

has two real and positive roots if the condition λ = κβ2γ <1/2 is fulfilled.

Figure 2. Function g(t) = κ2 t2 − 1

β t + γ

We define the following real sequences:

sk = tk − g′(tk)−1g(tk)

ak = −g′(tk)−1g′′(tk)(sk − tk)

tk+1 = sk −12g′(tk)−1qkg′′(tk)(sk − tk)2

(3.2)

where qk =(1− 1

2ak

)−1, for k = 0, 1, . . ., and t0 = 0. If

sk and ak are substituted in the formula of tk+1 it impliesthat

tk+1 := tk −1

1− 12· g

′′(tk)g(tk)g′(tk)2

· g(tk)g′(tk)

.

This proves that the formula (3.2) are just a different formof Halley method for the equation g(t) = 0, where t0 = 0.According to the theorem of local convergence for the Hal-ley method in R1, [5, T. 6.10.1, pag. 134], the sequences{tk} and {sk} are convergent, with an R-order of conver-gence equal to 3, and:

limk→∞

tk = r1 and limk→∞

sk = r1 ,

where r1 is the smallest real positive root of function g.The main lemma used in the proof of the convergence

theorem for the Halley-Altman method is presented next [3],[6], [9].

382382382382382382

Lemma 3.2. If for the positive and real constants M , L, βand γ, specified in the convergence conditions (a)−(c) andfor the sequences defined by (3.2), the relations:

1.∥∥y(k) − x(k)

∥∥ ≤ sk − tk ,

2.∥∥Γ (x(k)

)∥∥ ≤ −g′(tk)−1 ,

3.∥∥F ′′ (x(k)

)∥∥ ≤ g′′(tk) ,

are verified, therefore following inequalities are true:

1.∥∥∥[I + 1

2Γ(x(k)

)F ′′ (x(k)

) (y(k) − x(k)

)]−1∥∥∥

≤ 11− 1

2ak

,

2.∥∥x(k+1) − y(k)

∥∥ ≤ tk+1 − sk ,

3.∥∥F (x(k+1)

)∥∥ ≤ g(tk+1) ,

4.∥∥y(k+1) − x(k+1)

∥∥ ≤ sk+1 − tk+1 ,

5.∥∥∥∫ 1

0

[2F ′′ (x(k) + τ

(y(k) − x(k)

))(1− τ)

−F ′′ (x(k))]

dτ∥∥ ≤ L

3 (sk − tk) .

4 The convergence theorem

Let r1 be the smallest positive and real root of functiong, given by (3.1). Then:

r1 =1−√

1− 2λ

κβand θ =

r1

r2=

1−√

1− 2λ

1 +√

1− 2λ. (4.1)

Note that if κβ2γ = λ ∈ (0, 1/2) then θ ∈ (0, 1). Wepresuppose that

S(x(0), r1

)={

x∣∣∣ ∥∥∥x− x(0)

∥∥∥ ≤ r1

}⊂ D0 .

Theorem 4.1. If the convergence conditions (a)-(c) are ful-filled and we denote (4.1), then:

1. the Halley-Altman method, given by (1.3), is conver-gent;

2. {x(k)}, {y(k)} ⊂ S(x(0), r1

)for k = 0, 1, . . .;

3. {x(k)}, {y(k)} limit, denoted by x? is the solution ofequation F (x) = 0;

4. We have the following estimations for errors:∥∥∥x(k) − x?∥∥∥ ≤ r1 − tk ,

∥∥∥y(k) − x?∥∥∥ ≤ r1 − sk

r1 − tk = βγ1− θ2

1− θ3k θ3k−1 ,

for any k = 0, 1, . . . .

The proof of the Halley-Altman method convergenceuses the asymptotical behavior of the sequences {tk} and{sk}, given by (3.2), in respect to the sequence generatedby the Halley-Altman method. In order to define these twosequences is necessary to use the second degree function gdefined in (3.1).

5 The cubical convergence

In order to prove the 3rd order convergence of Halley-Altman method we will introduce the notion of S − orderof convergence defined by D. Chen in his article [2].

Definition 5.1. We state that a sequence{x(k)

}that is con-

vergent to x? in a metric space has a convergence of p > 0order, if p is the greatest power that fulfills the equality

Eg (g(tk+1), tk, sk)= g(tk+1)− C(tk, sk)(sk − tk)p = 0 ;

for C(·, ·) > 0. The function Eg is implied by the function

EF

(x(k), y(k), x(k+1)

)= F

(x(k+1)

)−R

(x(k), y(k), x(k+1)

),

where function R(x(k), y(k), x(k+1)

)is the representation

of F (x) = F(x(k+1)

), given by lemma 2.2.

Definition 5.2. In the context of the definition for the S-order of convergence the asymptotic error is

C(t∗, t∗) = limk→∞

g(tk+1)(sk − tk)p

;

where t∗ is the limit of the sequence {tk} and {sk}.

The following theorem proves the cubic S − order ofconvergence of the Halley-Altman method [3], [6], [9].

Theorem 5.3. For the Halley-Altman method (1.3) and theassociated function g, given by formula (3.1), we have theequality

g(tk+1) = C(tk, sk)(sk − tk)3 ,

where

C(tk, sk)

=κ3(sk − tk)

8g′(tk)2[1 + κ(sk−tk)

2g′(tk)

]2 − κ2

4g′(tk)[1 + κ(sk−tk)

2g′(tk)

] .

Corollary 5.4. From the method given by the formula (1.3)the S−order of convergence is 3, and the asymptotic erroris

C(r1, r1) =κ2β

4√

1− 2λ.

383383383383383383

6 Programs for the Halley-Altman method

In order to exemplify here is a nonlinear equation fromR2.

Example 6.1. We choose a value for the Mathcad variableORIGIN , namely 1.

ORIGIN := 1

We define the nonlinear function F : D ⊂ R2 → R2

F (x) :=(

x21 − 2 cos(3x1)− x2 − c1

x1 − x32 + 3 cos

(52x2

)+ c2

), (6.1)

where

c1 = 8.822260523769353 and c2 = 4.149013443610321 ,

for the nonlinear equation F (x) = 0. The Jacobian of func-tion F is

F ′(x) :=(

2x1 + 6 sin(3x1) −11 −3x2

2 − 152 sin

(52x2

) ) .

We define the identity matrix and the operators Γ and N

I := identity(2) , Γ(x) := F ′(x)−1

andN(x) := Γ(x) · F (x) ,

The function identity(n) was used to generate the identitymatrix of order n. The 2nd degree differential of the non-linear function F is the tensor, of dimension 2 × 2 × 2,F ′′(x) = F ′′

1 (x)&F ′′2 (x), where

F ′′1 (x) :=

(2 + 18 cos(3x1) 0

0 0

)and

F ′′2 (x) :=

(0 00 −6x2 − 75

4 cos(

52x2

) ) .

Let us define the operators A and H

A(x) := Γ(x) · augment(F ′′

1 (x) ·N(x), F ′′2 (x) ·N(x)

),

H(x) := I +12A(x)

(I − 1

2·A(x)

)−1

where the function augment concatenates horizontally thevectors F ′′

1 (x) ·N(x) and F ′′2 (x) ·N(x). Once these prepa-

rations are made we can now start to implement the programfor the Halley-Altman method.

Program 6.2. for the Halley-Altman method, given by (1.2),is

HA(x, ε) := z ← xT

δ ← H(x) ·N(x)while |δ| ≥ ε

x← x− δz ← stack(z, xT)δ ← H(x) ·N(x)

return z

For the initial vector x := (3.5, 3)T and ε := 10−13 theHA program returns the 6 iterations of the Halley-Altmanmethod, where the cubic convergence of the method can beobserved only from the second step. The program HA callsthe function stack that concatenates vertically two matriceswith the same number of columns

HA(x, 10−13

)=

3.5 33.1192099104756954 2.30865427409644443.0048412921980550 2.03319400807095603.0000062540379533 2.00005964688138433.0000000000000355 2.0000000000003375

3 2

.

For the initial vector x := (0.5, 0.5)T we notice a conver-gence towards a different solution than the solution (3, 2)T,

HA(x, 10−13

)=

0.5 0.51.5445787948945122 1.0528293563439608−0.4734173150998957 1.0379449600217274−1.4353718472387609 0.9186244405200327−2.9677655995119716 0.6921861936041878−2.8670018509587290 0.7455332156664843−2.8659580431150893 0.7457695338624732−2.8659580421002095 0.7457695339861085

.

The examples presented above show that a randomchoice of the start iteration does not ensure a cubic conver-gence of the Halley-Altman method from the first step, oras noticed in the second example, the convergence can leadto a different solution that the predicted one. It is obviousthat there are start iterations for which the method does notconverge.

7 Determination of the start iteration

An algorithm that determines the start iteration for theHalley-Altman method and ensures a cubic convergenceeven from the first step of the method is presented. This startiteration and the radius r1 define the sphere S

(x(0), r1

),

that contains all the terms of the sequence{x(k)

}, therefore

it also contains x?, the solution of the nonlinear equationF (x) = 0.

384384384384384384

As show in the section dedicated to the convergence ofthe Halley-Altman method, M and L are global constantsthat depend on the function F ′′(x) and the convex domainD0. Let us consider an ellipse with the center in (xc, yc)T,the semi axes a and b and the angle of rotation α (angle be-tween the OX and the ellipse ax of length 2a). The implicitequation of the ellipse is

E(x) =[(x1 − xc) cos(α) + (x2 − yc) sin(α)

a

]2+[−(x1 − xc) sin(α) + (x2 − yc) cos(α)

b

]2− 1 .

(7.1)

LetD0 =

{x∣∣E(x) ≤ 0, x ∈ R2

}(7.2)

denote a convex domain. The parametric equations for theellipse, (7.1), are:

xE(τ) = xc + a cos(τ) cos(α)− b sin(τ) sin(α),

yE(τ) = yc + a cos(τ) sin(α) + b sin(τ) cos(α),

where τ ∈ [0, 2π]. We consider m points, on the ellipse(7.1), whose coordinates are:

Pk =(xE(τk), yE(τk)

)T,

where τk = 2(k − 1)π/m, k = 1, 2, . . . ,m.We have the equation F (x) = θ, where the nonlinear

function F is given by (6.1) and θ = 0. In figure 3, the non-linear system is represented by the two curves. The convexpolygon P ⊂ D0 was considered with m sides (m = 13),with the m angles on the ellipse (7.1). The topological de-gree d depends on function F , on the considered domainand on the vector θ, the right term of the equation. For thegiven example in figure 3, the topological degree was calcu-lated relative to a convex polygon, with the algorithm pub-lished in [4]. The outcome is d(F, P, θ) = 1. The existencetheorem, that is constructed around the notion of Brouwertopological degree, states that any function with a topolog-ical degree not equal to 0 has at least one solution of theequation F (x) = θ in the considered domain P . If equa-tion F (x) = 0 is a solution in polygon P then results thatthis solution is in convex domain D0. This act is appointed,in the special literature, the separation of the solutions equa-tion.

Remark 7.1. In order to calculate the topological degree itis recommend to chose a relatively great value for m. If thevalue is greater then the polygon P approximates better theconvex domain D0.

Figure 3. The polygon P and function F

The following nonlinear programming problem deter-mines the M bounding of operator F ′′(x) over D0. Wedefine the norm for tensor F ′′(x)

nF ′′(x) := max(norme

(F ′′

1 (x)), norme

(F ′′

2 (x)))

,

where function norme, was called to compute the euclidiannorm of the square matrix. The function nF ′′ is a norm andit was considered by Rall in [8]. Let x ∈ D0 denote initialvector, namely the center of ellipse (7.1), x :=

(xc, yc

)T,

for the nonlinear programming problem. In the equationspace from Mathcad, defined by the space between the key-words given and minimize or maximize we define the non-linear programming problem.

Given E(x) ≤ 0 ξ := Maximize(nF ′′, x) .

The objective function value nF ′′ in ξ is the constant M

M := nF ′′(ξ) M = 33.98345549083061 .

The Lipschitz constant for the convex set D0 is obtainedby solving the following nonlinear programming problem.Let

L(u, v) := max

(norme

(F ′′

1 (u)− F ′′1 (v)

)|u− v|

,

norme(F ′′

2 (u)− F ′′2 (v)

)|u− v|

),

be the objective function, where the vector module is usewhich represents the euclidian norm of that vector. Let theinitial vectors u, v ∈ D0 be two vertexes relatively diamet-rically opposed of polygon P

u :=(

P1,1

P2,1

)v :=

(P1,bm

2 cP2,bm

2 c

).

385385385385385385

We consider the nonlinear programming problem

Given E(u) ≤ 0 E(v) ≤ 0(µω

):= Maximize(L, u, v) .

The objective function L value in the optimum solution(µ, ω)T is the Lipschitz constant of F ′′(x) over D0.

L := L(µ, ω) L = 53.99929363561056 .

Because the constants κ, β and γ depend on x, we willsolve a nonlinear programming problem that fulfills theconditions for constants κ, β and γ. We define the followingfunctions:

β(x) := norme (Γ(x)) ,γ(x) := |F (x)| ,κ(x) := M ·

√3 + 2·L

3·M2·β(x) ,

λ(x) := κ(x) · β(x)2 · γ(x)

and the objective function

θ(x) :=1−

√1− 2 · λ(x)

1 +√

1− 2 · λ(x).

The initial vector from the set D0 is chosen, namely thecenter of the ellipse that defines the convex domain D0.

Given E(x) ≤ 0 λ(x) <12

s := Maximize(θ, x) .

The optimization outcome is:

s =(

3.0138773301520772.008064840979473

).

This solution of the nonlinear programming problem isthe start iteration x(0), for the Halley-Altman method. Ac-cording to theorems 4.1 and 5.3 for this iteration the methodconverges with a S-order of convergence equal to 3, evenfrom the first iteration, and all iterations are within thesphere S

(x(0), r1

). The value of the objective function is:

θ(s) = 0.3761672492198908 ,

and the asymptotical error measures:

Ea =κ(s)2 · β(s)

4√

1− 2 · λ(s)= 485.95460226191153 .

The values that define the function g given by (3.1) and theradius r1 of the sphere that contains all the iterations for thegiven sequence of the Halley-Altman method

κ := κ(s) = 60.10159645743778000 ,β := β(s) = 0.24393865650486676 ,γ := γ(s) = 0.11107626743135408 ,

r1 = 0.03728834628541981 .

For the start iteration s, determined using the nonlinear pro-gramming problem the Halley-Altman method generates,using program 6.2, the following iterations:

HA(s, 10−13

)= 3.0138773301520770 2.0080648409794730

3.0000031629106085 2.00000095941745533 2.0000000000000004

.

8 Conclusions

The algorithm presented in the example above inR2 canbe generalized for any Rn, where n ≥ 3. This algorithmis a solution for the start iteration problem for the Halley-Altman method. Instead of a random choice for the start iter-ation, a convex domain D0 is chosen, for which the topolog-ical degree of the nonlinear function F is not zero. The costof finding a convex domain (7.2) with a topological degreenot equal to zero is much smaller that the cost of finding astart iteration for the Halley-Altman method that ensures anorder of convergence equal to 3 from the first step.

References

[1] M. ALTMAN. Concerning the method of tangent hyperbolasfor operator equations. Bull. Acad. Pol. Sci., Serie Sci. Math.,Ast. et Phys., 9:633–637, 1961.

[2] D. CHEN. On a new definition of order of convergence ingeneral iterative methods I: One point iterations. Technicalreport, Departament of Mathematical Sciences, University ofArkansas, 1991.

[3] D. CHEN. Ostrowski-Kantorovich theorem and S-order ofconvergence of Halley method in Banach spaces. Comment.Math. Univ. Carolinae, 34(1):153–163, 1993.

[4] O. CIRA. Algorithm for determination of initial iteration ofnumerical methods for the solution of nonlinear equation. An.Universitatii de Vest din Timisoara. Ser. Sti. Mat., XXVI, fasc.3:17–23, 1988.

[5] O. CIRA. Metode numerice pentru rezolvarea ecuatiilor al-gebrice. Ed. Academiei Romane, Bucuresti, 2005.

[6] J. M. GUTIEREZ and M. A. HERNANDEZ. A family ofChebyshev-Halley type methods in Banach spaces. Bull. Aust.Math. Soc., 55:113–130, 1997.

[7] L. V. KANTOROVICH and G. P. AKILOV. Functional Analy-sis. Pergamon Press, second edition, June 1982.

[8] L. B. RALL. Quadratic equations in Banach spaces. Rend.Circ. Mat. Palermo, 10(2):314–332, 1961.

[9] X. YE, C. LI, and W. SHEN. Convergence of the variantsChebyshev-Halley iteration family under the Holder condi-tion of the first derivative. J. Comput. Appl. Math., 2006.

386386386386386386