a little on newton, raphson and halley, a little more on ... euroad workshop - trond steiha… ·...
TRANSCRIPT
![Page 1: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/1.jpg)
A little on Newton, Raphson and Halley,a little more on Simpson.
Why we should or should not care
Trond Steihaug1
1Department of Informatics, University of Bergen.Currently visiting scholar Oxford University
Tuesday July 2, 2019
22nd EuroAD Workshop, Imperial College 1-2 July 2019
Trond Steihaug Newton, Raphson, Halley & Simpson 1 / 21
![Page 2: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/2.jpg)
Content
Computational science existed long before the dawn of computers.
Four towering figures: Newton, Raphson, Halley and Simpson
The methods of Newton and Raphson
The classical test example of Newton
Halley’s method
Early “rediscoveries” of Halley’s method
Simpson
Other contributors (Ward, Maclaurin, Jones, . . . )
Kepler’s equation
History of numerical analysis is a part of history of mathematics
Trond Steihaug Newton, Raphson, Halley & Simpson 2 / 21
![Page 3: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/3.jpg)
This is not about derivatives . . .
Newton evaluating fluxions Griewank evaluating derivatives
Trond Steihaug Newton, Raphson, Halley & Simpson 3 / 21
![Page 4: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/4.jpg)
Nonlinear equations
Newton’s method first in print in Wallis Algebra 1685.
Raphson’s method in 1690, no reference to Wallis or Newton.
Halley’s (two methods) in 1694 with reference to Raphson
Newton
Let g0(s) = f (x0 + s)For k = 0, 1, 2, . . . ,m − 1
Compute gk(s) ≈ gk(s)Solve for sk in gk(s) = 0Let gk+1(s) = gk(sk + s)
x = x0 +∑m−1
k=0 sk
gk(s) = gk(sk) + g ′(sk)s
Raphson
For k = 0, 1, 2, . . . ,m − 1Compute gk(s) ≈ f (xk + s)Solve for sk in gk(s) = 0Let xk+1 = xk + sk
x = xm
Raphson: gk(s) = f (xk) + f ′(xk)s
Trond Steihaug Newton, Raphson, Halley & Simpson 4 / 21
![Page 5: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/5.jpg)
Newton’s method before Wallis 1685
De analysiThe method appears in the manuscript De analysi peræquationes numero terminorum infinitas around 1669.De analysi was printed in 1711 in an edition edited by WilliamJones.The commentary section of the translation to English of thiswork that Stewart in 1745 observes that the methods ofNewton and Raphson are the same.
De methodisAn updated version of the method appears in the manuscriptDe methodis fluxionum et serierum infinitarum written in 1671De methodis is translated and commented by Colson in 1736.Published in print in 1685. it is reasonable to assume thatWallis based his printed presentation in 1685 on a transcript ofthe manuscript from 1671.
Leipniz’ Excerpta from De analysi 1676. (Unknowndistribution)
Trond Steihaug Newton, Raphson, Halley & Simpson 5 / 21
![Page 6: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/6.jpg)
The test example of Newton 1669/71: f (x) = x3 − 2x − 5
Proponatur æquatio y3 − 2y − 5 = 0 resolvenda, Et sit 2numerus utcunque inventus qui minus quam decima sui partediffert a radice quæsita.Let the equation y3 − 2y − 5 = 0 be proposed for solution and letthe number 2 be found, one way or other, which differs from therequired root by less than its tenth part. Newton’s handwriting in the
manuscript De methodis MS Add. 3960.14, Cambridge University Library,
Cambridge, UK. Transcribed and translated by Whiteside
f (x) = x3 − 2x − 5 and x0 = 2
Trond Steihaug Newton, Raphson, Halley & Simpson 6 / 21
![Page 7: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/7.jpg)
Newton’s method f (x) = x3 − 2x − 5 and x0 = 2
Using the notation on previous slide we have:
g0(s) = f (2 + s) = s3 + 6s2 + 10s − 1 and s0 = 110 .
g1(s) = g0( 110 + s) =
= s3 +63
10s2 +
1123
100s +
61
1000and s1 = − 61
11230
.g2(s) = g1(− 61
11230 + s) =
= s3+35 283
5 615s2+
351 906 913
31 528 225s+
32 878 756
177 030 983 375and s2 = −
32 878 756
1 975 957 316 495
.
x = 2 + 110 −
6111230 −
32 878 7561 975 957 316 495 = 4 138 744 325 037
1 975 957 316 495 .
The sequence is 2, 2.1, 2.094568121104185, 2.094551481698199.
Trond Steihaug Newton, Raphson, Halley & Simpson 7 / 21
![Page 8: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/8.jpg)
Newton’s cubic equation
To simplify the computation Newton and Wallis aretruncating (in second iteration) 61/11230 to 0.0054. Thismakes computing s2 by hand easier.
The sequence of corrections or updates is then 0.1, -0.0054and -0.00004852 with final iterate x = 2.09455148
Final error 1.5 10−9 (3 iterations) (compared to the root).
-1 -0.5 0 0.5 1 1.5 2 2.5
x
-8
-6
-4
-2
0
2
4
6Newton Example
x3 -2x-5
-0.15 -0.1 -0.05 0 0.05 0.1
s
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5Newton shifted polynomials
g0(s)
g1(s)
g2(s)
Trond Steihaug Newton, Raphson, Halley & Simpson 8 / 21
![Page 9: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/9.jpg)
Raphson’s method
Raphson’s method (x0 = 2):
s0 = − f (2)f ′(2) = −1
10 and x1 = 2110 .
s1(s) = − f ( 2110
)
f ′( 2110
)= −
6110001123100
and x2 = 2110 −
6111230 = 11761
5615 .
s2 = − f ( 117615615
)
f ′( 117615615
)= −
32 878 756177 030 983 375
351 906 91331 528 225
= − 32 878 7561 975 957 316 495 .
x ≈ x3 = x2 + s2 = 4 138 744 325 0371 975 957 316 495 .
Raphson is generating the sequence of iterates 2.1, 2.0946,2.094551483, and 2.0945514815427104141 with an error of3.7 10−13 in the final iterate (4 iterations)
Trond Steihaug Newton, Raphson, Halley & Simpson 9 / 21
![Page 10: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/10.jpg)
Raphson and Halley
gk(s) = f (xk) + f ′(xk)s +1
2f ′′(xk)s2
Solving the quadratic equation gk(s) = 0 will give what is calledHalley’s irrational method or using the approximation√
a2 − b ≈ a− ab
2a2 − 12b
where a and b are real numbers giving Halley’s rational method.
Method Update sk
Raphson sk = − f (xk )f ′(xk )
Halley Rational sk = − f ′(xk )f (xk )
f ′(xk )2− 12f ′′(xk )f (xk )
Halley Irrational sk = − f ′(xk )−√
f ′(xk )2−2f (xk )f ′′(xk )
f ′′(xk )
Trond Steihaug Newton, Raphson, Halley & Simpson 10 / 21
![Page 11: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/11.jpg)
Halley’s rational method on f (x) = x3 − 2x − 5, x0 = 2
xk+1 = xk + sk , sk = − f ′(xk)f (xk)
f ′(xk)2 − 12 f′′(xk)f (xk)
Halley’s rational formula and two iterations
s0 = − f ′(2)f (2)
f ′(2)2− 12f ′′(2)f (2)
= 553 and x1 = 111
53 ,
f (x1) = 25·11533 , f
′(x1) = 5·6269532 and f ′′(x2) = 2·32·37
53 ,
s1 = 1103344052079190773 , x2 = 1 090 082 546 191
52 079 190 773
The sequence 2, 2.094339622641509, and 2.094551481540164with an error 2.2 10−12 in the final iterate.
Halley well aware of triple number of figures in the iterates.
Trond Steihaug Newton, Raphson, Halley & Simpson 11 / 21
![Page 12: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/12.jpg)
Early rediscovery of Halley rational method
Halley’s approximation
gk(s) = f (xk) + f ′(xk)s +1
2f ′′(xk)s2
Replace s2 ≈ − f (xk )f ′(xk )s.
The approximate function is then
gk(s) = f (xk) +
(f ′(xk)− 1
2
f (xk)
f ′(xk)f ′′(xk)
)s.
This derivation was used by Wolff in 1713, Holliday 1745 andMclaurin in 1748.This derivation is usually credited Frame from 1944.This derivation is also suitable for deriving Halley’s method tosystems of nonlinear equations. Let F : Rn 7→ Rn and
s(1)k = −F ′(xk)−1F (xk). Then(
F ′(xk) +1
2F ′′(xk)s
(1)k
)sk = F (xk), xk+1 = xk + sk .
This can be rewritten as
xk+1 = xk−
{I +
1
2L(xk)
[I − 1
2L(xk)
]−1}
(F ′(xk))−1F (xk), k = 0, 1, . . . ,
where
L(x) = (F ′(x))−1F ′′(x)(F ′(x))−1F (x), x ∈ Rn.
which is the ’standard’ way to express the Halley method.
Trond Steihaug Newton, Raphson, Halley & Simpson 12 / 21
![Page 13: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/13.jpg)
Simpson’s Contribution 1740
Raphson’s test cases:(9 out of 33) (to the left) and Simpson’s cases
Raphson x0 Iter
x2 − 2 1 4
x3 − 37945 33 3
x4 − 2741583974 229 2
x5 − 2327834559873 298 1
x2 + 587x − 987459 746 2
x2 − 5x − 31 8 3
−x2 + 8x − 14 2 4
x3 + 24x − 587914 83 2
−x3 + 300x − 1000 3 4
Simpson x0 Iter
−x3 + 300x − 1000 3.5 2√
1− x +√
1− 2x2 +√
1− 3x3 − 2 .5 2(y +
√y2 − x2 − 10
x +√
y2 + x − 12
) (56
)2 49
(x − x
(x+y)2
)− 25(1− x2
(1+y)2 )
81(1− x2
(1+y)2 )− 49( xy− xy
(1+x)2 )
(.8.6
)2
(xx + yy − 1000xy + yx − 100
) (4.52.5
)2
Simpson uses fluxion (derivative) notation.
Trond Steihaug Newton, Raphson, Halley & Simpson 13 / 21
![Page 14: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/14.jpg)
Other contributions
Wallis 1685, 1693 (Short discussion of Raphson’s method)
Halley 1694 contains three examples
Raphson 1690, 1697 (2nd edition). The Newton testcase istaken out!
Ward 1695
Sault 1694 (6 of Raphson’s examples)
Parson and Wastell 1704
Jones 1706 (Halley rational and irrational)
Wolff 1713 (Raphson and Halley) (Translated to English in1739)
Ronayne 17717 (Raphson)
Colson 1736 (Methods of Newton and Raphson nearly agree)
Simpson 1740 (Kepler’s equation) and 1745
Stewart 1745
Diderot 1751
Trond Steihaug Newton, Raphson, Halley & Simpson 14 / 21
![Page 15: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/15.jpg)
Newton’s and Raphson’s methods are the same
In 1745 Stewart made a translation and extensivecommentaries of Newton’s text De analysi transcribed andedited by Jones in 1711.
Stewart is the first to point out that the methods of Newtonand Raphson are the same:
And as to Mr. Raphson’s Method of Approximation in the ex-tracting the Roots of Equations, published in his Analysis Equa-tionum universalis, it is, in effect, the very same with our Au-thor’s Method here laid down;
He also derives Halley rational and irrational formulas.
In the expanded French translation in 1751 by Diderot of thedictionary by Chambers from 1728, Diderot uses Raphson’smethod but adds
Cette methode pour approcher les racines des equationsnumeriques, est due a M. Newton.
Trond Steihaug Newton, Raphson, Halley & Simpson 15 / 21
![Page 16: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/16.jpg)
The approach of Colin Maclaurin (1698–1746)
In Maclaurin treatise of algebra from 1748.
Let g0(s) = f (x0 + s)For k = 0, 1, 2, . . . ,m − 1
Compute gk(s) ≈ gk(s)Solve for sk in gk(s) = 0Let gk+1(s) = gk(sk + s)
x = x0 +∑m−1
k=0 sk
gk(s) = gk(sk) + g ′(sk)s + 12g′′(sk)s2
x3 − 15x2 + 63x − 50 = 0 and x3 − 2x − 5 = 0 with linearapproximation (The Newton way)
Substitution 1 + y for x in x3 − 15x2 + 63x − 50 = 0 gives−1 + 36y − 12y2 + y3 = 0. Maclaurin takes quadraticapproximation and and in effect use Halley’s rational method.
Better would be replace s2 ≈ − f (xk )f ′(xk )s and s3 ≈
(f (xk )f ′(xk )
)2s
Trond Steihaug Newton, Raphson, Halley & Simpson 16 / 21
![Page 17: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/17.jpg)
Kepler’s Equation M = E − e sin(E )
M = 2πt
T,
a = (semi)major axis ,
b = (semi)minor axis,
e =
√1− b2
a2
Derived by Newton in Philosophiae Naturalis PrincipiaMathematica 1687, 1713, 1726 (3rd ed).
Newton derived a solution technique that was shown by JohnCouch Adams in 1882 to be Newton-Raphson.
Newton gave no numerical example.
Trond Steihaug Newton, Raphson, Halley & Simpson 17 / 21
![Page 18: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/18.jpg)
Simpson and Kepler’s equation
Simpson is using the time after passing of the aphelion(farthest point in the elliptic orbit)
Simpson derived the equivalent equation
E + e sin(E ) = D
and a solution technique in 1740 (Newton–Raphson)
Example I: Equation E + 2.86479 sin(E ) = 72.21
Demonstrated (using degrees) with 2 iterations.
Simpson: Sequence: 70◦, 69◦ 31′ 16100
′′(69.536), and
69◦ 31′ 34′′ (69.5261)The sequence is (rounded) 70, 69.5261, 69.5262
Trond Steihaug Newton, Raphson, Halley & Simpson 18 / 21
![Page 19: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/19.jpg)
A (more) modern approach
Joseph Fenn was an Irish mathematics master. His New andComplete System of Algebra or Specious Arithmetic (1767) can beconsidered one of the first “modern” algebra texts. The term“specious arithmetic” is defined as “the calculus of quantities asdesignated by letters of the alphabet.”
Name f (x) x0 Iterations
Chap LXI√1− x +
√1− 2x2 +
√1− 3x3 − 2 .5 2
Chap LXIII
(y +
√y 2 − x2 − 10
x +√
y 2 + x − 12
) (56
)2
Chap LXIV
49(x − x
(x+y)2
)− 25(1− x2
(1+y)2 )
81(1− x2
(1+y)2 )− 49( xy− xy
(1+x)2 )
(.8.6
)2
All examples from Simpson using Leibniz notation.
Trond Steihaug Newton, Raphson, Halley & Simpson 19 / 21
![Page 20: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/20.jpg)
Concluding Remarks
“Newton = Raphson” was known around 1750
Traub, 1964: The Halley method must share with the secantmethod the distinction of being the most frequentlyrediscovered method in the literature.
Halley’s rational method rediscovered also in the 1700s.
Simpson first to consider “non-polynomial” equations.
Fourier of 1818 and 1831 the Newton-Raphson is attributedto Newton. The immense popularity of Fourier’s writing led tothe universal adoption of the name ”Newton’s method” forthe Newton-Raphson process
Joseph Lagrange in 1798 derives Raphson’s method and writesthat the Newton’s method and Raphson’s method are the samebut presented differently and Raphson’s method is plus simple quecelle de Newton.
Thank you for listeningTrond Steihaug Newton, Raphson, Halley & Simpson 20 / 21
![Page 21: A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less](https://reader033.vdocuments.us/reader033/viewer/2022060309/5f0a4f327e708231d42b0619/html5/thumbnails/21.jpg)
Worth Reading
Tjalling J. Ypma, Historical development of theNewton-Raphson method. SIAM Review Vol.37(4), pp. 531 -551, 1995.
Nick Kollerstrom. Thomas Simpson and ’Newton’s method ofapproximation’: an enduring myth. BJHS, Vol. 25, pp.347-54, 1992.
David J. Thomas. JOSEPH RAPHSON, F.R.S.. Notes Rec.R. Soc. Lond. Vol.4(4), pp. 151-167. 1990
Trond Steihaug. Computational science in the eighteenthcentury. Test cases for the methods of Newton, Raphson, andHalley: 1685 to 1745, Numerical Algorithms. 2019.https://doi.org/10.1007/s11075-019-00724-8,
Trond Steihaug Newton, Raphson, Halley & Simpson 21 / 21