solutions for exercisestrenchea/math1070/solutions_math1070_2020.pdf · 2020. 11. 16. · solutions...
TRANSCRIPT
Solutions for exercises1. Section 1.1 THE TAYLOR POLYNOMIAL.
pn(x;a) =n
∑i=0
(x−a)n
n!f (n)(a) (Taylor approximation polynomial)
≡ f (a)+11!(x−a) f ′(a)+
12!(x−a)2 f ′′(a)+
13!(x−a)3 f ′′′(a)+ · · ·+ 1
n!(x−a)n f (n)(a),
provided f , f ′, · · · , f (n) are well-defined at x = a.
Exercise 2. Produce the linear and quadratic Taylor polynomials for the following cases. Graph the function andthese Taylor polynomials.(a) f (x) =
√x, a = 1
(b) f (x) = sin(x),a = π
4(c) f (x) = exp(cos(x)), a = 0(d) f (x) = log(1+ ex), a = 0Solution:(a) f (x) = x1/2, f ′(x) = 1
2 x−1/2, f ′′(x) =− 14 x−3/2, hence f (1) = 1, f ′(1) = 1
2 , f ′′(1) =− 14 and
p2(x;1) = 1+12(x−1)− 1
8(x−1)2.
(b) f (x) = sin(x), f ′(x) = cos(x), f ′′(x) =−sin(x), hence f (π
4 ) =√
22 , f ′(π
4 ) =√
22 , f ′′(π
4 ) =−√
22 , and
p2
(x;
π
4
)=
√2
2+
√2
2
(x− π
4
)−√
24
(x− π
4
)2.
(c) f (x) = ecos(x), f ′(x) =−sin(x)ecos(x), f ′′(x) =(−cos(x)+ sin2(x)
)ecos(x), then f (0) = e, f ′(x) = 0, f ′′(x) =−e,
therefore
p2(x;0) = e− ex2
2.
(d) f (x) = log(1+ ex), f ′(x) = ex
1+ex , f ′′(x) = ex
(1+ex)2 , hence f (0) = log(2), f ′(0) = 12 , f ′′(0) = 1
4 , so
p2(x;0) = log(2)+12
x+18
x2.
Exercise 3. Produce a general formula for the degree n polynomials for the following functions, all using a = 0 asthe point of approximation. Graph the function and these Taylor polynomials.(a) f (x) = 1
1−x(b) f (x) = sin(x)(c) f (x) =
√1+ x
(d) f (x) = cos(x)(e) f (x) = (1+ x)1/3
Solution:1
(a) f (x) = 11−x = (1− x)−1, f ′(x) = (1− x)−2, f ′′(x) = 2(1− x)−3, f ′′′(x) = 3!(1− x)−4, · · · , f (n)(x) = n!(1−
x)−(n+1) = n! 1(1−x)n+1 , hence f (n)(0) = n!, so
11− x
≈ pn(x) = 1+ x+ x2 + · · ·+ xn about x = 0.
(b) f (x) = sin(x), f ′(x) = cos(x), f ′′(x) =−sin(x), f ′′′(x) =−cos(x), f iv(x) = sin(x), . . ., f (4m)(x) = (−1)2m sin(x),f (4m+1)(x) = (−1)2m+1 cos(x), f (4m+2)(x) = (−1)2m+2 sin(x), f (4m+3)(x) = (−1)2m+3 cos(x), . . . . Therefore alleven-order derivatives vanish when evaluated at a = 0, and the Taylor polynomial will contain only odd terms:
p2n(x;0) = x− x3
3!+
x5
5!−·· ·+(−1)n−1 x2n−1
(2n−1)!≈ sin(x) ∀n≥ 1. (1.1)
(c) f (x) =√
1+ x(d) f (x) = cos(x)
p2n(x;0) = 1− x2
2!+
x4
4!−·· ·+(−1)n x2n
(2n)!≈ cos(x) ∀n≥ 0. (1.2)
(e) f (x) = (1+ x)1/3
Exercise 4. Does f (x) = 3√
x have a Taylor polynomials approximation of degree 1, based on expanding about x = 0?x = 1? Explain and justify your answers.Solution: Since f (x) = 3
√x ≡ x1/3, f ′(x) = 1
3 x−23 , then f ′(0) is not well defined, while f ′(1) = 1
3 . Thereforep1(x;0) := f (0)+ x f ′(0) does not exist, while p1(x;1) := f (1)+(x−1) f ′(1)≡ 1+ 1
3 (x−1).>> x=linspace(0.5,1.5,100);
>> hold all
>> plot(x,x.^(1/3),’b-’)
>> plot(x,1+1/3*(x-1),’r.-.’)
Exercise 8.Let f (x) = ex; recall the formulae (1.3) for pn(x;0) and
pn(x;a) = ean
∑j=0
(x−a) j
j!.
Compare pn(x;0) and pn(x,1) to ex on [−1,2] for n = 1,2,3.Solution: For p1 comparison:>> x=linspace(-1,2,100);
>> hold all
>> plot(x,exp(x),’b-’,x,1+x,’r.-.’);
>> plot(x,exp(1).*(1+(x-1)),’m.-.’);
>> legend(’e^x’,’p_2(x,0)’,’p_2(x,1)’)
For p2 comparison:>> clear all
>> clc
>> x=linspace(-1,2,100);
>> hold all
>> plot(x,exp(x),’b-’,x,1+x+1/2*x.^2,’r.-.’);
>> plot(x,exp(1).*(1+(x-1)+1/2*(x-1).^2),’m.-.’)
>> legend(’e^x’,’p_2(x,0)’,’p_2(x,1)’)
2
For p3 comparison:>> clear all
>> clc
>> x=linspace(-1,2,100);
>> hold all
>> plot(x,exp(x),’b-’,x,1+x+1/2*x.^2+1/6*x.^3,’r.-.’);
>> plot(x,exp(1).*(1+(x-1)+1/2*(x-1).^2+1/6*(x-1).^3),’m.-.’)
>> legend(’e^x’,’p_3(x,0)’,’p_3(x,1)’)
Exercise 10.(a) Produce the Taylor polynomials of degree 1,2,3,4 for f (x) = ex2
with a = 0 the point of approximation.(b) Using the Taylor polynomials for et , substitute t = x2 to obtain polynomial approximations for ex2
. Comparewith the results in (a).Solution:
(a) We have that f (x) = ex2, f ′(x) = 2xex2
, f ′′(x) = (2 + 4x2)ex2, f ′′′(x) = (8x + 4x + 8x3)ex2
= (12x + 8x3)ex2,
f (4)(x) = (12+24x2 +24x2 +16x4)ex2= (12+48x2 +16x4)ex2
hence f (0) = 1, f ′(0) = 0, f ′′(0) = 2, f ′′′(0) =0, f (4)(0) = 12, and
p4(x) = 1+22!
x2 +124!
x4 ≡ 1+ x2 +12
x4.
(b) As x≈ 0, then also t = x2 ≈ 0, and the function f (t) = et has the Taylor approximation about t ≈ 0 being
et ≈ Tn(t) = 1+ t +12
t2 + · · ·+ 1n!
tn,
ex2 ≈ p2n(x) = 1+ x2 +12(x2)2 + · · ·+ 1
n!(x2)n.
The second degree Taylor polynomial approximation gives
et ≈ T2(t) = 1+ t +12
t2,
ex2 ≈ T2(x2) = 1+ x2 +12(x2)2 ≡ 1+ x2 +
12
x4,
Exercise 11. The quotient
g(x) =ex−1
x
is undefined for x = 0. Approximate ex by using Taylor polynomials of degrees 1, 2, and 3, in turn to determine anatural definition of g(0).Solution: The Taylor polynomial of degree n for ex for x≈ 0 is
pn(x;0) = 1+ x+12
x2 + · · ·+ 1n!
xn ≈ ex, (1.3)
hence
g(x)≈ pn(x;0)−1x
=�1+ x+ 1
2 x2 + · · ·+ 1n! xn−�1
x= 1+
12
x+ · · ·+ 1n!
xn−1 := qn−1(x).
3
Using continuity we can define g(0) = 1.Since q( j)
n−1(0) =1
j+1 ,∀ j = 0 : n−1, we can also define values for the derivatives of g at 0:
g( j)(0) =1
j+1, ∀ j = 0 : n−1.
Exercise 12. The quotient
g(x) =log(1+ x)
x
is undefined for x= 0. Approximate log(1+x) by using Taylor polynomials of degrees 1, 2, and 3, in turn to determinea natural definition of g(0).Solution: The Taylor polynomial of degree n for log(1+ x) for x≈ 0 is
pn(x;0) = x− 12
x2 +13
x3 + · · ·+ (−1)n−1
nxn ≈ log(1+ x), (1.4)
hence
g(x)≈ pn(x;0)x
=x− 1
2 x2 + 13 x3 + · · ·+ (−1)n−1
n xn
x= 1− 1
2x+
13
x2 + · · ·+ (−1)n−1
nxn−1 := qn−1(x).
Using continuity we can define g(0) = 1.Since q( j)
n−1(0) =(−1) j
j+1 j!,∀ j = 0 : n−1, we can also define values for the derivatives of g at 0:
g( j)(0) =(−1) j
j+1j!, ∀ j = 0 : n−1.
4
2. Section 1.2 THE ERROR IN TAYLOR’S POLYNOMIAL.
f (x) = pn(x;a)+Rn(x,a)
where the remainder (in differential form) is
Rn+1(x;a) :=(x−a)n+1
(n+1)!f (n+1)(cx)∈ Pn+1
provided f , f ′, · · · , f (n), f (n+1) are well-defined in a vicinity of x = a, and cx is a point between a and x.
Exercise 2. Find the degree 2 Taylor polynomial for f (x) = ex sin(x), about the point a = 0. Bound the error in thisapproximation when x ∈ [−π
4 ,π
4 ].Solution: Since f ′(x) = ex
(sin(x)+ cos(x)
), f ′′(x) = ex
(sinx+2cosx− sinx
)= 2ex cos(x), f ′′′(x) = 2ex
(cos(x)−
sinx), we have
f (x) = p2(x;0)+R3(x;0),
p2(x;0) = x+x2
2!2 = x+ x2, R3(x;0) =
13!
x32ecx(
cos(cx)− sin(cx))
where cx is between 0 and x. Then for x ∈ [−π
4 ,π
4 ] we have
|R3(x;0)| ≤ 13!
π3
43 2∣∣∣ecx(
cos(cx)− sin(cx))∣∣∣︸ ︷︷ ︸
≤1
≤ π3
192≈ 0.1615.
Exercise 5. How large should the degree 2n−1 be chosen in
sin(x) = x− x3
3!+
x5
5!−·· ·+(−1)n−1 x2n−1
(2n−1)!︸ ︷︷ ︸p2n−1(x;0)
+(−1)n x2n+1
(2n+1)!cos(c)︸ ︷︷ ︸
R2n+1(x)
(2.1)
to have
|sin(x)− p2n−1(x)| ≤ 0.001 ∀x ∈[− π
2,
π
2
]?
Check your results by evaluating the resulting p2n−1(x) at x = π
2 .Solution: The remainder
R2n+1(x)≡ sin(x)− p2n−1(x;0) = (−1)n x2n+1
(2n+1)!cos(c)
(where c is between x and 0, hence c ∈[− π
2 ,π
2
]) can be bounded above by
∣∣∣R2n+1(x)∣∣∣≤ (π/2)2n+1
(2n+1)!
5
therefore ∣∣∣R2n+1(x)∣∣∣≤ 0.001
provided
(π/2)2n+1
(2n+1)!≤ 0.001.
Since {1000
(π/2)2n+1
(2n+1)!
}n=3:5
= {4.6818,0.1604,0.0036},
the error is satisfied for all n≥ 4.Let us check if the value of |R9(π/2)|= |p7(π/2)− sin(π/2)| ≤ 0.001?. Indeed,
|p7(π/2)− sin(π/2)|=∣∣∣π
2− (π/2)3
3!+
(π/2)5
5!− (π/2)7
7!−1∣∣∣≡ 0.0001569 < 0.001.
Exercise 6. Let pn(x;0) be the Taylor polynomial of degree n of the function f (x) = log(1− x) about a = 0. Howlarge n should be chosen to have | f (x)− pn(x;0)| ≤ 10−4 for x ∈ [− 1
2 ,12 ]? For x ∈ [−1, 1
2 ]?Solution: Note that by the approximation in Exercise 16 below (setting t =−x) we have
log(1− x) =−x− 12
x2− 13
x3−·· ·− 1n
xn− 1n+1
xn+1 1(1− cx)n+1 ,
(with cx between 0 and x). We now remark that for 0≤ cx ≤ x≤ 12 we have
0≥−cx ≥−x≥−12, 1≥ 1− cx ≥ 1− x≥ 1
2, 1≤ 1
1− cx≤ 1
1− x≤ 2,
so
xn+1 ≤ xn+1
(1− cx)n+1 ≤xn+1
(1− x)n+1 ≤ 1,
which gives a ”not so good” approximation. So we need something else!Use formula (1.20) in the book:
log(1− x) =−(
x+12
x2 + · · ·+ 1n
xn)− 1
1− cx
xn+1
n+1(1.20)
with cx between 0 and x, so the remainder is
|Rn+1(x)| ≤1
|1− cx||xn+1|n+1
and since |x/(1− cx)| ≤ 1 for x ∈ [−1/2,1/2],
|Rn+1(x)| ≤|xn|
n+1≤ 1
2n(n+1)
which holds for n≥ 10.For x ∈ [−1, 1
2 ]
|Rn+1(x)| ≤|xn|
n+1≤ 1
2(n+1)6
which holds for n≥ 4999.
Exercise 8. How large should n be chosen in
ex = 1+ x+12
x2 + · · ·+ 1n!
xn︸ ︷︷ ︸pn(x;0)
+xn+1
(n+1)!ec︸ ︷︷ ︸
Rn+1(cx)
(2.2)
(where c is between x and 0) to have
|ex− pn(x;0)| ≤ 10−5, ∀x ∈ [−1,1].
Solution: Since we can bound the remainder on [−1,1] by∣∣ xn+1
(n+1)!ec∣∣≤ 1
(n+1)!e
(as the exponential function is increasing) the error will be satisfied provided 1(n+1)! e≤ 10−5, i.e.,
105e≤ (n+1)!,
which holds for all n≥ 8.(271,828.1828459046 versus 8! = 40,320.00,9! = 362,880.00)
Exercise 9. Use Taylor polynomials with remainder term to evaluate the following limits:(a) limx→0
1−cos(x)x2
(b) limx→0log(1+x2)
2x
(c) limx→0log(1−x)+xex/2
x3
Hint: Use Taylor polynomials for the standard functions [e.g., cos(t), log(1+ t) and et] to
obtain polynomial approximations to the numerators of these fractions; and then simplify
the results.
Solution:(a) Recall that for the cos(x) we have about 0 the following Taylor expansion
cos(x) = 1− x2
2!+
x4
4!−·· ·+(−1)n x2n
(2n)!+(−1)n+1 x2n+2
(2n+2)!cos(cx). (2.3)
For n = 1 we then have
1− cos(x)x2 =
1−(1− x2
2! +x4
4! cos(cx))
x2 =12!− x2
4!cos(cx)→
12,
since lim→0x2
4! cos(cx) = 0.(b) Using log(1+u) = u− 1
2 u2 1(1+cu)2 with cu between u and 0, we have that
log(1+ x2) = x2 +12
x4 1(1+ c2
x)2
and
limx→0
log(1+ x2)
2x= 0.
7
(c) limx→0log(1−x)+xex/2
x3
Exercise 10. Verify
11− x
= 1+ x+ · · ·+ xn +xn+1
1− x. (2.4)
Solution: This is a trivial identity: move the last therm from the right-hand side to the left-hand side, and thenmultiply by (1− x):
1− xn+1 = (1+ x+ · · ·+ xn)(1− x)
Exercise 13. Evaluate
I =∫ 1
0
ex−1x
dx
within an accuracy of 10−6.Hint: Replace ex by a general Taylor polynomial approximation plus its remainder.
Solution: From (2.2) we have (with cx between 0 and x)
ex = 1+ x+12!
x2 + · · ·+ 1n!
xn +xn+1
(n+1)!ecx
hence the integrand writes
ex−1x
=�1+ x+ 1
2 x2 + · · ·+ 1n! xn + xn+1
(n+1)! ecx −�1x
= 1+12!
x+ · · ·+ 1n!
xn−1 +xn
(n+1)!ecx
and the integral becomes
I =∫ 1
0
ex−1x
dx =∫ 1
0
(1+
12!
x+ · · ·+ 1n!
xn−1 +xn
(n+1)!ecx)
dx
=(
x+1
2 ·2!x2 + · · ·+ 1
n ·n!xn)∣∣∣1
0+∫ 1
0
xn+1
(n+1) · (n+1)!ecx dx
= 1+1
2 ·2!+ · · ·+ 1
n ·n!+∫ 1
0
xn+1
(n+1) · (n+1)!ecx dx.
The error term is then∫ 1
0
xn+1
(n+1) · (n+1)!ecx dx≤
∫ 1
0
xn+1
(n+1) · (n+1)!edx =
e(n+1) · (n+1)!
(since cx is between 0 and x, is then also in the interval [0,1], and the exponential function being increasing). Theerror being less than 10−6 is guaranteed for n such that
e(n+1) · (n+1)!
≤ 10−6, ore ·106
(n+1) · (n+1)!≤ 1,
{ e ·106
(n+1) · (n+1)!
}n=7;8;9
={
8.43;0.83;0.07}
hence for all n≥ 8. This implies
I ≈ 1+1
2 ·2!+ · · ·+ 1
8 ·8!≈ 1.317901815239985
8
>> S = 0;
>> for i=1:8
S = S + 1/(i*factorial(i));
end
>> S
S =
1.317901815239985
Exercise 14.(a) Obtain a Taylor polynomial with remainder for f (t) = 1
1+t2 about a = 0.Hint: substitute x =−t2 into (2.4)
11− x
= 1+ x+ · · ·+ xn +xn+1
1− x. (*)
(b) Obtain a Taylor polynomial withe remainder for g(x) = arctan(x). Do this by integrating the result in (a) andusing
arctan(x) =∫ x
0
11+ t2 dt.
Solution:(a) Using the substitution x =−t2 in (2.4)/(*) we obtain
11+ t2 = 1− t2 + t4− t6 + · · ·+(−1)nt2n +(−1)n+1 t2(n+1)
1+ t2 .
(b) Integrating we obtain
arctan(x) =∫ x
0
11+ t2 dt =
∫ x
0
11+ t2 dt =
∫ x
0
(1− t2 + t4− t6 + · · ·+(−1)nt2n +(−1)n+1 t2(n+1)
1+ t2
)dt
= x− x3
3+
x5
5− x7
7+ · · ·+(−1)n x2n+1
2n+1+(−1)n+1
∫ x
0
t2(n+1)
1+ t2 dt.
Therefore
arctan(x) = p2n+1(x)+R2n+3(x),
where the Taylor polynomial is
p2n+1(x) = x− x3
3+
x5
5− x7
7+ · · ·+(−1)n x2n+1
2n+1,
and the remainder (obtained by the mean value theorem for definite integrals) is
R2n+3(x) = (−1)n+1∫ x
0
t2(n+1)
1+ t2 dt = (−1)n+1 11+ c2
x
∫ x
0t2(n+1)dt = (−1)n+1 1
1+ c2x
x2n+3
2n+3, (2.5)
with cx between 0 and x.
Exercise 15. Define f (x) =∫ x
01−e−t
t dt. Find a Taylor polynomial approximation of degree 2 for f (x). Give a formulafor the approximation error, bounding it on the interval 0≤ x≤ δ with δ > 0. What is the error bound for δ = 0.1?
9
Solution: Using the Taylor approximation for ex, (see expression (2.2)),
ex = 1+ x+12
x2 + · · ·+ 1n!
xn︸ ︷︷ ︸pn(x;0)
+xn+1
(n+1)!ec︸ ︷︷ ︸
Rn+1(cx)
with cx is between x and 0,
substitute x =−t
e−t = 1− t +12
t2−·· ·+ (−1)n
n!tn︸ ︷︷ ︸
pn(t;0)
+(−1)n+1 tn+1
(n+1)!ec
t︸ ︷︷ ︸Rn+1(ct )
with ct is between −t and 0,
and obtain
f (x) =∫ x
0
1− e−t
tdt =
∫ x
0
�1−(�1− t + 1
2 t2−·· ·+ (−1)n
n! tn− (−1)n+1 tn+1
(n+1)! ect
)t
dt
=∫ x
0
t− 12 t2 + · · ·− (−1)n
n! tn +(−1)n+1 tn+1
(n+1)! ect
tdt =
∫ x
0
(1− 1
2t + · · ·− (−1)n
n!tn−1 +(−1)n+1 tn
(n+1)!ect)
dt
= x− 122 x2 + · · ·− (−1)n
n ·n!xn +
(−1)n+1
(n+1)!
∫ x
0tnect dt.
Therefore the Taylor approximation of degree 2 for f (x) is obtained for n = 2
p2(x;0) = x− 122 x2,
and the error term can be bounded as follows (for x ∈ [0,δ ])∣∣R3(x;0)∣∣= ∣∣∣− 1
6
∫ x
0t2ect dt
∣∣∣≤ 16
∫ x
0t2etdt ≤ 1
6
∫ x
0t2exdt = ex 1
6
∫ x
0t2dt =
x3
3ex ≤ δ 3
18eδ .
When δ = 0.1, the error bound is ∣∣R3(x;0)∣∣≤ δ 3
3eδ ≈ 6.1398 ·10−5.
Exercise 16. Define
g(x) =∫ x
0
log(1+ t)t
dt.
(a) Give a Taylor polynomial approximation to g(x) about x = 0.(b) Bound the error in the degree n approximation for |x| ≤ 1
2 .(c) Find n so as to have a Taylor approximation with an error at most 10−7 on [− 1
2 ,12 ].
Solution: The Taylor polynomial approximation with remainder for log(1+ t) about a = 0 (see (1.4)) is
log(1+ t) = t− 12
t2 +13
t3 + · · ·+ (−1)n−1
ntn +
1(n+1)!
tn+1{log(1+ t)(n+1)}|t=ct
= t− 12
t2 +13
t3 + · · ·+ (−1)n−1
ntn +
1(n+1)!
tn+1(−1)nn!(1+ ct)−(n+1)
= t− 12
t2 +13
t3 + · · ·+ (−1)n−1
ntn +
(−1)n
n+1tn+1 1
(1+ ct)n+1 , (2.6)
10
(a) Then we can approximate g(x) by
g(x) =∫ x
0
log(1+ t)t
dt ≈∫ x
0
t− 12 t2 + 1
3 t3 + · · ·+ (−1)n−1
n tn
tdt
=∫ x
0
(1− 1
2t +
13
t2 + · · ·+ (−1)n−1
ntn−1
)dt
= x− x2
4+
x3
9+ · · ·+(−1)n−1 xn
n2 .
(b) The error in the degree n approximation, using (2.6), is
∫ x
0
(−1)n
n+1 tn+1 1(1+ct )n+1
tdt =
(−1)n
n+1
∫ x
0tn 1(1+ ct)n+1 dt,
hence it can be bounded for |x| ≤ 12 as follows∣∣∣ (−1)n
n+1
∫ x
0tn 1(1+ ct)n+1 dt
∣∣∣≤ 1n+1
∣∣∣∫ x
0tndt
∣∣∣= 1(n+1)2 |x|
n+1 ≤ 12n+1(n+1)2 .
(c) The estimate above yields that an error of 10−7 is obtained provided
12n+1(n+1)2 ≤ 10−7, 2n+1(n+1)2 ≥ 107
>> n=10:13;
>> 2.^(n+1).*(n+1).^2
ans =
247808 589824 1384448 3211264
which means any n≥ 12.
Exercise 19. Find the Taylor polynomial about x = 0 for
f (x) = log(1+ x
1− x
), x ∈ [−1,1].
Solution: First we rewrite f (x) as
f (x) = log(1+ x
1− x
)= log(1+ x)− log(1− x)
and use (2.6) with t = x and t =−x respectively to obtain
f (x) = log(1+ x)− log(1− x)
= x− 12
x2 +13
x3 + · · ·+ (−1)n−1
nxn +
(−1)n
n+1xn+1 1
(1+ cx)n+1
−(− x− 1
2x2 +
13(−x)3 + · · ·+ (−1)n−1
n(−x)n +
(−1)n
n+1(−x)n+1 1
(1+ c−x)n+1
)11
= 2(
x− 12
x2 +13
x3 + · · ·+ (−1)n−1
nxn)
+(−1)n
n+1xn+1 1
(1+ cx)n+1 +(−1)n
n+1(−x)n+1 1
(1+ c−x)n+1 .
Therefore the Taylor approximation polynomial is
pn(x;0) = 2(
x− 12
x2 +13
x3 + · · ·+ (−1)n−1
nxn). (**)
and the error term can be bounded on [−1,1] as∣∣∣Rn+1(x;0)∣∣∣= ∣∣∣ (−1)n
n+1xn+1 1
(1+ cx)n+1 +(−1)n
n+1(−x)n+1 1
(1+ c−x)n+1
∣∣∣≤ 1
n+1|x|n+1 1
|1+ cx|n+1 +1
n+1|x|n+1 1
|1+ c−x|n+1
≤ 1n+1
|x|n+1 1|1+ cx|n+1 +
1n+1
|x|n+1 1|1+ c−x|n+1 ≤
1n+1
|x|n+1(
1+1
|1+ x|n+1
). (***)
Exercise 20.
(a) Using the previous problem give a Taylor polynomial approximation to log(2), and bound the error.Hint: what is the needed value of x in order to use the previous results.
(b) Consider evaluating log(z) for z ∈ [1/2,1]. Use the previous problem to bound the error.Solution:(a) Setting 1+x
1−x = 2 we obtain x =− 13 , and therefore
log(2)≈ pn
(−1
3;0)= 2(− 1
3− 1
2132 −
13
133 −·· ·−
1n
13n
),
with the error bound∣∣∣Rn+1
(− 1
3;0)∣∣∣≤ 1
(n+1)3n+1
(1+
3n+1
2n+1
)=
1(n+1)
( 13n+1 +
12n+1
).
(b) As above, we set 1+x1−x = z for z ∈ [ 1
2 ,1], hence x = z−1z+1 ∈ [− 1
3 ,0] and using (**) we obtain
log(z)≈ pn
( z−1z+1
;0)= 2( z−1
z+1− 1
2(z−1)2
(z+1)2 +13(z−1)3
(z+1)3 + · · ·+ (−1)n−1
n(z−1)n
(z+1)n
),
with the error bound given by (***)∣∣∣Rn+1(z;0)∣∣∣≤ 1
n+1|z−1|n+1
|z+1|n+1
(1+|z+1|n+1
(2z)n+1
)=
1n+1
|z−1|n+1( 1|z+1|n+1 +
1(2z)n+1
).
Exercise 21.(a) Consider evaluating π by using
π = 4arctan(1).
Using the results of Exercise 14, how many terms would be needed on the Taylor approximation
arctan(x)≈ p2n−1(x)
to calculate π with an accuracy of 10−10? Is this a practical method of evaluating π?12
(b) Suggest another computation of π using the series for arctan(x), giving a more rapidly convergent method.Solution:(a) In order to have the error in
π = 4arctan(1) = 4p2n−1(1;0)+4R2n+3(1)≈ 4p2n−1(1;0)
to be 10−10, by (2.5) it is sufficient to have (recall that cx is now between 0 and 1)
4∣∣∣R2n+3(1)
∣∣∣≡ 4∣∣∣(−1)n+1 1
1+ c2x
12n+3
2n+3
∣∣∣= 42n+3
∣∣∣ 11+ c2
x
∣∣∣≤ 42n+3
≤ 10−10
which holds for n≥ 2 ·1010− 32 , or n≥ 2 ·1010, a very large number of terms!!!
(b) Let us use similarly
π = 6arctan( 1√
3
)= 6p2n−1
( 1√3
;0)+6R2n+3
( 1√3
)≈ 6p2n−1(
√3;0)
with the error (cx between 0 and 1√3)
6∣∣∣R2n+3
( 1√3
)∣∣∣≡ 6∣∣∣(−1)n+1 1
1+ c2x
13n+3/22n+3
∣∣∣= 62n+3
13n+3/2 ≤ 10−10.
Then the number of terms would be n≥ 18.>> n=linspace(16,19,4);
>> e=6*10^(10)./((2*n+3).*(3.^(n+3/2)))
e =
7.6641 2.4166 0.7642 0.2423
13
3. Section 1.3 POLYNOMIAL EVALUATION. For a polynomial of degree n
pn(x) = a0 +a1x+a2x2 + ·+anxn, a0 6= 0
the nested multiplication method
pn(x) = a0 + x(
a1 + x(a2 + · · ·x(an−1 +anx)
)· · ·)
uses only n multiplications.
Exercise 9. Evaluate
p(x) = 1− 13!
x3 +16!
x6− 19!
x9 +1
12!x12− 1
15!x15
as efficiently as possible. How many multiplications are necessary? Assume all coefficients have been computed andstored for later use.Solution: It takes two multiplications to evaluate x3. Then write
p(x) = 1+ x3{− 1
3!+ x3
[ 16!
+ x3(− 1
9!+ x3( 1
12!− 1
15!x15))]}
which would take five more multiplications; hence for any x it takes seven multiplications to evaluate p(x).
Exercise 10. Show how to evaluate the function
f (x) = 2e4x− e3x +5ex +1
efficiently.Hint: consider letting z = ex.
Solution: With z = ex, the function writes
f (x) = 2z4− z3 +5z+1 = 1+ z(
5+ z2(−1+2z))
14
4. Section 2.1 FLOATING-POINT NUMBERS.”Scientific notation” of a non-zero real number (in base 10):
y = σ · y ·10e
where
σ =+1 or −1 (sign)y ∈ [1,10) (significand)e is an integer (exponent)
while real number in binary format (base 2)
x = σ · x ·2e
where
σ =+1 or −1 (sign)x ∈ [1,2) (significand)e is an integer. (exponent)
DEFINITION 4.1 (IEEE single precision floating-point representation). The set of single precision floating-pointnumbers is denoted by
F(2,24,−126,127) = {0}∪{
x ∈ R; x = (−1)σ 2e(1+ 23
∑i=1
ai2−i)}where
• 2 represents the base 2,• 24 is the precision of x (i.e., x = 1.a1a2 · · ·a23, with 24 (!!!) positions represented exactly; note that here
ai ∈ {0,1} are arbitrary in base 2),• the exponent e has limited values: −126≤ e≤ 127.
REMARK 1.• The single floating-point numbers occupy 32 bits of memory: 1 position for the sign σ , 23 by the significand
(mantissa), and 8 positions for the exponent e: 28 = 256 = all the exponents between -126 and 127.• The LARGEST INTEGER which can be represented exactly in single precision is
M = 224 = 16,777,216
function [infinity] = bug_SeriesOf1
snew = 1;
sold = 0;
while snew - sold > eps
sold = single(snew);
snew = single(sold+1);
end
infinity = snew;
fprintf(’The values of infinity in single precision is %d\n’,infinity);
end
15
>> bug_SeriesOf1
The values of infinity in single precision is 16777216
ans =
single
16777216
• The number of single precision floating point numbers (the cardinal of the set F(2,23,−126,127)) is
1+2∗222 ∗ (254) = 2,130,706,433
• The smallest and the largest single precision numbers are
xmin = 2−126 = 1.175494350822288e−38
xmax = 2128 ∗ (1−2−23) = 3.402823263561193e+38
DEFINITION 4.2 (IEEE double precision floating-point representation). The set of double precision floating-point numbers is denoted by
F(2,53,−1022,1023) = {0}∪{
x ∈ R; x = (−1)σ 2e(1+ 52
∑i=1
ai2−i)}where
• 2 represents the base 2,• 53 is the precision of x (i.e., x = 1.a1a2 · · ·a23 · · ·a52, with 53 (!!!) positions represented exactly; note that
here ai ∈ {0,1} are arbitrary in base 2),• the exponent e has limited values: −1022≤ e≤ 1023.
REMARK 2.• The double floating-point numbers occupy 64 bits of memory: 1 position for the sign σ , 52 by the significand
(mantissa) [the first position of 1 is not counted], and 8 positions for the exponent e: 211 = 2048 = all theexponents between -1022 and 1023.
• The LARGEST INTEGER which can be represented exactly in double precision is
M = 253 = 9007199254740992 (9.007199254740992e+15)
>> factorial(18)+1-factorial(18)
ans =
1
>> factorial(19)+1-factorial(19)
ans =
0
>> factorial(18)
16
ans =
6.402373705728000e+15
>> factorial(19)
ans =
1.216451004088320e+17
Check that:>> factorial(18)+ 100 - factorial(18) = 100
ans =
100
>> factorial(19)+ 100 - factorial(19)
ans =
96
>> factorial(20)+ 100 - factorial(20)
ans =
0
For n≥ 19, n! has the correct order of magnitude, but only the first 15 digits are accurate!• The number of double precision floating point numbers (the cardinal of the set F(2,53,−1022,1023)) is
1+2∗251 ∗ (2046) = 9.214364837600035e+18
(The size of Universe - number of atoms - is about 1080.)• The smallest and the largest double precision numbers are
xmin = 2−1022 = 2.225073858507201e−308
xmax = 21023 ∗ (1−2−52) = 8.988465674311578e+307MATLAB : 170! = 7.257415615307994e+306 < xmax, 171! = Inf
DEFINITION 4.3 (machine epsilon). ε is the distance between 1 and the nearest floating-point number, i.e., isthe smallest number in F(β , t,L,U) such that 1+ ε > 1, namely
ε = β1−t (epsilon machine)
>> eps
ans =
2.220446049250313e-16
The machine epsilon (β−t )
in single precision : 2−23 = 1.192092895507812e−07
in double precision : 2−52 = 2.220446049250313e−16
17
REMARK 3. In an interval of the form [2e,2e+1), the floating-point numbers are equally spaced, at absolutedistance
2e+1−t ,
where t is the number of significant digits.Therefore in single precision, the distance is
2e−23, with −127≤ e≤ 126,
while in double precision, the distance is
2e−52, with −1022≤ e≤ 1023.
Indeed, two consecutive numbers in this interval are of the form 2e ·1.a1 · · ·a52 and 2e ·1.a1 · · ·{a52 +1}, hence thedifference 2e ·2−52.
REMARK 4. Unlike the absolute distance, the relative distance between two consecutive floating-point num-bers has a periodic behaviour, and depends only on the mantissa (the integer formed by the significant digits)1a1a2 · · ·at−1.Indeed, relative distance between two consecutive numbers 2e+1−t ·1a1 · · ·at−1 and 2e+1−t ·1a1 · · ·{at−1 +1}, is
∆xx
=2e+1−t ·1a1 · · ·{at−1 +1}−2e+1−t ·1a1 · · ·at−1
2e+1−t ·1a1 · · ·at−1=
2e+1−t
2e+1−t ·1a1 · · ·at−1=
11a1 · · ·at−1
∈[ 1
2t −1,
12t−1
]DEFINITION 4.4. A real number x = σ · 1.a1a2 · · ·at−1 · · ·2e (written in base β = 2) has a floating-point
representation
f l(x) =σ ·1.a1a2 · · · at−1at · · · ·2e ( f l(x))
x =σ ·1.a1a2 · · ·at−1at · · · ·2e
where
at−1 =
= at−1 if at <
β
2
or
at−1 +1 if at ≥ β
2
, (rounding)
alternatively
at−1 = at−1. (chopping)
REMARK 5.• f l(x) ∈ F, ∀x ∈ R.• f l(x) = x, ∀x ∈ F.• ∀x,y ∈ R we have that f l(x)≤ f l(y). (monotonicity property)
REMARK 6. Note that the real number x = 0.1 has a finite representation in base 10:
(0.1)10 = 1 ·10−1,
while in base 2 has in infinite representation:
x10 = (0.1)10 =(0.001100110011001100 · · ·
)2 = 1.100110011001100 · · · ∗2−3 =: x2,
18
and therefore its floating-point representation/approximation does not coincide with it:
x2 :=1.100110011001100 · · ·at−2at−1︸ ︷︷ ︸t significant digits
|ataa+1 · · ·001100 · · · ∗2−3
f l(x2) =1.100110011001100 · · ·at−2at−1︸ ︷︷ ︸t significant digits
|ataa+1 · · ·001100 · · · ∗2−3,
DEFINITION 4.5. We remark that everything written so far holds for numbers which have the exponent e withinthe range from the definition of F.
• For R 3 x ∈ (−∞,−xmax)∪ ( xmax︸︷︷︸βU (1−β )
,∞), f l(x) is not defined !!! (overflow)
• If x ∈ (−xmin, xmin︸︷︷︸β L−1
) the operation of rounding f l(x) is defined (anyway). (underflow)
MATLAB : 2−1074 = 4.940656458412465e−324, 2−1075 = 0
PROPOSITION 4.6. For a real number x ∈ R such that xmin≤ x≤ xmax it can be shown that
f l(x) = x(1+δ ), with |δ | ≤ u
where u = 12 β 1−t ≡ 1
2 ε is the roundoff unit (or machine precision).
Exercise 5. The following MATLAB program produced the given output. Explain the results.
(4.1)
x = 0.0;
while x < 1.0
x = x + 0.1;
%disp([x, sqrt(x)]);
end
disp([x(end), sqrt(x(end))]);
>> Exercise2_2_5
1.100000000000000 1.048808848170151
Solution: The intention was to STOP at x = 1.0, but due to floating-point representation (the decimal number 0.1 isnot represented exactly due to rounding) the number 1.0 is not obtained exactly by the incrementation. Therefore,the code stops only after the 11th round of the loop.
If we try insteadx = 0.0; while x <= 1.0-eps
x = x + 0.1;
%disp([x, sqrt(x)]);
end
disp([x(end), sqrt(x(end))]);
we obtain only 10 iterations in the loop>> Exercise2_2_5
1.000000000000000 1.000000000000000
Recall that by Remark 6 , (0.1)10 has an infinite representation in base 2
(0.1)10 =(0.001100110011001100 · · ·
)2,
19
hence its 64 bits floating-point approximation has a rounding error in the 53rd binary significant digit.Exercise 6. Let x ∈ R+ be a positive real number:
x = 1.a1a2 · · ·at−1|at · · · ·2e
Consider a computer using a positive binary floating-point representation with t bits of precisions in the significand,i.e.,
f l(x) =1.a1a2 · · · at−1at · · · ·2e ( f l(x))
x =1.a1a2 · · ·at−1at · · · ·2e.
Assume that chopping (rounding towards zero) is used in going from x ∈R outside the computer to its floating-pointrepresentation f l(x) inside the computer.(a) Show that
0≤ x− f l(x)≤ 2e−t+1
(b) Show that x≥ 2e, and use this to show
x− f l(x)x
≤ 2−t+1.
(c) Let
x− f l(x)x
=−ε
and then solve for f l(x). What are the bounds on ε?(This result extends to x < 0, with the assumption of x > 0 being used to simplify the algebra.)Solution: If chopping is used, then
f l(x) =1.a1a2 · · ·at−1at · · · ·2e
x =1.a1a2 · · ·at−1at · · · ·2e,
x− f l(x) =(1.a1a2 · · ·at−1at · · ·−1.a1a2 · · ·at−1
)·2e = 0.0102 · · ·0t−1|at · · · ·2e
= 0.at · · · ·2e−(t−1). (*)
(a) Using (*) it is easy to see that
0≤ x− f l(x)≤ 2e−t+1.
(b) Obviously x≥ 2e, and therefore
0≤ x− f l(x)x
≤ 2e−t+1
2e = 2−t+1.
(c) This means
f l(x) = x(1+ ε)
and from above we have ∣∣∣x− f l(x)x
∣∣∣= |ε| ≤ uchopping = 2−t+1
which in double precision (t = 53) gives uchopping = 2.220446049250313e−16.20
Exercise 7. Let x ∈ R+ be a positive real number:
x = 1.a1a2 · · ·at−1|at · · · ·2e
Consider a computer using a positive binary floating-point representation with t bits of precisions in the significand,i.e.,
f l(x) =1.a1a2 · · · at−1at · · · ·2e ( f l(x))
x =1.a1a2 · · ·at−1at · · · ·2e.
Assume that rounding is used in going from x ∈ R outside the computer to its floating-point approximation f l(x)inside the computer.(a) Show that
−2e−t ≤ x− f l(x)≤ 2e−t
(b) Show that x≥ 2e, and use this to show
x− f l(x)x
≤ 2−t .
(c) Let
x− f l(x)x
=−ε
and then solve for f l(x). What are the bounds on ε?(This result extends to x < 0, with the assumption of x > 0 being used to simplify the algebra.)
Solution: If rounding is used, then we have two options• If at = 0, then at−1 = at−1, and
f l(x) =1.a1a2 · · ·at−1 at︸︷︷︸0
at+1 · · · ·2e
x =1.a1a2 · · ·at−1 at︸︷︷︸0
at+1 · · · ·2e,
x− f l(x) =(1.a1a2 · · ·at−10at+1 · · ·−1.a1a2 · · ·at−1
)·2e = 0.0102 · · ·0t−1|0tat+1 · · · ·2e
= 0.at+1 · · · ·2e−t ≤ 2e−t . (**)
• If at = 1, then at−1 = at−1 +1, and
f l(x) =1.a1a2 · · ·at−1 +1 at︸︷︷︸1
at+1 · · · ·2e
x =1.a1a2 · · ·at−1 at︸︷︷︸1
at+1 · · · ·2e,
so
f l(x)− x =(1.a1a2 · · ·at−1 +1−1.a1a2 · · ·at−11at+1 · · ·
)·2e
=(0.0102 · · ·1t−1−0.0102 · · ·0t−11at+1 · · ·
)·2e
= (1.00−0.1at+1 · · ·︸ ︷︷ ︸≤0.1
) ·2e−(t−1) ≤ 2−1 ·2e−t+1 = 2e−t (***)
21
(a) Using (**) and (***) we see that
−2e−t ≤ x− f l(x)≤ 2e−t .
(b) Obviously x≥ 2e, and therefore
0≤ |x− f l(x)||x|
≤ 2e−t
2e = 2−t .
(c) This means
f l(x) = x(1+ ε)
and from above we have ∣∣∣x− f l(x)x
∣∣∣= |ε| ≤ urounding = 2−t
which in double precision (t = 53) gives the machine precision in double precision on 64 bits
urounding = 1.110223024625157e−16.
22
5. Section 2.2 ERRORS: DEFINITIONS, SOURCES AND EXAMPLES.
DEFINITION 5.1 (Absolute and Relative Errors). Let xtrue or xT denote the true of a quantity x (with somemeasuring units), and by xapproximate or xA the approximate value (obtained by physical measurements or numericalapproximations). Then we define the following errors in the approximate value xA:
• Error(xA) := xT − xA ≡ xtrue− xapproximate (absolute error)(Note that Error(xA) has the same units as xT and xA.)
• Rel(xA) := Error(xA)xT
= xT−xAxT
(relative error)(Note that Rel(xA) does not have units!)
DEFINITION 5.2 (Number of significant digits).The approximate value xA has t significant digits relative to the true value xA if the number of leading digits in xAare correct relative to the corresponding digits in xT .More precisely: if the (t +1)th digit in the magnitude of the absolute error |xT − xA| is less or equal than 5 (or halfthe base where the calculations are performed), then xA has at least t significant digits of accuracy relative to xT .
PROPOSITION 5.3. It can be shown that if∣∣∣xT − xA
xT
∣∣∣≤ 5 ·10−(t+1),
then xA has t significant digits of accuracy relative to xT .
5.1. Subsection 2.2.1 SOURCES OF ERROR.(E1) Modelling errors “All models are wrong, but some are useful”, an aphorism attributed to George Box.(E2) Blunders and Mistakes: arithmetic and/or programming errors(E3) Physical Measurements errors(E4) Machine representation and Floating-point Arithmetic errors(E5) Mathematical Approximation errors: (local) truncation errors
5.2. Subsection 2.2.2 LOSS-OF-SIGNIFICANCE ERRORS. When two “nearly equal” quantities are subtracted,leading significant digits are lost. For example, in MATLAB floating-point double precision we have that>> 1-(1+1.e-12-1)/(1.e-12)
ans =
-8.890058234101161e-05while, in exact arithmetic, the answer is obviously 0 !!!
EXAMPLE 1. When evaluating the function
f (x) =1− cos(x)
x2
for values of x close to 0, we expect to obtain in the limit x→ 0 that
limx→0
f (x) =12.
Due to floating-point arithmetic, this is what we obtain in double precision23
n=[-1:-1:-10]’;
x=[1.0*10.^n];
xsquare=x.^2;
cosx=cos(x);
numerator=[1-cos(x)];
f=[numerator./x.^2];
T = table(x,xsquare,cosx,numerator,f)
x xsquare cosx numerator f
______ _______ _________________ ____________________ _________________
0.1 0.01 0.995004165278026 0.00499583472197429 0.499583472197429
0.01 0.0001 0.999950000416665 4.99995833347366e-05 0.499995833347366
0.001 1e-06 0.999999500000042 4.99999958325503e-07 0.499999958325503
0.0001 1e-08 0.999999995000000 4.99999996961265e-09 0.499999996961265
1e-05 1e-10 0.999999999950000 5.00000041370185e-11 0.500000041370186
1e-06 1e-12 0.999999999999500 5.00044450291171e-13 0.500044450291171
1e-07 1e-14 0.999999999999995 4.99600361081320e-15 0.499600361081320
1e-08 1e-16 1 0 0
1e-09 1e-18 1 0 0
1e-10 1e-20 1 0 0
Notice that the values of f grow towards 0.5, with the best value obtained for x = 1.0 ∗ 10−5, namely f (10−5) ≈0.500000041370186, while for x smaller than 10−8, f (x)≈ 0!!!If we instead use (2.3) to rewrite f (x) as follows
f (x) =1− cos(x)
x2 =�1−(�1− x2
2! +x4
4! −·· ·+(−1)n x2n
(2n)! +(−1)n+1 x2n+2
(2n+2)! cos(cx))
x2
=12!− x2
4!+ · · ·− (−1)n x2n−2
(2n)!− (−1)n+1 x2n
(2n+2)!cos(cx),
then, in order to have an approximation of machine epsilon order, say 10−16 for any |x|< 0.1, we need to estimatethe remainder
|R2n| :=∣∣∣(−1)n+1 x2n
(2n+2)!cos(cx)
∣∣∣≤ 10−2n
(2n+2)!≤ 10−16
which holds for n≥ 5, (indeed 10−10
12! = 2.087675698786810∗10−19).f=[numerator./x.^2];
f_approx = 1/factorial(2) - x.^2 /factorial(4) + x.^4/factorial(6) - x.^6/factorial(8) + x.^10/factorial(12);
TT = table(x,f,f_approx)
x f f_approx
______ _________________ _________________
0.1 0.499583472197429 0.499583472197421
0.01 0.499995833347366 0.499995833347222
0.001 0.499999958325503 0.499999958333335
0.0001 0.499999996961265 0.499999999583333
1e-05 0.500000041370186 0.499999999995833
24
1e-06 0.500044450291171 0.499999999999958
1e-07 0.499600361081320 0.5
1e-08 0 0.5
1e-09 0 0.5
1e-10 0 0.5
See also Exercise 5.4.
0.999985 0.99999 0.999995 1 1.000005 1.00001 1.000015 1.00002-4
-3
-2
-1
0
1
2
3
410
-15 Noise/Error in function evaluation: -1+x(3+x(-3+x)) (x-1)3
(x-1)3 in nested form: -1+x(3+x(-3+x))
0.9997 0.9998 0.9999 1 1.0001 1.0002 1.0003-1
0
1
2
3
4
5
6
7
8
910
-15 Noise/Error in function evaluation: 1+x(-4+x(6+x(-4+x))) (x-1)4
(x-1)4
FIG. 5.1. Noise in function evaluations: (x−1)3 and (x−1)4, evaluated in their nested forms.
5.3. Subsection 2.2.3 NOISE IN FUNCTION EVALUATION.
5.4. Subsection 2.2.4 UNDERFLOW AND OVERFLOW ERRORS.
Exercise 1. Calculate the error, relative error, and number of significant digits in the following approximationsxA ≈ xT :(a) xT = 28.254,xA = 28.271(b) xT = 0.0028254,xA = 0.028271(c) xT = e,xA = 19
7(d) xT =
√2,xA = 1.414
(e) xT = ln(2),xA = 0.7Solution:(a)
Error(xA) = xT − xA ≡ 28.254−28.271 =−0.0170,
Rel(xA) =xT − xA
xA≡ −0.0170
28.271=−6.0132 ·10−4.
xA = 28.271 has 3 significant digits of accuracy relative to xT = 28.254 since the error in the fourth digit (7versus 5) is 2, less than 5.
(b) xT = 0.0028254,xA = 0.028271(c) xT = e,xA = 19
7
Error(xA) = xT − xA ≡ e− 197
= 0.0040,
Rel(xA) =xT − xA
xA≡ 0.0040
e= 0.0015.
25
xA = 2.714285714285714 has 3 significant digits of accuracy relative to xT = 2.718281828459046 since theerror in the fourth digit (8 versus 4) is 4, less than 5.
(d) xT =√
2,xA = 1.414(e) xT = ln(2),xA = 0.7
Error(xA) = xT − xA ≡ ln(2)−0.7 =−0.0069,
Rel(xA) =xT − xA
xA≡ −0.0069
ln(2)=−0.0099.
xA = 0.70 has 1 significant digit of accuracy relative to xT = 0.693147180559945 since∣∣∣xT − xA
xA
∣∣∣≡ 0.0099 = 0.99 ·10−2 ≤ 5 ·10−1−1,
(see Proposition 5.3).
Exercise 5. In some situations, loss-of-significance errors can be avoided by rearranging the function beingevaluated, as was done with f (x) in (2.23):
f (x) = x√
x+1−√
x1
·√
x+1+√
x√x+1+
√x=
x√x+1+
√x. (2.23)
Do something similar for the following cases, in some cases using trigonometric identities. In all but case (b), assumex is near 0.
(a)1− cos(x)
x2
(b) log(x+1)− log(x), x large(c) sin(a+ x)− sin(a)(d) 3√
x+1−1
(e)√
4+ x−2x
Solution: The idea is to use identities to “transform” the functional in such a way to avoid taking differences ofterms of ‘approximately equal’ size.(a) Here we could use cos(x) = 2sin2(x/2)
1− cos(x)x2 =
1−(1−2sin2(x/2)
)x2 =
2sin2(x/2))x2
or 1 = cos2(x)+ sin2(x)
1− cos(x)x2 =
1− cos(x)x2
1+ cos(x)1+ cos(x)
=1− cos2(x)
x2(1+ cos(x))=
sin2(x)x2(1+ cos(x))
.
See also Example 1:f=[numerator./x.^2];
f_approx = 1/factorial(2) - x.^2 /factorial(4) + x.^4/factorial(6) - x.^6/factorial(8) + x.^10/factorial(12);
f_ex5a = (sin(x)).^2./(x.^2.*(1+cos(x)));
T = table(x,xsquare,cosx,numerator,f)
TT = table(x,f,f_approx,f_ex5a)
26
x f f_approx f_ex5a
______ _________________ _________________ _________________
0.1 0.499583472197429 0.499583472197421 0.499583472197423
0.01 0.499995833347366 0.499995833347222 0.499995833347222
0.001 0.499999958325503 0.499999958333335 0.499999958333335
0.0001 0.499999996961265 0.499999999583333 0.499999999583333
1e-05 0.500000041370186 0.499999999995833 0.499999999995833
1e-06 0.500044450291171 0.499999999999958 0.499999999999958
1e-07 0.499600361081320 0.5 0.5
1e-08 0 0.5 0.5
1e-09 0 0.5 0.5
1e-10 0 0.5 0.5
(b) log(x+1)− log(x), x large(c) sin(a+ x)− sin(a)(d) Use the identity y3−1 = (y−1)(y2 + y+1) with y = 3
√x+1 to obtain
3√x+1−1 = ( 3√x+1−1)(x+1)2/3 +(x+1)1/3 +1(x+1)2/3 +(x+1)1/3 +1
=(x+1)3/3−1
(x+1)2/3 +(x+1)1/3 +1=
x(x+1)2/3 +(x+1)1/3 +1
.
(e) Use (a−b)(a+b) = a2−b2
√4+ x−2
x=
√4+ x−2
x
√4+ x+2√4+ x+2
=4+ x−4
x(√
4+ x+2)=
1√4+ x+2
.
Exercise 6. Use Taylor polynomial approximations to avoid the loss-of-significance errors in the followingformulas when x is near 0.
(a)ex−1
x
(b)1− e−x
x
(c)ex− e−x
2x
(d)log(1− x)+ xe
x2
x3
(e)1− (1− x)
√2−1
x
(f)x− sin(x)
x3
(g)x− sin(x)
tan(x)
(h)x+ log(1− x)
x2
Solution:27
(a) Use ex ≈ 1+ x+ 12! x2 + 1
3! x3 + · · ·+ 1n! xn to obtain
ex−1x≈ �
1+ x+ 12! x2 + 1
3! x3 + · · ·+ 1n! xn−�1
x= 1+
12!
x+13!
x2 + · · ·+ 1n!
xn−1
(b) Similarly use e−x ≈ 1− x+ 12! x2− 1
3! x3 + · · ·+ (−1)n
n! xn
1− e−x
x≈ �
1−(�1− x+ 1
2! x2− 13! x3 + · · ·+ (−1)n
n! xn)
x= 1− 1
2!x+
13!
x2−·· ·+ (−1)n−1
n!xn−1
(c) As above
ex− e−x
2x≈
(�1+ x+ZZ
12! x2 + 1
3! x3 + · · ·+����HHHH
1(2m)! x2m + 1
(2m+1)! x2m+1)−(�1− x+ZZ
12! x2− 1
3! x3 + · · ·+���HHH
12m! x2m + (−1)2m+1
(2m+1)! x2m+1)
2x
= 1+13!
x2 +15!
x4 · · ·+ 1(2m+1)!
x2m
(d) Use formula (1.20) to approximate log(1− x)≈−x− 12 x2− 1
3 x3−·· ·− 1n+1 xn+1 and (1.3) to obtain
log(1− x)+ xex2
x3 ≈(−�x−@@
12 x2− 1
3 x3−·· ·− 1n+1 xn+1
)+ x(�1+HHH(x/2)+ (x/2)2
2! + (x/2)3
3! + · · ·+ (x/2)n
n!
)x3
=
(− 1
3 +1
22·2!
)x3 + · · ·+
(− 1
n+1 +1
2n·n!
)xn+1
x3
=(− 1
3+
122 ·2!
)+ · · ·+
(− 1
n+1+
12n ·n!
)xn−2, n≥ 2.
(e)1− (1− x)
√2−1
x
(f)x− sin(x)
x3
(g)x− sin(x)
tan(x)
(h)x+ log(1− x)
x2
Exercise 13. Find an accurate value of
f (x) =
√1+
1x−1
for large values of x. Calculate
limx→∞
x f (x).
Solution: When x is very large (i.e., x→ ∞), or equivalently when 1x → 0, the two quantities in f (x) are “close” to
28
each other√
1+ 1x ≈ 1 and there is a loss-of-significance error. To avoid taking the difference, we can rewrite f (x)
as follows:
f (x) =
√1+
1x−1 =
√x+1
x−1 =
√x+1−
√x√
x=
√x+1−
√x√
x
√x+1+
√x√
x+1+√
x= �x+1−�x√
x(√
x+1+√
x)
=1
√x(√
x+1+√
x) = 1√
x2 + x+ x
29
6. Section 2.3 PROPAGATION OF ERROR.
6.1. Propagated error in multiplication. Compare xT · yT versus xA · yA.First let us rearrange the relative error definition:
Rel(xA) =xT − xA
xT, xA = xT (1−Rel(xA)), yA = yT (1−Rel(yA)),
and now evaluate the relative error in xA · yA relative to xT · yT :
Rel(xA · yA) :=xT · yT − xA · yA
xT · yT≡ xT · yT − xT (1−Rel(xA)) · yT (1−Rel(yA))
xT · yT
=���xT · yT − xT · yT
(�1−Rel(xA)−Rel(yA)+Rel(xA) ·Rel(yA)
)xT · yT
= Rel(xA)+Rel(yA)+Rel(xA) ·Rel(yA)
≈ Rel(xA)+Rel(yA),
for Rel(xA),Rel(yA)� 1.
6.2. Propagated error in division. Compare xT/yT versus xA/yA.Similarly we evaluate the relative error in xA/yA relative to xT/yT as follows:
Rel(xA · yA) :=xTyT− xA
yAxTyT
≡xTyT− xT (1−Rel(xA))
yT (1−Rel(yA))xTyT
= 1− 1−Rel(xA)
1−Rel(yA)=�1−Rel(yA)− (�1−Rel(xA))
1−Rel(yA)
=Rel(xA)−Rel(yA)
1−Rel(yA)
≈ Rel(xA)−Rel(yA),
for Rel(yA)� 1.
6.3. Propagated error in addition and subtraction. Compare xT ± yT versus xA± yA. We start as above
Rel(xA± yA) :=(xT ± yT )− (xA± yA)
xT ± yT≡ xT ± yT − xT (1−Rel(xA))∓ yT (1−Rel(yA))
xT ± yT
=��xT ±ZZyT −��xT + xT ·Rel(xA))∓ZZyT ± yT Rel(yA))
xT ± yT
=xT ·Rel(xA))± yT Rel(yA))
xT ± yT,
which by taking absolute values yields
|Rel(xA± yA)|=∣∣∣xT ·Rel(xA))± yT Rel(yA))
xT ± yT
∣∣∣≤ |xT | · |Rel(xA)|+ |yT | · |Rel(yA)||xT ± yT |
≤ (|Rel(xA)|+ |Rel(yA)|)|xT |+ |yT ||xT±yT |
.
30
If xT ,yT are “close” in magnitude and opposite sign, |xT+yT | ≡ 0 and the term in the right-hand side of the estimateabove could be very large!!!
|Rel(xA± yA)| ≤ (|Rel(xA)|+ |Rel(yA)|) ·O(∞) !?!
We say that the “subtraction” operation is ill-conditioned: small errors in two real numbers of same signs yields(possibly) large errors when subtracting them.(This is related to the loss-of-significance errors.)
EXAMPLE 2. Let ε = eps= 2.220446049250313e−16 be the machine epsilon in the double-precision floatingpoint arithmetic, and consider the following numbers. The true and approximate values are:
xT = 1+500ε, xA = 1−500ε,
yT = 1, yA = 1.
Therefore the relative errors are respectively
Rel(xA) =1000ε
1+500∗ ε= 2.220446049250066e−13,
Rel(yA) = 0.
Nevertheless, the relative error in the difference xA− yA is “quite large”:
Rel(xA− yA) =(xT − yT )− (xA− yA)
xT − yT≡ 500ε− (−500∗ ε)
500ε= 2 6≈ Rel(xA)−Rel(yA) !!!
6.4. Propagated error in function evaluation.Let f ∈ C1[a,b], and xT ,xA ∈ [a,b], assumed to be ‘close’ to each other. Then by the Taylor approximation, theabsolute error in f (xA) can be approximated as
Error ( f (xA)) := f (xT )− f (xA) = f ′(c) · (xT − xA), with c between xT and xA, hence≈ f ′(xT ) · (xT − xA),
and therefore the relative error in f (xA) is
Rel( f (xA))≈f ′(xT )
f (xT )· (xT − xA)≡
f ′(xT )
f (xT )xT ·
xT − xA
xT=
f ′(xT )
f (xT )xT ·Rel(xT ).
We note that the conditioning number: f ′(xT )f (xT )
xT could be very large.For Example, if f (x) = 2x and
xT = 1000.xA = 1000.0001Rel(xA) =−9.999999997489795e−08;f (xT ) = 1.071508607186267e+301f (xA) = 1.071582881077358e+301Rel( f (xA)) =−6.931231582904155e−05;
f ′(xT )
f (xT )xT = 6.931471805599452e+02.
31
7. Section 2.4 SUMMATION.Recall that due to floating-point approximation, the addition operation in “computer arithmetic” is not associative.
EXAMPLE 3. Let n ∈ N be an integer, and consider the finite sum:
S = 0.1+0.2+ · · ·+0.1 ·n≡ 0.1∗ (1+2+ · · ·n) = n(n+1)2 ·10
.
(Recall also Exercise 5 in Section 2.1 (4.1), about 0.1 not having a finite representation in base 2.)%Error is summation
n=1.e+11;
Sexact = n*(n+1)/20
Sinc = 0; Sdec = 0;
for i=1:n
Sinc = Sinc + i/10;
end
for j=n:-1:1
Sdec = Sdec + j/10;
end
Sinc
Sdec
Sexact =
5.000000000050000e+20
Sinc =
5.000000000031216e+20
Sdec =
5.000000000082726e+20
32
8. Section 4.1 POLYNOMIAL INTERPOLATION.
The aim of polynomial interpolation is to have a compact representation of available (large set of) data.It could also be used to obtain an approximation of a function which is known only by a finite number of values, at agiven number of points.For example, given a set of points {(xi, f (xi))}N
i=1, we would want to approximate the given function f (·) by apolynomial (global or piecewise), which can be easier to manipulate (via integration, derivation, etcetera).
More general, we can ask the following question:
Given (n+1) pairs of points {(xi,yi) | i = 0,1, · · · ,n}
{(xi,yi)}ni=0 = {(x0,y0),(x1,y1),(x2,y2), · · · ,(xn,yn)}
find a function Φ(·) such that
Φ(xi) = yi, ∀i = 0,1, · · · ,n
saying that Φ(·) interpolates {yi}ni=0 at the nodes {xi}n
i=0.
We note that in the above, the function Φ could be a global polynomial
given (n+1) pairs (xi,yi) find Πn ∈ Pn, an interpolating polynomial:Πn(xi) = yi, i = 0,1, . . . ,n (polynomial interpolation)
or a piecewise polynomial (spline functions).
THEOREM 8.1. Given (n+1) distinct nodes {xi}ni=0:
x0,x1, · · · ,xn (nodes)
and (n+1) corresponding values {yi}ni=0:
y0,y1, · · · ,yn (values)
there exists a unique global (interpolating) polynomial Πn ∈ Pn such that :
Πn(xi) = yi, ∀i = 0,1, · · · ,n.
Proof. Existence: constructive. Let us define the following polynomials
`i(x) =n
∏j=0j 6=i
x− x j
xi− x j≡
(x − x0) · (x − x1) · · · (x − x j−1) · (x − x j+1) · · · (x − xn)
(xi− x0) · (xi− x1) · · ·(xi− x j−1) · (xi− x j+1) · · ·(xi− xn).
(Lagrange characteristic polynomial)
We note 1 that the Lagrange characteristic polynomials {`i(·)}i=0:n form a basis of Pn,
`i(x j) = δi j =
{1 if i = j0 if i 6= j
1{`i}i=0:n are also orthogonal, with the inner product 〈 f ,g〉=∫
∞
−∞e−x2
f (x)g(x)dx.
33
and therefore the interpolating polynomial is
Πn(x) =n
∑i=0
yi`i(x)≡ y0`0(x)+ y1`1(x)+ · · ·+ yn`n(x). (interpolating polynomial in Lagrange form)
Uniqueness: by contradiction.
DEFINITION 8.2 (NEWTON DIVIDED DIFFERENCES). Let x0,x1,x2, · · · ,xn be n+1 distinct nodes
f [x0,x1] =f (x1)− f (x0)
x1− x0(1st-order divided difference)
f [x0,x1,x2] =f [x1,x2]− f [x0,x1]
x2− x0(2nd-order divided difference)
f [x0,x1,x2,x3] =f [x1,x2,x3]− f [x0,x1,x2]
x3− x0(3rd-order divided difference)
f [x0, · · · ,xn] =f [x1, · · · ,xn]− f [x0, · · · ,xn−1]
xn− x0(nth-order Newton divided difference)
THEOREM 8.3. Let f ∈Cn(I) where x0,x1, · · · ,xn ∈ I are n+1 distinct nodes. Then there exists ∃c ∈ I such that
f [x0,x1, · · · ,xn] =1n!
f (n)(c). (8.1)
Proof. This is a consequence of the Langrange Mean Value Theorem.PROPOSITION 8.4.
f [x0,x1, · · · ,xn] = f [xi0 ,xi1 , · · · ,xin ], ∀ permutations (i0, i1, · · · , in).
Also, in case of “repeating nodes”:
f [x0,x0] = limx1→x0
f [x0,x1] = limx1→x0
f (x1)− f (x0)
x1− x0= f ′(x0), (8.2)
f [x0,x0,x1] =f [x0,x1]− f [x0,x0]
x1− x0
(8.2)==
f [x0,x1]− f ′(x0)
x1− x0, (8.3)
f [x0,x0, · · · ,x0︸ ︷︷ ︸(n+1) terms
] =1n!
f (n)(x0). (8.4)
THEOREM 8.5.
Πn(x) =n
∑i=0
ωi(x) f [x0, · · · ,xi]
≡ f (x0)+ω1(x) f [x0,x1]+ · · ·+ωn(x) f [x0,x1, · · · ,xn]
≡ f (x0)+(x− x0) f [x0,x1]+ · · ·+(x− x0) · · ·(x− xn−1) f [x0,x1, · · · ,xn](interpolating polynomial in Newton form)
COROLLARY 8.6. Note that in the ”repeated nodes case”: x0 = x1 = · · · = xn, using property (8.4), the(interpolating polynomial in Newton form) gives
Πn(x) = f (x0)+(x− x0) f ′(x0)+(x− x0)2 f ′′(x0)
2!+ · · ·+(x− x0)
n f (n)(x0)
n!(8.5)
34
which is exactly the (Taylor approximation polynomial).
Exercise 1. Given the data points (0,2),(1,1), find the following:(a) The straight line interpolating this data.(b) The function f (x) = a+bex interpolating this data.
(Hint: Find a and b so that f (0) = 2, f (1) = 1.)(c) The function g(x) = a
b+x interpolating this data.In each instance, graph the interpolating function.Solution:(a) The divided differences are
xi yi [yi,y j]0 21 1 1−2
1−0 =−1
hence
Π1(x) = 2− x.
(b) Since f (0) = a+b≡ 2, f (1) = a+be≡ 1, we have that a = 1−2e1−e ,b = 1
1−e , so
f (x) =1−2e1− e
+1
1− eex.
(c) Since g(0) = ab ≡ 2,g(1) = a
b+x ≡ 1, we have that a = 2,b = 1, so
g(x) =2
1+ x.
x = linspace(0,1,100);
x_nodes = [0 1];
y_values = [2 1];
P1 = 2 - x;
f = (1-2*exp(1) + exp(x))/(1-exp(1));
g = 2./(1+x);
plot(x,P1,’b.-.’,x,f,’r.-.’,x,g,’k.-.’,x_nodes,y_values,’g*’)
legend(’P1’,’f’,’g’,’data points’)
Exercise 2.(a) Find the function P(x) = a+bcos(πx)+ csin(πx), which interpolates the data
xi yi0 20.5 51 4
(b) Find the quadratic polynomial interpolating this data.In each instance, graph the interpolating function.Solution:(a) Since P(0) = a+b≡ 2,P(0.5) = a+ c≡ 5,P(1) = a−b≡ 4, we have that a = 3,b =−1,c = 2.
35
xi yi [yi,y j] [y0,y1,y2]0 20.5 5 5−2
0.5 = 61 4 4−5
1−0.5 =−2 −2−61 =−8
(b) The divided differences are
hence
P2(x) = 2+6x−8x(x−0.5) =−8x2 +10x+2.
x = linspace(0,1,100);
x_nodes = [0 0.5 1];
y_values = [2 5 4];
P = 3 - cos(pi*x) + 2*sin(pi*x);
P2 = -8*x.^2 + 10*x + 2;
plot(x,P,’b.-.’,x,P2,’r.-.’,x_nodes,y_values,’g*’)
legend(’P’,’P2’,’data points’)
Exercise 12.(a) For n = 3, explain why
`0(x)+ `1(x)+ `2(x)+ e�3 (x) = 1 ∀x.
(Hint: It is unnecessary to actually multiply out and combine (Lagrange characteristic polynomial) `i(x). Use(interpolating polynomial in Lagrange form) and a suitable choice of {y0,y1,y2,y3}.)
(b) Generalize part (a) to an arbitrary degree n > 0.Solution:
Exercise 16. As a generalized interpolation problem, find the quadratic polynomial q(x) with deg q(x) ≤ 2which
q(0) =−1, q(1) =−1, q′(1) = 4.
Solution: Using (8.2) to substitute f [1,1] = q′(1), the Newton divided differences are and therefore the (interpolating polynomial in Newton form)
xi f (xi) f [xi,x j] f [x1,x2,x3]0 -11 -1 01 -1 4 4
writes
q(x) = q(0)+(x−0)q[0,1]+ (x−0)(x−1)q[0,1,1]
=−1+0 · x+4x(x−1) = 4x2−4x−1.
Exercise 24. Given the data below,
36
i xi f (xi)1 0.1 0.202 0.2 0.243 0.3 0.30
find f [x0,x1] and f [x0,x1,x2].Then calculate Π1(0.15) and Π2(0.15), the linear and quadratic interpolates evaluated at x = 0.15.Solution: The Newton divided differences are
i xi f (xi) f [xi,x j] f [x1,x2,x3]1 0.1 0.202 0.2 0.24 0.24−0.20
0.2−0.1 = 0.43 0.3 0.30 0.30−0.24
0.3−0.2 = 0.6 0.6−0.40.3−0.1 = 1
and therefore the interpolating polynomials (in Newton form) of degree one and two are
Π1(x) = 0.2+0.4(x−0.1),Π2(x) = 0.2+0.4(x−0.1)+1(x−0.1)(x−0.2).
while the values are
Π1(0.15) = 0.22,Π2(0.15) = 0.2175.
Exercise 25’. Let
f (x) =1
1+ x2 (Runge example)
and let x0 = 1,x1 = 2,x2 = 3. Calculate the divided differences f [x0,x1] and f [x0,x1,x2]. Using these divideddifferences, give the quadratic polynomial P2(x) that interpolates f (x) at the given node points {x0,x1,x2}. Graphthe error on the interval [1,3].Solution: The divided differences are hence the polynomial is
xi f (xi) f [xi,x j] f [x1,x2,x3]1 1/22 1/5 1/5−1/2
2−1 =− 310
3 1/10 1/10−1/53−2 =− 1
10−1/10+3/10
3−1 = 110
P2(x) =12+− 3
10(x−1)+
110
(x−1)(x−2).
x = linspace(1,3,100) ;
P2 = 1/2 - 3/10 * (x-1) + 1/10 * (x-1) .* (x-2) ;
f = 1 ./ (1 + x .^2) ;
plot( x , f-P2 , ’r.-.’ )
Exercise 26.37
(a) By using the program divdiff, calculate the divided differences D0 = f (x0), D1 = f [x0,x1], · · · ,D5 = f [x0,x1,x2,x3,x4,x5], for f (x) = ex.Use x0 = 0,x1 = 0.2,x2 = 0.4, . . . ,x5 = 1.0.
(b) Using the results of (a), calculate Pj(x) for x = 0.1,0.3,0.5 and j = 1, · · · ,5. Compare these results to the truevalues of ex.
Solution:% Exercise 4.1.26
x_nodes = linspace(0.0,1.0,6);
y_values = exp(x_nodes);
x_eval = linspace(0.1,0.5,3);
divdif_y = divdif(x_nodes,y_values);
fprintf ( 1, ’ j %d %d %d %d %d %d \n’,1:length(x_nodes));
fprintf ( 1, ’ Djf %1.4f %1.4f %1.4f %1.4f %1.4f %1.4f \n\n’,divdif_y);
p_eval = interp(x_nodes,divdif_y,x_eval);
fprintf ( 1, ’ j P_j(%g) P_j(%g) P_j(%g)\n’,x_eval);
for i = 1:length(x_nodes)-1
p_eval = interp(x_nodes(1:i+1),divdif_y(1:i+1),x_eval);
fprintf ( 1, ’ %d %1.7f %1.7f %1.7f \n’, i, p_eval);
end
fprintf ( 1, ’True %1.7f %1.7f %1.7f\n’,exp(x_eval));
>> Exercise_4_1_26
j 1 2 3 4 5 6
Djf 1.0000 1.1070 0.6127 0.2261 0.0626 0.0139
j P_j(0.1) P_j(0.3) P_j(0.5)
1 1.1107014 1.3321041 1.5535069
2 1.1045740 1.3504863 1.6454179
3 1.1052523 1.3498080 1.6488094
4 1.1051584 1.3498643 1.6487156
5 1.1051730 1.3498581 1.6487218
True 1.1051709 1.3498588 1.6487213
Exercise 27. Repeat Problem 26 with f (x) = arctan(x).Solution:% Exercise 4.1.27
x_nodes = linspace(0.0,1.0,6);
y_values = atan(x_nodes);
x_eval = linspace(0.1,0.5,3);
divdif_y = divdif(x_nodes,y_values);
fprintf ( 1, ’ j %d %d %d %d %d %d \n’,1:length(x_nodes));
38
fprintf ( 1, ’ Djf %1.4f %1.4f %1.4f %1.4f %1.4f %1.4f \n\n’,divdif_y);
p_eval = interp(x_nodes,divdif_y,x_eval);
fprintf ( 1, ’ j P_j(%g) P_j(%g) P_j(%g)\n’,x_eval);
for i = 1:length(x_nodes)-1
p_eval = interp(x_nodes(1:i+1),divdif_y(1:i+1),x_eval);
fprintf ( 1, ’ %d %1.7f %1.7f %1.7f \n’, i, p_eval);
end
fprintf ( 1, ’True %1.7f %1.7f %1.7f\n’,atan(x_eval));
>> Exercise_4_1_27
j 1 2 3 4 5 6
Djf 0.0000 0.9870 -0.1786 -0.1857 0.1698 -0.0572
j P_j(0.1) P_j(0.3) P_j(0.5)
1 0.0986978 0.2960933 0.4934889
2 0.1004834 0.2907366 0.4667050
3 0.0999263 0.2912936 0.4639197
4 0.0996717 0.2914464 0.4636651
5 0.0996116 0.2914722 0.4636393
True 0.0996687 0.2914568 0.4636476
Exercise 28. The following data are taken from a polynomial p(x) of degree ≤ 5. What is the polynomial and whatis its degree?
xi f (xi)-2 -5-1 10 11 12 73 25
Solution: The Newton divided differences are and therefore the interpolating polynomial (in Newton form), of degree
xi f (xi) f [xi,x j] f [xi,x j,xk] f [· · · ] f [· · · ] f [· · · ]-2 -5-1 1 1−(−5)
−1−(−2) = 60 1 1−1
0−(−1) = 0 0−60−(−2) =−3
1 1 0 0 0−(−3)1−(−2) = 1
2 7 6 6−02−0 = 3 3−0
2−(−1) = 1 03 25 18 18−6
3−1 = 6 6−33−0 = 1 0 0
three, is:
Π3(x) =−5+6(x+2)−3(x+2)(x+1)+(x+2)(x+1)x.
39
9. Section 4.2 ERROR IN POLYNOMIAL INTERPOLATION.We are evaluating the error made when ”replacing” a function f by its interpolating polynomial Πn f , obtained byinterpolation on the nodes x0,x0, · · · ,xn and node values f (x0), f (x1), · · · , f (xn).
DEFINITION 9.1. Given a n+1 nodes x0,x1,xn, the nodal polynomial is the polynomial of degree n+1, definedas the product of all monomials (x− xi), namely:
ωn+1(x) := (x− x0)(x− x1) · · ·(x− xn)≡n
∏i=0
(x− xi) ∈ Pn+1 (nodal polynomial)
REMARK 7. We note that
ω′n+1(x) =
n
∑j=0
n
∏i=0i 6= j
(x− xi), ω′n+1(xκ) =
n
∏i=0i6=κ
(xκ − xi),
and therefore the (Lagrange characteristic polynomial), respectively the (interpolating polynomial in Lagrange form)write, respectively:
`i(x) =n
∑i=0
ωn+1(x)(x− xi)ωn+1(xi)
,
(Πn f )(x) =n
∑i=0
ωn+1(x)(x− xi)ωn+1(xi)
f (xi).
THEOREM 9.2. Let x0,x1, · · · ,xn be distinct n+1 nodes, and an arbitrary value x in the domain of definition offunction f . Let now denote by Ix the smallest interval containing the nodes x0,x1, · · · ,xn and also x.Assuming that f ∈Cn+1(Ix), then there exists an arbitrary cx ∈ Ix such that the interpolation error at x ∈ Ix is givenby
En(x) := f (x)− (Πn f )(x) =f (n+1)(cx)
(n+1)!ωn+1(x) (interpolation error)
= f [x0,x1, · · · ,xn,x]ωn+1(x), (interpolation error / Newton form)
where ωn+1 is the (nodal polynomial) of degree n+1.
REMARK 8 (Drawbacks of global interpolation on equally spaced nodes.The Runge phenomenon).Notice that the error is influenced by the behavior of the nodal polynomial ωn+1, which can highly fluctuate onequally spaced nodes, for n large. See the Runge counterexample and interpolation runge.m.
(See the runge phenomenon.m, runge example.m and interpolation runge.m.)40
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
FIG. 9.1. Global interpolation polynomials on equally spaced nodes, x nodes = [-3 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3],y values = [0 0 0 0 0 0 1 0 0 0 0 0 0].
-6 -4 -2 0 2 4 6
0
0.5
1
1.5
2
Runge counterexample on equally spaced nodes
Runge function 1./(1+x. 2 )
interpolation polynomial
nodes
-6 -4 -2 0 2 4 6
0
0.5
1
1.5
2
Runge counterexample on equally spaced nodes
Runge function 1./(1+x. 2 )
interpolation polynomial
nodes
-6 -4 -2 0 2 4 6
0
1
2
3
4
5
106 Runge counterexample on equally spaced nodes
Runge function 1./(1+x. 2 )
interpolation polynomial
nodes
FIG. 9.2. The Runge function f (x) = 11+x2 interpolated on 10, 15 and 50 equally spaced nodes.
Exercise 2. Consider interpolating f (x) = arctan(x) from a table of values of the function f given at equallyspaced values of x for 0≤ x≤ 0.8; the x entries are given in steps of h = 0.01.(a) Bound the error f (x)−P1(x) of linear interpolation in this table. The value of x is to satisfy x0 < x < x1, with
x0 and x1 adjacent x entries in the table.(b) Bound the error f (x)−P2(x) of quadratic interpolation in this table. The value of x is to satisfy x0 < x < x2,
with x0,x1 and x2 adjacent x entries in the table.Solution:
f (x) = arctan(x), f ′(x) =1
1+ x2 ≡ (1+ x2)−1,
f ′′(x) =−2x(1+ x2)−2 ≡− 2x(1+ x2)2 , f ′′′(x) = (6x2−2)(1+ x2)−3 ≡ 6x2−2
(1+ x2)3 ,
f (4)(x) =24x−24x3
(1+ x2)−4 .
Using the (interpolation error) formula we then have:41
-5 -4 -3 -2 -1 0 1 2 3 4 5-400
-300
-200
-100
0
100
200
300
400Nodal polynomial on 5 equally spaced nodes in [-5,5]
nodes
nodal polynomial
-5 -4 -3 -2 -1 0 1 2 3 4 5-5
-4
-3
-2
-1
0
1
2
3
4
510
5 Nodal polynomial on 11 equally spaced nodes in [-5,5]
nodes
nodal polynomial
-5 -4 -3 -2 -1 0 1 2 3 4 5-1.5
-1
-0.5
0
0.5
1
1.510
11 Nodal polynomial on 21 equally spaced nodes in [-5,5]
nodes
nodal polynomial
FIG. 9.3. Three nodal polynomials in [−5,5] using 5, 11 and 21 equally spaced nodes. Notice the magnitude on the y-axis!
(a) for n = 1
E1(x) := f (x)− (Π1 f )(x) =f′′(cx)
2!ω2(x)≡−
12!
2cx
(1+ c2x)
2 (x− x0)(x− x1),
where x,cx ∈ [x0,x1]. Since
maxx∈[x0,x1]
(x− x0)(x− x1) =(x1− x0)
2
4, (9.1)
maxy∈[x0,x1]
f ′′(y) = f ′′(√3
3)=
3√
38
,
and h≡ x1− x0, we have that
|E1(·)| ≤12!
3√
38
h2
4≈ 8.1190×10−6. (*)
(b) Similarly, for n = 2
E2(x) := f (x)− (Π2 f )(x) =f′′′(cx)
3!ω3(x)≡
13!
6c2x−2
(1+ c2x)
3 (x− x0)(x− x1)(x− x2)
where x,cx ∈ [x0,x2]. Since
maxx∈[x0,x2]
∣∣(x− x0)(x− x1)(x− x2)∣∣= 2
√3
9h3, (9.2)
maxy∈[x0,x2]
| f ′′′(y)|= | f ′′′(0)|= 2,
where h = x1− x0 ≡ x2− x1. Then we have
|E2(·)| ≤13!·2 · 2
√3
9h3 ≈ 1.2830×10−7. (**)
Exercise 6. Suppose a table of values of f (x) = arctan(x),x ∈ [0,1] is to be constructed, with values of arctan(x)given with a spacing of h.
42
1. If linear interpolation is used in this table, how small should h be in order for the interpolation error to beless than 5×10−6? For notational assumptions, see Exercise 2 (a).
2. If quadratic interpolation is used in this table, how small should h be in order for the interpolation error tobe less than 5×10−6? For notational assumptions, see Exercise 2 (b).
Solution:1. From (*) in Exercise 2 (a) we have that
|E1(x)| ≤3√
364
h2
hence in order for the error to be less than 5×10−6 it suffices to have
h≤
√5×10−6 64
3√
3≈ 0.0078.
2. Similarly, from (**) in Exercise 2 (b)
|E2(x)| ≤2√
327
h3
and
h≤ 3
√5×10−6 9
√3
2≈ 0.0339.
Exercise 7. Consider constructing a table of values of f (x) =√
x for x ∈ [1,100], with values of f (x) given forx = 0,h,2h, · · · . Choose h so that when the linear interpolation is used in this table, the error is bounded by 5×10−6.Discuss and compare the linear interpolation error near x = 1 and x = 100.Solution: Since f ′(x) = 0.5x−1/2, f ′′(x) = − 1
4 x−32 , on any interval [xi,xi+1], the interpolation error is bounded by
(see (interpolation error)) we have that
|E1(x)|=∣∣ f ′′(x)
2!(x− xi)(x− xi+1)
∣∣≤ 12!
14
maxx∈[1,100]
|x−32 |︸ ︷︷ ︸
=1
h2
4=
132
h2, ∀x ∈ [1,100].
The error is less than 5×10−6 provided h≤ 0.0126.When x≈ 1,
|E1(x)| ≈12!
14
h2
4=
132
h2 ≈ 0.0312×h2,
while when x≈ 100
|E1(x)| ≤12!
14
10−3 h2
4≈ 0.00003125×h2.
Exercise 8. Let f (x) = x4 +√
2x3 +πx. Verify whether f [0,1,2,3,4] = f [0,1,π,e,−1].
Solution: Since
f (4)(x) = 4! ∀x,
43
using (interpolation error / Newton form) we conclude that indeed f [0,1,2,3,4] = f [0,1,π,e,−1] = 1.Exercise 9. Let f (x) = a0+a1x+ · · ·+anxn be a polynomial of degree less than or equal to n, and let {x0,x1, · · · ,xn}
be distinct points. What is the value of f [x0,x1, · · · ,xn]?Solution: Similar to Exercise 8, since f (n)(x) = n!an, from (interpolation error / Newton form) we have that
f [x0,x1, · · · ,xn] =f (n)(cx)
n!= an.
44
10. Piecewise Interpolations.
REMARK 9.(i) As seen in Figure 9.2, on the very smooth Runge counter-example function f (x) = 1
1+x2 , global interpolationon equally spaced nodes does not guarantee uniform convergence Πn f → f as n→ ∞.(See also Figure 9.3 for the behavior of the nodal polynomial.)
(ii) On the other hand, equally spaced nodes are computationally convenient.(iii) (Lagrange) interpolation of low degree κ is sufficiently accurate, provided sufficiently small inter-
vals are considered. Indeed, by (interpolation error) we have that
|Eκ(x)| ≤| f (κ+1)(cx)|(κ +1)!
|ωκ+1(x)|︸ ︷︷ ︸≤Chκ+1
. for f ∈Cκ+1[a,b], (10.1)
on any consecutive h-equally spaced nodes.(For example, see (9.1) for κ = 1: |ω2(x)| ≤ 1
4 h2 and (9.2) for κ = 2: |ω3(x)| ≤ 2√
39 h3.)
10.1. Section 4.3 INTERPOLATION USING SPLINE FUNCTIONS.The goal is to approximate a given function with splines, allowing a piecewise interpolation with global smoothness.
DEFINITION 10.1. Let x0,x1, · · · ,xn be (n+1) distinct nodes in [a,b], a = x0 < x1 < · · ·< xn = b.The function sk(x) on [a,b] is a spline of degree k relative to the nodes x j if(1) sk is piecewise polynomial:
sk∣∣[x j ,x j+1]
∈ Pk, j = 0,1, · · · ,n−1 (10.2)
(2) global smoothness/continuity at nodes:
sk ∈Ck−1[a,b] (10.3)
We denote Sk the space of splines sk on [a,b] relative to n+1 distinct nodes.
dim(Sk) = n+ k.
PROPOSITION 10.2. For a function f ∈Cκ+1([a,b]) we have the following uniform convergence resullt
‖ f − sκ‖∞ ≤Chκ+1‖ f (κ+1)‖∞.
Proof. Use (10.1).PROPOSITION 10.3. For a function square integrable satisfying f (m) ∈ L2(a,b) for all m = 0, · · ·κ + 1, there
exists a positive constant C > 0, independent of h such that
‖( f − sκ)(m)‖L2(a,b) ≤Chκ+1−m‖ f (κ+1)‖L2(a,b).
In particular, for κ = 1 (piecewise linear interpolation) and m = 0 or m = 1 we have that
‖ f − s1‖L2(a,b) ≤C1h2‖ f ′′‖L2(a,b), ‖( f − s1)′‖L2(a,b) ≤C2h‖ f ′′‖L2(a,b).
For κ = 3 (piecewise cubic interpolation) and m = 0,1,2,3 we have that
‖ f − s3‖L2(a,b) ≤C1h4‖ f (4)‖L2(a,b), ‖( f − s1)′‖L2(a,b) ≤C2h3‖ f 4‖L2(a,b),
‖( f − s3)′′‖L2(a,b) ≤C3h2‖ f (4)‖L2(a,b), ‖( f − s1)
′′′‖L2(a,b) ≤C4h‖ f 4‖L2(a,b).
45
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Piecewise (linear and cubic splines) interpolation of the "Runge" example
data
exact
piecewise linear
not-a-knot cubic spline
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Piecewise (linear and cubic splines) interpolation of the "Runge" example
data
exact
piecewise linear
not-a-knot cubic spline
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Piecewise (linear and cubic splines) interpolation of the "Runge" example
data
exact
piecewise linear
not-a-knot cubic spline
FIG. 10.1. The Runge function f (x) = 11+x2 interpolated with piecewise linear and not-a-knot cubic splines, on 6, 8 and 10 equally spaced
nodes. (Compare with Figure 9.2 - for global interpolation on equally spaced nodes.)
Proof. Use (10.1), Rolle’s theorem and the Cauchy-Bunyakovsky-Schwarz inequality.Hence a small interpolation error can be obtained even for low κ , provided h is sufficiently “small”.REMARK 10. We remark that relations (10.2)-(10.3) do not suffice to fully characterize a spline of degree k.
Indeed, by condition (10.2) we see that (k+1)n coefficients si j must be determined (k+1 coefficients, n inter-vals):
sk, j(x) := sk
∣∣∣[x j ,x j+1]
(x) =k
∑i=0
si, j(x− x j)i, if x ∈ [x j,x j+1], j = 0, · · · ,n−1.
On the other hand, the continuity at the (interior) nodes condition (10.3) only provides k(n−1) conditions:
s(m)k, j−1(x j) = s(m)
k, j (x j), j = 1, · · · ,n−1, m = 0,1, · · · ,k−1.
Therefore the definition leaves
(k+1)n︸ ︷︷ ︸unknowns
−k(n−1)︸ ︷︷ ︸conditions
= n+ k
undetermined coefficients.Assuming that the splines were interpolatory, i.e.,
sk(x j) = f j, j = 0, · · · ,n
where f0, · · · , fn are given values, we would still have
n+ k− (n+1) = k−1
unsaturated degrees of freedom.Further constraints are imposed, leading to
(i) periodic splines
s(m)k (a) = s(m)
k (b), m = 1,2, · · · ,k−1
(ii) natural splines: for k = 2`−1, with `≥ 2
s(`+ j)k (a) = s(`+ j)
k (b) = 0, j = 0,1, · · · , `−246
10.2. Cubic spline interpolation: k = 3.The cubic splines are particularly significant since they are
• splines of minimum degree which yield C2 approximation, and are• sufficiently smooth in presence of small curvatures (see Proposition 10.5 for the minimum norm property
of the curvature).In this case there are only k−1≡ 2 unsaturated degrees of freedom. So for the closure we have either
• natural cubic splines: s′′3(a) = s′′3(b) = 0,• not-a-knot spline with
– active nodes x0,x2, · · · ,xn−2,xn(hence n−2 intervals, 4n−8 unknowns; 3(n−3) smoothness on interior nodes PLUS n+1 interpo-lation conditions; giving 4n−8 equations) and
– x1,xn−1 as only interpolating nodesREMARK 11. MATLAB’s function spline uses not-a-knot cubic splines.
spline(x_nodes,y_values,x_evaluatepoint)
DEFINITION 10.4 (Natural cubic splines). There is a unique function s(x) such that(S1) s3(x) ∈ P3 on each subinterval [x j−1,x j], j = 1, · · · ,n;(S2) s3(x),s′3(x),s
′′3(x) are continuous on [a,b];
(S3) s′′3(a) = s′′3(b) = 0,which also interpolates the data {xi,yi}i=0,··· ,n.
10.3. Construction of natural cubic splines.Let us denote Mi := s′′3(xi), i = 0, · · · ,n, and according to (S3) set
M0 = Mn = 0.
Then, the cubic spline has the form
s3(x) =(x j− x)3M j−1 +(x− x j−1)
3M j
6(x j− x j−1)+
(x j− x)y j−1 +(x− x j−1)y j
x j− x j−1(4.63)
− 16(x j− x j−1)[(x j− x)M j−1 +(x− x j−1)M j], x ∈ [x j−1,x j],
where M1, · · · ,Mn−1 are solutions of the linear system
x j− x j−1
6M j−1 +
x j+1− x j−1
3M j +
x j+1− x j
6M j+1 =
y j+1− y j
x j+1− x j−
y j− y j−1
x j− x j−1, j = 1,2, · · · ,n−1. (4.64)
PROPOSITION 10.5 (Minimum norm property).Let f ∈C2[a,b] and s3 be the natural cubic spline interpolating f . Then∫ b
a
(s′′3(x)
)2dx≤∫ b
a
(f ′′(x)
)2dx (10.4)
where equality holds if and only if f = s3.REMARK 12. The property (10.4) holds even for splines that satisfy s′3(a) = f ′(a), s′3(b) = f ′(b).PROPOSITION 10.6. Let f ∈ C4[a,b] and [a,b] = ∪N−1
i=0 Ii be a partition of width hi, with h = maxhi , and β =h/maxi hi. Then the s3 cubic spline interpolating f satisfies
‖ f (r)− s(r)3 ‖∞ ≤Crh4−r‖ f (4)‖∞, r = 0,1,2,3
with C0 =5
384 , C1 =124 , C2 =
38 and C3 = (β +β−1)/2.
Exercise 2. Consider the data47
x 1 2 3 4 5y 3 1 2 3 2
(a) Find the piecewise linear interpolating functions `(x).(b) Find the cubic spline function s(x) that interpolates the data and satisfies the not-a-knot boundary conditions
s(z1) = f (z1), s(z2) = f (z2). (4.73)
Note in this case that n = 3,x1 = 1,x2 = 3,x3 = 5,z1 = 2,z2 = 4.Graph both s(x) and `(x) for x ∈ [0,5].Solution:(a) The Newton divided differences are hence the piecewise linear interpolating polynomial is:
x y [yi,y j]1 32 1 1−3
2−1 =−23 2 2−1
3−2 = 14 3 3−2
4−3 = 15 2 2−3
5−4 =−1
`(x) =
3−2(x−1), x ∈ [1,2]1+(x−2), x ∈ [2,3]2+(x−3), x ∈ [3,4]3− (x−4), x ∈ [4,5]
(b) Use (4.64) to find the cubic spline by hand.
x j− x j−1
6M j−1 +
x j+1− x j−1
3M j +
x j+1− x j
6M j+1 =
y j+1− y j
x j+1− x j−
y j− y j−1
x j− x j−1, j = 2,3, · · · ,n−1. (4.64’)
Then taking n = 3, hence j = 2 we get
x2− x1
6M1 +
x3− x1
3M2 +
x3− x2
6M3 =
y3− y2
x3− x2− y2− y1
x2− x1,
and using the not-a-knot conditions
s(2) = 1, s(4) = 3
in formula (4.63)
s(x) =(x j− x)3M j−1 +(x− x j−1)
3M j
6(x j− x j−1)+
(x j− x)y j−1 +(x− x j−1)y j
x j− x j−1
− 16(x j− x j−1)
[(x j− x)M j−1 +(x− x j−1)M j], x ∈ [x j−1,x j]
with j = 2,x1 = 1,x2 = 3,y1 = 3,y2 = 2 and x = z1 = 2 gives
M1 +M2 = 6,
respectively, with j = 3,x2 = 3,x3 = 5,y2 = 2,y3 = 2 and x = z2 = 4 gives
M2 +M3 =−4.48
1 1.5 2 2.5 3 3.5 4 4.5 51
1.5
2
2.5
3
3.5
piecewise linear
cubic spline
data points
FIG. 10.2. The piecewise linear and natural cubic spline for Exercise 2.
Therefore
M1 =254, M2 =−
14, M3 =−
154.
You could also use MATLAB’s built-in functions for the piecewise linear interpolant and the not-a-knotcubic spline:x_nodes = linspace(1,5,5); % x nodes
y_values = [3 1 2 3 2]; % y values
t = linspace(1,5,100); % values where the interpolants are evaluated
linear = interp1(x_nodes,y_values,t); % the piecewise interpolant
cubic = spline(x_nodes,y_values,t); % not-a-knot cubic spline
plot(t,linear,’k.-.’,t,cubic,’b.-.’, x_nodes,y_values,’ro’)
legend(’piecewise linear’,’cubic spline’,’data points’)
Exercise 3. Consider the data
x 0 1/2 1 2 3y 0 1/4 1 -1 -1
1. Find the piecewise linear interpolating function for the data.2. Find the piecewise quadratic interpolating function.3. Find the natural cubic spline that interpolates the data.4. Find the not-a-knot interpolating cubic spline. When using (4.73), let x1 = 0,x2 = 1,x3 = 3, and z1 =
12 ,z2 =
2.Solution:
1. The first-order Newton divided differences are and the piecewise linear polynomial is
`(x) =
0+1/2x, x ∈ [0,1/2]1/4+3/2(x−1/2), x ∈ [1/2,1]1−2(x−1), x ∈ [1,2]−1, x ∈ [2,3].
2. The second-order Newton divided differences are49
0 0.5 1 1.5 2 2.5 3-2
-1.5
-1
-0.5
0
0.5
1
1.5Exercise 4.3.3
piecewise linear
piecewise quadratic
not-a-knot cubic spline
natural cubic spline
data points
polyfit p3
0 0.5 1 1.5 2 2.5 3-2
-1.5
-1
-0.5
0
0.5
1
1.5Exercise 4.3.3, (c) and (d)
not-a-knot cubic spline
natural cubic spline
FIG. 10.3. The piecewise linear, piecewise quadratic, not-a-knot cubic spline, natural cubic spline and the cubic least-squares polyfit.
x y [yi,y j]0 0
1/2 1/4 1/4−01/2 = 1/2
1 1 1−1/41−1/2 = 3/2
2 -1 −1−12−1 =−2
3 -1 −1−(−1)3−2 = 0
and the piecewise quadratic polynomial is
Π2(x) ={
0+1/2x+ x(x−1/2)≡ x2, x ∈ [0,1]1−2(x−1)+(x−1)(x−2)≡ x2−5x+5, x ∈ [1,3].
3. Find the natural cubic spline that interpolates the data. Let denote the nodes and the values
x0 = 0,x1 = 1/2,x2 = 1,x3 = 2,x4 = 3, (nodes)y0 = 0,y1 = 1/4,y2 = 1,y3 =−1,y4 =−1, (values)
and use formula (4.64) to find the coefficients M0,M1, · · · ,M4, by taking j = 1,2,3
x j− x j−1
6M j−1 +
x j+1− x j−1
3M j +
x j+1− x j
6M j+1 =
y j+1− y j
x j+1− x j−
y j− y j−1
x j− x j−1,
x1− x0
6M0 +
x2− x0
3M1 +
x2− x1
6M2 =
y2− y1
x2− x1− y1− y0
x1− x0, (j=1)
x2− x1
6M1 +
x3− x1
3M2 +
x3− x2
6M3 =
y3− y2
x3− x2− y2− y1
x2− x1, (j=2)
x3− x2
6M2 +
x4− x2
3M3 +
x4− x3
6M4 =
y4− y3
x4− x3− y3− y2
x3− x2, (j=3)
hence
1/2−06
M0 +1−0
3M1 +
1−1/26
M2 =1−1/41−1/2
− 1/4−01/2−0
, (j=1)
50
x y [yi,y j] [yi,y j,yk]0 0
1/2 1/4 1/4−01/2 = 1/2
1 1 1−1/41−1/2 = 3/2 3/2−1/2
1−0 = 12 -1 −1−1
2−1 =−23 -1 −1−(−1)
3−2 = 0 0−(−2)3−1 = 1
1−1/26
M1 +2−1/2
3M2 +
2−16
M3 =−1−12−1
− 1−1/41−1/2
, (j=2)
2−16
M2 +3−1
3M3 +
3−26
M4 =−1− (−1)
3−2− −1−1
2−1, (j=3)
or equivalently
112
M0 +13
M1 +1
12M2 =
32− 1
2≡ 1, (j=1)
112
M1 +12
M2 +16
M3 =−2− 32≡−7
2, (j=2)
16
M2 +23
M3 +16
M4 =−1− (−1)
3−2− −1−1
2−1≡ 2. (j=3)
From the definition of a natural cubic spline we have that M0 =M4 = 0, and solving the above system
M0 +13
M1 +112
M2 = 1, (j=1)
112
M1 +12
M2 +16
M3 =−2− 32≡−7
2, (j=2)
16
M2 +23
M3 +16
M4 =−1− (−1)
3−2− −1−1
2−1≡ 2. (j=3)
we obtain
M0 = 0, M1 =387, M2 =−
687, M3 =
387, M4 = 0.
Now using definition (4.63) we obtain the expressions on each interval for s3(x):
s3(x) =(x j− x)3M j−1 +(x− x j−1)
3M j
6(x j− x j−1)+
(x j− x)y j−1 +(x− x j−1)y j
x j− x j−1(4.63)
− 16(x j− x j−1)[(x j− x)M j−1 +(x− x j−1)M j], (on x ∈ [x j−1,x j])
by taking j = 1,2,3,4 respectively
s3(x) =(x1− x)3M0 +(x− x0)
3M1
6(x1− x0)+
(x1− x)y0 +(x− x0)y1
x1− x0− 1
6(x1− x0)[(x1− x)M0 +(x− x0)M1],
((j=1)⇒ x ∈ [x0,x1])
=(x2− x)3M1 +(x− x1)
3M2
6(x2− x1)+
(x2− x)y1 +(x− x1)y2
x2− x1− 1
6(x2− x1)[(x2− x)M1 +(x− x1)M2],
((j=2)⇒ x ∈ [x1,x2])51
=(x3− x)3M2 +(x− x2)
3M3
6(x3− x2)+
(x3− x)y2 +(x− x2)y3
x3− x2− 1
6(x3− x2)[(x3− x)M2 +(x− x2)M3],
((j=3)⇒ x ∈ [x2,x3])
=(x4− x)3M3 +(x− x3)
3M4
6(x4− x3)+
(x4− x)y3 +(x− x3)y4
x4− x3− 1
6(x4− x3)[(x4− x)M3 +(x− x3)M4].
((j=4)⇒ x ∈ [x3,x4])
Substituting in the values for the nodes, values and the coefficients Mi we obtain
s3(x) =x3 ·38/7
6/2+
x/41/2− 1
61/2 · x ·38/7, (x ∈ [0,1/2])
=(1− x)3 ·38/7− (x−1/2)3 ·68/7
6(1−1/2)+
(1− x)/4+(x−1/2)1−1/2
− 16(1−1/2)[(1− x) ·38/7− (x−1/2)68/7],
(x ∈ [1/2,1])
=−(2− x)3 ·68/7+(x−1)3 ·38/7
6(2−1)+
(2− x)− (x−1)2−1
− 16(2−1)[−(2− x) ·68/7+(x−1) ·38/7],
(x ∈ [1,2])
=(3− x)3 ·38/7
6(3−2)+−(3− x)− (x−2)
3−2− 1
6(3−2)(3− x)38/7. (x ∈ [2,3])
Finally
s3(x) =
3821
x3 +1
21x, x ∈ [0,1/2]
−10621
x3 +727
x2− 10721
x+67, x ∈ [1/2,1]
5321
x3− 877
x2 +37021
x− 477, x ∈ [1,2]
−1921
x3 +577
x2− 49421
x+145
7, x ∈ [2,3]
4. Find the not-a-knot interpolating cubic spline. When using (4.73), let x1 = 0,x2 = 1,x3 = 3, and z1 =12 ,z2 =
2.(See Exercise 4 3 3.m and Figure 10.3.)x_nodes = [ 0 1/2 1 2 3]; % x nodes
y_values = [ 0 1/4 1 -1 -1]; % y values
t = linspace(0,3,300); % values where the interpolants are evaluated
t1 = linspace(0,1,100);
t2 = linspace(1,3,200);
linear = interp1(x_nodes,y_values,t); % the piecewise interpolant
quadratic1 = t1.^2;
quadratic2 = t2.^2 - 5*t2 + 5;
cubic = spline(x_nodes,y_values,t); % not-a-knot cubic spline
p3 = polyfit(x_nodes,y_values,3) % polynomial of degree 3 approximated in L2 sense
p3eval = polyval(p3,t); %p3 polynomial evaluated
figure()
hold all
52
plot(t,linear,’k.-.’,[t1 t2],[quadratic1 quadratic2],...
’m.-.’,t,cubic,’b.-.’, x_nodes,y_values,’g*’)
plot(t,p3eval,’r.-.’)
legend(’piecewise linear’,’piecewise quadratic’,’cubic spline’,...
’data points’,’polyfit p3’)
end
Exercise 5. Use the MATLAB built-in function spline to interpolate (see also (Runge example))
f (x) =1
1+ x2 , x ∈ [−5,5],
from Example 4.2.5 (the Runge phenomenon) with the following sets of x nodes:(i) {−5,−2.5,0,2.5,5}
(ii) {−5,−3.5,−2,0,2,3.5,5}(iii) {−5,−4.5,−4,−3,−2,−1,0,1,2,3,4,4.5,5}In each case, graph the spline function and the function f (x). Compare with the global polynomial interpolation.Solution:
-5 -4 -3 -2 -1 0 1 2 3 4 5-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
cubic (i)
cubic (ii)
cubic (iii)
Runge
FIG. 10.4. Three not-a-knot cubic spline approximations of the Runge example for Exercise 5.
(See Exercise 4 3 5.m.)x_nodes1 = [ -5, -2.5 , 0 , 2.5 , 5]; % x nodes
x_nodes2 = [ -5, -3.5, -2 , 0 , 2, 3.5 , 5];
x_nodes3 = [ -5, -4.5, -4, -3, -2 , -1, 0 , 1, 2, 3, 4, 4.5 , 5];
y_values1 = 1./(1+x_nodes1.^2); % y values
y_values2 = 1./(1+x_nodes2.^2);
y_values3 = 1./(1+x_nodes3.^2);
t = linspace(-5,5,200); % values where the interpolants are evaluated
cubic1 = spline(x_nodes1,y_values1,t); % not-a-knot cubic spline
cubic2 = spline(x_nodes2,y_values2,t);
cubic3 = spline(x_nodes3,y_values3,t);
runge = 1./(1+t.^2);
% figure(1)
% plot(t,cubic1,’k.-.’,t, runge,’b-’)
53
% legend(’cubic interpolant’,’Runge’)
% figure(2)
% plot(t,cubic2,’k.-.’,t, runge,’b-’)
% legend(’cubic interpolant’,’Runge’)
% figure(3)
% plot(t,cubic3,’k.-.’,t, runge,’b-’)
% legend(’cubic interpolant’,’Runge’)
figure(4)
plot(t,cubic1,’r.-.’,t,cubic2,’b.-.’,t,cubic3,’k.-.’,t,runge,’g-’)
legend(’cubic (i)’,’cubic (ii)’,’cubic (iii)’,’Runge’)
Exercise 10. Is the following function
s(x) ={
(x−1)3, x ∈ [0,1]2(x−1)3, x ∈ [1,2]
a cubic spline on the interval x ∈ [0,2]?Solution: Since
s′(x) ={
3(x−1)2, x ∈ [0,1]6(x−1)2, x ∈ [1,2]
, s′′(x) ={
6(x−1), x ∈ [0,1]12(x−1), x ∈ [1,2]
we have that s,s′,s′′ are continuous at x = 1. Moreover, s is cubic on each subinterval, and therefore is a cubic spline.
Exercise 12. Define
s(x) =
2x3, x ∈ [0,1]x3 +3x2−3x+1, x ∈ [1,2]9x2−15x+9, x ∈ [2,3]
Verify that s(x) is a cubic spline function on [0,3]. Is it a natural cubic spline function on this interval?Solution: Since
s′(x) =
6x2, x ∈ [0,1]3x2 +6x−3, x ∈ [1,2]18x−15, x ∈ [2,3]
, s′′(x) =
12x, x ∈ [0,1]6x+6, x ∈ [1,2]18, x ∈ [2,3]
we see that s,s′,s′′ are continuous at x = 1 and x = 2. Being cubic on each subinterval, s is therefore a cubic spline.s′′(0) = 0, but s′′(3) = 18 6= 0, hence s is not a natural cubic spline.
Exercise 13. Define
s(x) ={
x3−3x2 +2x+1, x ∈ [1,2]−x3 +9x2−22x+17, x ∈ [2,3].
Is s(x) a cubic spline on [1,3]? Is it a natural cubic spline function?Solution: Since
s′(x) ={
3x2−6x+2, x ∈ [1,2]−3x2 +18x−22, x ∈ [2,3]
, s′′(x) ={
6x−6, x ∈ [1,2]−6x+18, x ∈ [2,3].
54
we see that s,s′,s′′ are continuous at x = 2. Being cubic on each subinterval, s is therefore a cubic spline.Also s′′(1) = 0, s′′(3) = 0, hence s is a natural cubic spline.
Exercise 14. Is the following function a cubic spline on [0,3]?
s(x) =
x3, x ∈ [0,1]2x−1, x ∈ [1,2]3x2−9, x ∈ [2,3]
Solution: We have
s′(x) =
3x2, x ∈ [0,1]2, x ∈ [1,2]6x, x ∈ [2,3]
, s′′(x) =
6x, x ∈ [0,1]0, x ∈ [1,2]6, x ∈ [2,3]
hence
s(1) = 1, s(2) = 3,s′(1−) = 1 6= 2 = s′(1+),
and therefore s is not a cubic spline.
Exercise 15. Define
s(x) ={
x3 +2x2 +1, x ∈ [1,2]−2x3 +βx2−36x+25, x ∈ [2,3]
For a special value of β , s(x) is a cubic spline function on [1,3]. Find that value of β and then verify that s(x) is acubic spline function on [1,3]. Is it a natural cubic spline function on this interval?Solution: Since
s(2−) = 17, s(2+) =−63+4β ,
in order s to be continuous we have
β = 20,
hence
s(x) ={
x3 +2x2 +1, x ∈ [1,2]−2x3 +20x2−36x+25, x ∈ [2,3]
s′(x) ={
3x2 +4x, x ∈ [1,2]−6x2 +40x−36, x ∈ [2,3]
, s′′(x) ={
6x+4, x ∈ [1,2]−12x+40, x ∈ [2,3] .
We have that
s′(2−) = 20 = s′(2+),s′′(2−) = 16 = s′(2+),
hence s is a cubic spline. Also, since
s′′(1) = 10 6= 0
s is not a natural cubic spline.55
Exercise 16. Is there a choice of coefficients {a,b,c,d} for which the following function is a cubic spline?
s(x) =
(x+1)3, x ∈ [−2,−1]ax3 +bx2 + cx+d, x ∈ [−1,1](x−1)2, x ∈ [1,2]
Solution: Since
s′(x) =
3(x+1)2, x ∈ [−2,−1]3ax2 +2bx+ c, x ∈ [−1,1]2(x−1), x ∈ [1,2]
, s′′(x) =
6(x+1), x ∈ [−2,−1]6ax+2b, x ∈ [−1,1]2, x ∈ [1,2]
by imposing the continuity condition, we obtain the following
s(−1−) = 0 =−a+b− c+d = s(−1+), s(1−) = a+b+ c+d = 0 = s(1+),
s′(−1−) = 0 = 3a−2b+ c = s′(−1+), s′(1−) = 3a+2b+ c = 0 = s′(−1+),
s′′(−1−) = 0 =−6a+2b = s′′(−1+), s′′(1−) = 6a+2b = 2 = s′′(−1+).
This overdetermined homogeneous system does not have a solution, so s cannot be a cubic spline.
56
11. ORTHOGONAL POLYNOMIALS IN APPROXIMATION THEORY.
11.1. APPROXIMATION FUNCTIONS BY GENERALIZED FOURIER SERIES.
DEFINITION 11.1. A function w = w(x) which is non-negative and integrable on (−1,1) is called a weightfunction. For example
w(x) = 1 (for ‘orthogonality’ of Legendre polynomials)
w(x) =1√
1− x2(for ‘orthogonality’ of Chebyshev polynomials)
DEFINITION 11.2 (Orthogonal polynomials). We say that the family of polynomials {pk;k = 0,1, · · · ,deg(pk) =k,}, i.e., pk ∈ Pk are mutually orthogonal on (−1,1) with respect to the weight w(·) if
∫ 1
−1pk(x)pm(x)dx = 0, ∀k 6= m. (orthogonal polynomials)
DEFINITION 11.3 (weighted scalar product and norm). The following define the weighted scalar product andweighted norm
〈 f ,g〉w :=∫ 1
−1f (x)g(x)w(x)dx (weighted scalar product)
‖ f‖2w := 〈 f , f 〉
12w (weighted norm)
for the (weighted) function space
L2w(−1,1) = { f : (−1,1)→ R,
∫ 1
−1f 2(x)w(x)dx < ∞}.
For example, with w(x) = 1√1−x2
, the functions f (x) = 1 and g(x) = x are mutually orthogonal with respect to the
L2w inner product
∫ 1
−11 · x 1√
1− x2=−
√1− x2
∣∣∣∣1−1
= 0.
DEFINITION 11.4 (Generalized Fourier series). For all f ∈ L2w, the series
S f (x) :=∞
∑k=0
fk pk(x), (generalized Fourier series)
where
fk =〈 f , pk〉w‖pk‖2
w(kth Fourier coefficient)
is called the generalized Fourier series of f , fk is the k-th Fourier coefficient.PROPOSITION 11.5. S f convergens in average (or in L2
w sense) to f :
limn→∞‖ f − fn‖w = 0, (11.1)
57
where fn is the truncation of order n
fn(x) =n
∑k=0
fk pk(x) ∈ Pn. (truncation of order n)
Moreover, the following Parseval’s equality holds
‖ f‖2w =
∞
∑k=0
f 2k ‖pk‖2
w, (Parseval equality)
and for any n
‖ f − fn‖2w =
∞
∑k=n+1
f 2k ‖pk‖2
wn↗∞−→ 0.
PROPOSITION 11.6 (minimization property). fn ∈ Pn satisfies the minimization property
‖ f − fn‖w = minq∈Pn‖ f −q‖w. (11.2)
i.e., fn is the orthogonal projection of f over Pn in the sense of L2w: fn ≡ ProjectionPn
( f ).Proof. We note that by the (truncation of order n) formula and the orthogonality definition we have
〈 f − fn,q〉w = 〈∞
∑k=n+1
fk pk(x)︸ ︷︷ ︸∈Pn+1
,q(x)︸︷︷︸∈Pn
〉w = 0.
Then using this property and the Cauchy-Schwarz inequality we have
‖ f − fn‖C2w = 〈 f − fn, f −q〉+
�����
��〈 f − fn︸ ︷︷ ︸∈Pn+1
,q− fn︸ ︷︷ ︸∈Pn
〉
= 〈 f − fn, f −q〉 ≤XXXXX‖ f − fn‖w‖ f −q‖w, ∀q ∈ Pn,
which concludes the proof.DEFINITION 11.7 (monic polynomials). The space of monic polynomials is
P1n =
{p(x) =
n
∑k=0
akxk, an = 1}.
THEOREM 11.8 (recursive 3-term formula).For any family of monic orthogonal polynomials {pk}, the following recursive 3-term formula holds
pk+1(x) = (x−αk)pk(x)−βk pk−1(x), k ≥ 0,p−1(x) = 0, p0(x) = 1
where
αk =〈xpk, pk〉w〈pk, pk〉w
, αk =〈pk+1, pk+1〉w〈pk, pk〉w
, k ≥ 0.
58
11.2. THE CHEBYSHEV POLYNOMIALS.Consider the Chebyshev weight
w(x) =1√
1− x2on x ∈ (−1,1). (Chebyshev weight)
The space of square integrable functions with respect to the (Chebyshev weight) w(x) is
L2w(−1,1) = { f : (−1,1)→ R :
∫ 1
−1f 2(x)(1− x2)−1/2dx < ∞}.
The inner product and norm are
〈 f ,g〉w =∫ 1
−1f (x)g(x)(1− x2)−1/2dx, (11.3)
‖ f‖w =
(∫ 1
−1f 2(x)(1− x2)−1/2dx
)1/2
.
DEFINITION 11.9 (Chebyshev polynomials). The (Chebyshev polynomials) are defined as
Tk(x) = cos(kθ), θ arccosx, k = 0,1, · · · . (Chebyshev polynomials)
PROPOSITION 11.10 (3-term recursion of (Chebyshev polynomials)). The (Chebyshev polynomials) can begenerated recursively by
Tk+1(x) = 2xTk(x)−Tk−1(x), k = 1,2, · · · ,T0(x) = 1, T1(x) = x.
In particular, for any k ≥ 0 notice that
Tk ∈ Pk.
Proof. Using the trigonometric identities
cos(n+1)θ = cosnθ cosθ − sinnθ sinθ
cos(n−1)θ = cosnθ cosθ + sinnθ sinθ
and adding them we obtain
cos(n+1)θ︸ ︷︷ ︸Tn+1(x)
= 2cosnθ︸ ︷︷ ︸Tn(x)
cosθ − cos(n−1)θ︸ ︷︷ ︸=Tn−1(x)
the 3-term recursion.COROLLARY 11.11.
Tn(x) = 2n−1xn + terms of lower degree,
in other words 12n−1 Tn is a monic polynomial.
For example
T2(x) = 2x2−1, T3(x) = 4x3−3x, T4(x) = 8x4−8x2 +1.59
REMARK 13. Using the well-known trigonometric relations
〈Tk,Tn〉w = 0 if k 6= n,
〈Tk,Tn〉w =
{c0 = π, if n = 0,cn =
π
2 , if n 6= 0.
it follows that the (Chebyshev polynomials) are mutually orthogonal with respect with the inner product (11.3).The Chebyshev series of f ∈ L2
w(−1,1) takes the form
C f =∞
∑k=0
fkTk, with fk =1ck
∫ 1
−1f (x)Tk(x)(1− x2)−1/2 dx.
REMARK 14. From the definition, the (Chebyshev polynomials) satisfy the following bound
‖Tn‖∞ = maxx∈[−1,1]
∣∣Tn(x)∣∣= 1, for every n. (11.4)
Let us recall now the (interpolation error) formula for global polynomial interpolation is
f (x)− (Πn f )(x) =f (n+1)(cx)
(n+1)!ωn+1(x), x ∈ [−1,1].
where ωn+1 = (x−x0) · · ·(x−xn), a monic polynomial, is the (nodal polynomial). Let us ask the following question
what are the nodes x0, · · · ,xn which minimize the “size” of ωn+1(x)?Or, in other words, what is the nodal polynomial ωn+1(x) of minimal size?
PROPOSITION 11.12. The nodal polynomial of minimal size is ωn+1(x) = 12n Tn+1(x),
‖ωn+1‖∞ ≥ ‖21−(n+1)Tn+1‖∞ ≡ ‖2−nTn+1‖∞,
the minimal size is 12n ,
and the interpolation nodes are the zeros of Tn+1(x):
x j = cos(
2 j+12n+2
π
), j = 0,1, · · · ,n.
THEOREM 11.13 (Min-max theorem of Chebyshev).∥∥∥∥ 12n−1 Tn
∥∥∥∥∞
≤ minp∈P1
n
‖p‖∞. (11.5)
11.3. THE LEGENDRE POLYNOMIALS.The Legendre polynomials are (orthogonal polynomials) on (−1,1) w.r.t. the weight function w(x) = 1, forming thespace L2
w(−1,1)≡ L2(−1,1).DEFINITION 11.14. The Legendre polynomials can be defined by the 3-term recursion formula
Lk+1(x) =2k+1k+1
xLk(x)−k
k+1Lk−1(x), k = 1,2, · · · (11.6)
60
L0(x) = 1, L1(x) = x (3-term Legendre recursion)
or equivalently by
L0(x) = 1, Ln(x) =1
n!2ndn
dxn
((x2−1)n
). (Legendre polynomials)
REMARK 15. For all k = 0,1, · · · we have that
Lk ∈ Pk,
〈Lk,Lm〉= δk,m
(k+
12
)−1, k,m = 0,1,2, · · · (orthogonality)
For any f ∈ L2(−1,1), its Legendre series is
L f =∞
∑k=0
fkLk with fk =(k+1/2
)∫ 1
−1f (x)Lk(x)dx.
Examples:
L0(x) = 1, L1(x) = x, L2(x) =12(3x2−1), L3(x) =
12(5x3−3x), L4(x) =
18(35x4−30x2 +3
). (11.7)
61
12. Chapter 5. Numerical integration and Differentiation.
12.1. Section 5.0 NUMERICAL INTEGRATION.Let f ∈ L1(a,b) (a real integrable function). Computing explicitly the definite integral
I( f ) =∫ b
af (x)dx
may be difficult or even impossible.DEFINITION 12.1 (Quadrature formula).
Any explicit formula that is suitable for providing an approximation of I( f ) is said to be a quadrature formula ornumerical integration formula.
EXAMPLE 4. For example, consider replacing f by an approximation fn (depending on n ∈ N) and integratingfn exactly.We would expect to have I( f )≈ In( f ) :=
∫ ba f (x)dx, namely
f (x)≈ fn(x),
I( f ) :=∫ b
af (x)dx≈
∫ b
afn(x)dx := In( f ).
REMARK 16. Let f ∈C0[a,b]. Then the quadrature error
En( f ) = I( f )− In( f )≡∫ b
a
(f (x)− fn(x)
)dx (quadrature error)
satisfies
∣∣En( f )∣∣≤ ∫ b
a
∣∣ f (x)− fn(x)∣∣dx≤ (b−a)
∥∥ f − fn∥∥
∞.
Therefore, if for some n we have that ‖ f − fn‖∞ ≤ ε , then∣∣En( f )
∣∣≤ ε(b−a).REMARK 17. The approximating function fn should be easily integrable, as it would be the case if for example
fn ∈ Pn.Therefore we naturally consider now the following approach, i.e., we approximate f ≈ (Πn f ) by the Lagrangeinterpolating polynomial over n+1 distinct nodes x0,x1, · · · ,xn ∈ [a,b]
fn(x) := (Πn f )(x) =n
∑i=0
f (xi)`i(x),
and where `i(x) is the characteristic Lagrange polynomial of degree n associated with the node xi (see e.g. (Lagrange characteristic polynomial)).
DEFINITION 12.2 (Lagrange quadrature formula).Integrating exactly Πn f we obtain the following quadrature formula
In( f ) =n
∑i=0
f (xi)∫ b
a`i(x)dx. (Lagrange quadrature formula)
REMARK 18. We note that (Lagrange quadrature formula) is a special instance of the following quadratureformula
In( f ) =n
∑i=1
αi f (xi), (quadrature formula)
62
where
αi =∫ b
a`i(x)dx, (quadrature coefficients or weights)
x0,x1, · · · ,xn (quadrature nodes)
REMARK 19. The (Lagrange quadrature formula) can be generalized to the (Hermite quadrature formula) byapproximating f with its Hermite interpolating polynomial Hn f , i.e. f ≈ (Hn f ), to obtain
1
∑κ=0
n
∑i=0
αik f (κ)(xi) (Hermite quadrature formula)
We say that both the (Lagrange quadrature formula) and (Hermite quadrature formula) are interpolatory quadra-ture formulae, as in both cases f ≈Πn f or f ≈ Hn f are obtained by exact integration of interpolating polynomials.
DEFINITION 12.3 (Degree of exactness / degree of precision).We define as the degree of exactness (D.O.E.) of a (quadrature formula) the maximum integer r ≥ 0 for which
In( f )≡ I( f ), ∀ f ∈ Pr
i.e.,n
∑i=0
αi f (xi) =∫ b
af (x)dx ∀ f ∈ Pr.
PROPOSITION 12.4. Any interpolatory quadrature formula that uses n+1 distinct nodes has degree of exactnessat least n.
Proof. Let f ≡ (Πn f ), where Πn f is the interpolating polynomial on the n+1 nodes. Then
I(Πn f ) :=∫ b
a(Πn f )(x)dx =
∫ b
a
( n
∑i=0
f (xi)`i(x))
dx =n
∑i=0
f (xi)∫ b
a`i(x)dx = In( f ),
so r ≥ n.REMARK 20. The degree of exactness of a (Lagrange quadrature formula) can be as large as 2n+1, in the case
of the so-called Gaussian quadrature formulae, where the quadrature nodes (also called Gauss nodes) are the zerosof Chebyshev polynomials.
Interpolatory Quadrature: instances of formula (Lagrange quadrature formula) for n = 0,1,2.Particular cases of the Newton-Cotes formulae:The midpoint rule (n = 0), trapezoidal rule (n = 1), Cavalieri-Simpson rule (n = 2).
12.1.1. THE MIDPOINT OR RECTANGLE FORMULA.
This is the case corresponding to (Lagrange quadrature formula) for n = i = 0, i.e. approximating f on the interval[a,b] by the constant value (polynomial of degree 0) f
( a+b2
), and integrate exactly:
f (x)≈ (Π0 f )(x) := f(a+b
2),
I( f ) :=∫ b
af (x)dx≈
∫ b
af(a+b
2
)dx≡ (b−a) f
(a+b2
):= I0( f ),
63
giving
I0( f ) = (b−a)︸ ︷︷ ︸:=α0
f(a+b
2
), (midpoint quadrature rule)
which in the language of (quadrature formula) has• weight α0 = b−a• quadrature node x0 =
a+b2 .
THEOREM 12.5. If f ∈C2[a,b], the (midpoint quadrature error) is
E0( f ) =h3
24f ′′(ξ ), (midpoint quadrature error)
where h = b−a, and ξ ∈ (a,b) is an arbitrary point.COROLLARY 12.6. The degree of exactness of the (midpoint quadrature rule) is 1: D.O.E. =1.Proof. Since for any constant and affine functions f (r = 0,r = 1) we have f ′′= 0, we obtain by the (midpoint quadrature error)
formula that the error is zero
E0( f ) = 0≡ I( f )− I0( f ),
hence the degree of exactness is 1.REMARK 21. If h= b−a the length of the integration interval is not sufficiently small, the (midpoint quadrature error)
can be quite large.This drawback is common to all numerical integration formulae and can be overcome by using composite rules.
THE COMPOSITE MIDPOINT FORMULA
In order to minimize the error in the midpoint rule on relatively large intervals [a,b], instead of replacing f over [a,b]by Π0 f , we replace it with a composite Lagrange polynomial of degree , on m≥ 1 subintervals.Let us introduce the following quadrature nodes:
xk = a+(2k+1)H2, k = 0, · · · ,m−1, (quadrature nodes)
where H =b−a
m. (mesh size)(
x0 = a+H2, x1 = a+H +
H2, · · · , xm−1 = a+(m−1)H +
H2.)
Then using the (midpoint quadrature rule) we obtain
I0,m = Hm−1
∑k=0
f (xk) (composite midpoint formula)
and by (midpoint quadrature error) we have respectivelyTHEOREM 12.7 (The composite midpoint error).
E0,m =b−a
24H2 f ′′(ξ ), (composite midpoint error)
provided f ∈C2[a,b], where ξ ∈ (a,b).(Due to the dependence on H2 of the (composite midpoint error) formula, we say that the (composite midpoint formula)has the order of infinitesimal = 2.
64
Proof. Using the additivity of the integrals, the (midpoint quadrature error) formula and the discrete mean valuetheorem we have
E0,m =m−1
∑k=0
H3
24f ′′(ξk) =
H3
24m f ′′(ξ ) =
b−a24
H2 f ′′(ξ ),
which concludes the proof.
REMARK 22 (asymptotic error formula). If instead of using the discrete mean value theorem we use the defini-tion of the Riemann sum, from the proof of Theorem 12.7 we obtain
E0,m( f ) =m−1
∑k=0
124
H3 f ′′(ξk) =1
24H2 H
m
∑k=0
f ′′(ξk)︸ ︷︷ ︸≈∫ b
a f ′′(x)dx
namely
E0,m( f )≈ H2
24(
f ′(b)− f ′(a)). (asymptotic midpoint error formula)
Since the (asymptotic midpoint error formula) provides an estimate of the error
E0,m( f ) = I( f )− I0,m( f ),
we can then also write the true value I( f ) as
I( f ) = I0,m( f )+E0,m( f )≈ I0,m( f )+H2
24(
f ′(b)− f ′(a)),
which leads to the following definition.
DEFINITION 12.8 (Corrected midpoint rule).
Icorrected0 ( f ) := I0,m( f )+
H2
24(
f ′(b)− f ′(a)). (corrected midpoint rule)
REMARK 23 (order of infinitesimal - ratio of consecutive errors).From Theorem 12.7 or by the (asymptotic midpoint error formula) we notice that the ratio of the errors correspondingto composite midpoint rule with respectively m and 2m nodes (”doubling the number of nodes, halving the mesh”)satisfies
E0,m
E0,2m=
b−a24
(b−a)2
m2 f ′′(ξ )b−a24
(b−a)2
(2m)2 f ′′(ξ )≈ 22, and
log(E0,m/E0,2m)/ log(2)≈ 2. (order of infinitesimal)
Section 5.1 THE TRAPEZOIDAL AND SIMPSON RULES
65
12.1.2. THE TRAPEZOIDAL RULE.
This is the case corresponding to (Lagrange quadrature formula) for n = 1, i = 0,1, i.e. approximating f on theinterval [a,b] by the piecewise linear interpolating polynomial (of degree n = 1) Π1 f , and integrate exactly:
f (x)≈ (Π1 f )(x) := f (a)x−ba−b
+ f (b)x−ab−a
,
I( f ) :=∫ b
af (x)dx≈
∫ b
a
(f (a)
x−ba−b
+ f (b)x−ab−a
)dx := I1( f ),
giving
I1( f ) =∫ b
a
(f (a)
x−ba−b
+ f (b)x−ab−a
)dx = f (a)
∫ b
a
x−ba−b
dx+ f (b)∫ b
a
x−ab−a
dx
=f (a)
a−b
∫ b
a(x−b)dx+
f (b)b−a
∫ b
a(x−a)dx =
f (a)a−b
12(x−b)2
∣∣∣x=b
x=a+
f (b)b−a
12(x−a)2
∣∣∣x=b
x=a
=−a−b2
f (a)+b−a
2f (b) =
b−a2(
f (a)+ f (b))
(trapezoidal quadrature rule)
I1( f ) =b−a
2︸ ︷︷ ︸α0
f (a)+b−a
2︸ ︷︷ ︸α1
f (b) (trapezoidal quadrature rule)
which in the language of (quadrature formula) has• weights: α0 = α1 =
b−a2
• quadrature nodes x0 = a,x1 = b.THEOREM 12.9. If f ∈C2[a,b], the (trapezoidal quadrature error) is
E1( f ) =− h3
12f ′′(ξ ), (trapezoidal quadrature error)
where h = b−a, and ξ ∈ (a,b) is an arbitrary point.COROLLARY 12.10. The degree of exactness of the (trapezoidal quadrature rule) is 1: D.O.E. =1, as the
(midpoint quadrature rule).Proof. Since for any constant and affine functions f (r = 0,r = 1) we have f ′′= 0, we obtain by the (trapezoidal quadrature error)
formula that the error is zero
E1( f ) = 0≡ I( f )− I1( f ),
hence the degree of exactness is 1.REMARK 24. If h= b−a the length of the integration interval is not sufficiently small, the (trapezoidal quadrature error)
can be quite large.This drawback is common to all numerical integration formulae and can be overcome by using composite rules.
THE COMPOSITE TRAPEZOIDAL FORMULA
In order to minimize the error in the trapezoidal rule on relatively large intervals [a,b], instead of replacing f over[a,b] by Π1 f , we replace it with a composite Lagrange polynomial of degree 1, on m≥ 1 subintervals.Let us introduce the following quadrature nodes:
xk = a+ kH, k = 0, · · · ,m, (quadrature nodes)
where H =b−a
m. (mesh size)
66
Then using the (trapezoidal quadrature rule) we obtain
I1,m =H2
m−1
∑k=0
(f (xk)+ f (xk−1)
)= H
(12
f (x0)+ f (x1)+ · · ·+ f (xm−1)+12
f (xm))
=H2
f (x0)+H f (x1)+ · · ·+H f (xm−1)+H2
f (xm) (composite trapezoidal formula)
and by (trapezoidal quadrature error) we have respectivelyTHEOREM 12.11 (The composite trapezoidal error).
E1,m =−b−a12
H2 f ′′(ξ ), (composite trapezoidal error)
provided f ∈C2[a,b], where ξ ∈ (a,b).(Due to the dependence on H2 of the (composite trapezoidal error) formula, we say that the (composite trapezoidal formula)has the order of infinitesimal = 2.
Proof. Using the additivity of the integrals, the (trapezoidal quadrature error) formula and the discrete meanvalue theorem we have
E1,m =m
∑k=0−H3
12f ′′(ξk) =−
H3
12m f ′′(ξ ),=−b−a
12H2 f ′′(ξ ),
which concludes the proof.
REMARK 25 (asymptotic error formula). If instead of using the discrete mean value theorem we use the defini-tion of the Riemann sum, from the proof of Theorem 12.11 we obtain
E1,m( f ) =m
∑k=0− 1
12H3 f ′′(ξk) =−
112
H2 Hm
∑k=0
f ′′(ξk)︸ ︷︷ ︸≈∫ b
a f ′′(x)dx
namely
E1,m( f )≈−H2
12(
f ′(b)− f ′(a)). (asymptotic trapezoidal error formula)
Since the (asymptotic trapezoidal error formula) provides an estimate of the error
E1,m( f ) = I( f )− I1,m( f ),
we can then also write the true value I( f ) as
I( f ) = I1,m( f )+E1,m( f )≈ I1,m( f )− H2
12(
f ′(b)− f ′(a)),
which leads to the following definition.
DEFINITION 12.12 (Corrected trapezoidal rule).
Icorrected1 ( f ) := I1,m( f )− H2
12(
f ′(b)− f ′(a)). (corrected trapezoidal rule)
67
REMARK 26 (order of infinitesimal - ratio of consecutive errors).From Theorem 12.11 or by the (asymptotic trapezoidal error formula) we notice that the ratio of the errors corre-sponding to composite trapezoidal rule with respectively m and 2m nodes (”doubling the number of nodes, halvingthe mesh”) satisfies
E1,m
E1,2m=− b−a
12(b−a)2
m2 f ′′(ξ )
− b−a12
(b−a)2
(2m)2 f ′′(ξ )≈ 22, and
log(E1,m/E1,2m)/ log(2)≈ 2. (order of infinitesimal)
12.1.3. THE CAVALIERI-SIMPSON RULE.
This is the case corresponding to (Lagrange quadrature formula) for n = 2, i = 0,1,2, i.e. approximating f on theinterval [a,b] by the piecewise quadratic interpolating polynomial (of degree n = 2) Π2 f on interpolating nodesx0 = a,x1 =
a+b2 ,x2 = b, and integrate exactly:
f (x)≈ (Π2 f )(x) := f (a)+f ( a+b
2 )− f (a)(b−a)/2
(x−a)++f (b)−2 f ( a+b
2 )+ f (a)(b−a)2/2
(x−a)(x− a+b2 ),
I( f ) :=∫ b
af (x)dx≈
∫ b
a(Π2 f )(x)dx := I2( f ),
giving
I2( f ) =b−a
6
(f (a)+4 f
(a+b2)+ f (b)
)(Cavalieri-Simpson quadrature rule)
I2( f ) =b−a
6︸ ︷︷ ︸α0
f (a)+4b−a
6︸ ︷︷ ︸α1
f ( a+b2 )+
b−a6︸ ︷︷ ︸α2
f (b) (Cavalieri-Simpson quadrature rule)
which in the language of (quadrature formula) has• weights: α0 =
b−a2 , α1 = 4 b−a
6 , α2 =b−a
2 .• quadrature nodes x0 = a,x1 =
a+b2 ,x2 = b.
REMARK 27. From the error analysis of the midpoint and trapezoidal rules, we note that the error in the mid-point rule is half the size of the error in the trapezoidal rule and of opposite sign, hence a linear combination of themidpoint and trapezoidal rules should be a better approximation. Indeed, notice that the (Cavalieri-Simpson quadrature rule)can be written as the linear combination of the (midpoint quadrature rule) and (trapezoidal quadrature rule):
I2( f ) =23
I0( f )+13
I1( f ),
with the error given in this next result.THEOREM 12.13. If f ∈C4[a,b], the (Simpson quadrature error) is
E2( f ) =− 190
(h2
)5f (4)(ξ ), (Simpson quadrature error)
where h = b−a, and ξ ∈ (a,b) is an arbitrary point.68
COROLLARY 12.14. The degree of exactness of the (Cavalieri-Simpson quadrature rule) is 1: D.O.E. =3.Proof. Since for any cubic functions f (r = 0,1,2,3) we have f (4)= 0, we obtain by the (Simpson quadrature error)
formula that the error is zero
E2( f ) = 0≡ I( f )− I2( f ),
hence the degree of exactness is 3.REMARK 28. If h= b−a the length of the integration interval is not sufficiently small, also the (Simpson quadrature error)
can be quite large.This drawback is common to all numerical integration formulae and can be overcome by using composite rules.
THE COMPOSITE CAVALIERI-SIMPSON FORMULA
In order to minimize the error in the Cavalieri-Simpson rule on relatively large intervals [a,b], instead of replacing fover [a,b] by Π2 f , we replace it with a composite Lagrange polynomial of degree 2, on m≥ 1 subintervals.Let us introduce the following quadrature nodes:
xk = a+ kH2, k = 0, · · · ,2m, (quadrature nodes)
where H =b−a
m. (mesh size)
Then using the (Cavalieri-Simpson quadrature rule) we obtain
I2,m( f ) =H6
[f (x0)+2
m−1
∑r=1
f (x2r)+4m−1
∑s=0
f (x2s+1)+ f (x2m)]
(composite Simpson formula)
and by (Simpson quadrature error) we have respectivelyTHEOREM 12.15 (The composite Simpson error).
E2,m( f ) =−b−a180
(H2
)4f (4)(ξ ), (composite Simpson error)
provided f ∈C4[a,b], where ξ ∈ (a,b).(Due to the dependence on H4 of the (composite Simpson error) formula, we say that the (composite Simpson formula)has the order of infinitesimal = 4.
Proof. Indeed, using the (Simpson quadrature error) formula and the discrete mean value theorem we have that
E2,m( f ) =m−1
∑k=0− 1
90
(H2
)5f (4)(ξk) =−
1180
(H2
)4H
m−1
∑k=0
f (4)(ξk)︸ ︷︷ ︸=m f (4)(ξ )
=−b−a180
(H2
)4f (4)(ξ ),
which concludes the proof.
REMARK 29. Verify the (composite Simpson error) formula for f (x) = x4 on [0,1], and on two subintervals.
I( f ) =∫ 1
0x4dx =
15,
I2,1( f ) =16
(04 +4 · 1
24 +14)=
524
, (using (Cavalieri-Simpson quadrature rule))
69
E2,1( f ) = I( f )− I2,1( f )≡ 15− 5
24=− 1
120,
f (4)(x) = 4!,
E2,1( f ) =−b−a180
(H2
)4f (4)(ξ )≡− 1
180
(12
)424 =− 1
120. (using (composite Simpson error))
I2,2( f ) =112
(04 +4 · 1
44 +124 +
124 +4 · 3
4
44 +14)=
77384
, (using (composite Simpson formula), H = 12 )
E2,2( f ) = I( f )− I2,2( f )≡ 15− 77
384=
384−3851920
=− 11920
,
E2,2( f ) =−b−a180
(H2
)4f (4)(ξ )≡− 1
180
(14
)424 =− 1
1920. (using (composite Simpson error))
REMARK 30 (asymptotic error formula). If instead of using the discrete mean value theorem we use the defini-tion of the Riemann sum, from the above we obtain
E2,m( f ) =m−1
∑k=0− 1
90
(H2
)5f (4)(ξk) =−
1180
(H2
)4H
m−1
∑k=0
f (4)(ξk)︸ ︷︷ ︸≈∫ b
a f (4)(x)dx
namely
E2,m( f )≈− 1180
(H2
)4(f (3)(b)− f (3)(a)
). (asymptotic Simpson error formula)
Since the (asymptotic Simpson error formula) provides an estimate of the error
E2,m( f ) = I( f )− I2,m( f ),
we can then also write the true value I( f ) as
I( f ) = I2,m( f )+E2,m( f )≈ I2,m( f )− 1180
(H2
)4(f (3)(b)− f (3)(a)
),
which leads to the following definition.
DEFINITION 12.16 (Corrected Simpson rule).
Icorrected2 ( f ) := I2,m( f )− 1
180
(H2
)4(f ′′′(b)− f ′′′(a)
). (corrected Simpson rule)
REMARK 31 (order of infinitesimal - ratio of consecutive errors).From Theorem 12.15 or by the (asymptotic Simpson error formula) we notice that the ratio of the errors correspond-ing to composite Cavalieri-Simpson rule with respectively m and 2m nodes (”doubling the number of nodes, halvingthe mesh”) satisfies
E2,m
E2,2m=− b−a
180(b−a)4
24m4 f (4)(ξ )
− b−a180
(b−a)4
24(2m)4 f ′′(ξ )≈ 24, and
log(E2,m/E2,2m)/ log(2)≈ 4. (order of infinitesimal)
70
12.1.4. RICHARDSON EXTRAPOLATION.“Richardson extrapolation is a sequence acceleration method, used to improve the rate of convergence of a sequenceof estimates of some value
I = limm→∞
Im.
In essence, given the value of Im for several values of m, we can estimate I by extrapolating the estimates to m = ∞.”We recall that by (composite midpoint error), (composite trapezoidal error), (composite Simpson error), the er-
rors in the midpoint, trapezoidal and Simpson rules are all of the form
I− Im ≈c
mp , (error form)
namely:
I− I0,m = E0,m =b−a
24H2 f ′′(ξ )≡ (b−a)3
24f ′′(ξ )
1m2 , (midpoint: p = 2,c = (b−a)3
24 f ′′(ξ ))
I− I1,m = E1,m =−b−a12
H2 f ′′(ξ )≡− (b−a)3
12f ′′(ξ )
1m2 , (trapezoidal: p = 2,c = (b−a)3
24 f ′′(ξ ))
I− I2,m = E2,m =−b−a180
(H2
)4f (4)(ξ )≡− (b−a)5
24 ·180f (4)(ξ )
1m4 , (Simpson: p = 4,c =− (b−a)5
24·180 f (4)(ξ ))
hence the constants c and p in (error form) differ with the method and the function.Using the general form of the error (error form) with 2m we have
I− I2m ≈c
2pmp ,
hence using again (error form) in the above we have
I ≈ I2m +c
2pmp ≈ I2m +12p
(I− Im
),
2p−12p I = I2m−
12p Im,
I ≈R2m :=2p
2p−1I2m−
12p−1
Im (Richardson extrapolation)
I− I2m ≈ I−R2m ≡2p
2p−1I2m−
12p−1
Im− I2m =1
2p−1(I2m− Im
)(Richardson’s error estimate in I2m)
REMARK 32. From (error form) we see that Im is an approximation of I with the order of convergence p, whilethe (Richardson extrapolation) R2m is an approximation of I with order at least p+1. (See also Figure 12.1.)
Im↘
I2m → R2m↘ ↘
I4m → R4m →
REMARK 33. In particular, from (Richardson’s error estimate in I2m) we obtain that in the case of the midpoint(p = 2), trapezoidal (p = 2) and Simpson rules (p = 2), the errors in the quadrature rule using 2m points are
I− I0,2m ≈13(I0,2m− I0,m
)(error in midpoint rule)
I− I1,2m ≈13(I1,2m− I1,m
)(error in trapezoidal rule)
71
I− I2,2m ≈1
15(I2,2m− I2,m
)(error in Simpson rule)
Example 5.1.3 , page 193; Example 5.1.5 , page 198. Compute the integral∫ 1
0e−x2
dx≈ 0.746824132812427
using the (composite) midpoint, trapezoidal, Simpson rule on 2 and 4 subintervals.Find the errors and the ratios by which the errors decreases (rates of convergence / order of infinitesimal).Solution: First we compute the Trapezoidal integrals on m = 2 subintervals:
I(e−x2) =
∫ 1/2
0(e−x2
)dx+∫ 1
1/2(e−x2
)dx
≈ I1,2(e−x2)
H=1/2====
H2
(f (0)+2 f (1/2)+ f (1)
)=
14(e0 +2e−1/4 + e−1)= 0.731370251828563
and m = 4 subintervals:
I(e−x2) =
∫ 1/4
0(e−x2
)dx+∫ 1/2
1/4(e−x2
)dx+∫ 3/4
1/2(e−x2
)dx+∫ 1
3/4(e−x2
)dx
≈ I1,4(e−x2)
H=1/4====
H2
(f (0)+2 f (1/4)+2 f (1/2)+2 f (3/4)+ f (1)
)=
18(e0 +2e−1/16 +2e−1/14 +2e−9/16 + e−1)= 0.742984097800381.
The errors are
E1,2 = I− I1,2 = 0.746824132812427−0.731370251828563 = 0.015453880983864,E1,4 = I− I1,4 = 0.746824132812427−0.742984097800381 = 0.003840035012046,
hence the computed order of infinitesimal (rate of convergence) is
log(E1,2/E1,4)/ log(2) = 2.008777822283925≈ 2.0088.
Similarly, we compute the Simpson integrals on m = 2 subintervals:
I(e−x2) =
∫ 1/2
0(e−x2
)dx+∫ 1
1/2(e−x2
)dx
≈ I2,2(e−x2)
H=1/2====
H6
(f (0)+4 f (1/4)+ f (1/2)+ f (1/2)+4 f (3/4)+ f (1)
)=
112(e0 +4e−1/16 +2e−1/4 +4e−9/16 + e−1)= 0.746855379790987
and m = 4 subintervals:
I(e−x2) =
∫ 1/4
0(e−x2
)dx+∫ 1/2
1/4(e−x2
)dx+∫ 3/4
1/2(e−x2
)dx+∫ 1
3/4(e−x2
)dx
≈ I1,4(e−x2)
H=1/4====
H6
(f (0)+4 f (1/8)+2 f (1/4)+4 f (3/8)+2 f (1/2)+4 f (5/8)+2 f (3/4)+4 f (7/8)+ f (1)
)72
=124(e0 +4e−1/64 +2e−1/16 +4e−9/64 +2e−1/14 +4e−25/64 +2e−9/16 +4e−49/64 + e−1)= 0.746826120527467.
The errors are
E2,2 = I− I2,2 = 0.746824132812427−0.746855379790987 =−3.124697856005110×10−5,
E2,4 = I− I2,4 = 0.746824132812427−0.746826120527467 =−1.987715040008275×10−6,
hence the computed order of infinitesimal (rate of convergence) is
log(E2,2/E2,4)/ log(2) = 3.974533843274072≈ 3.9745.
100
101
102
n
10-14
10-12
10-10
10-8
10-6
10-4
10-2
err
ors
Midpoint, Corrected midopint and Richardson M-extrapolation
Midpoint
corrected Midpoint
Richardson extrapolation
100
101
102
n
10-14
10-12
10-10
10-8
10-6
10-4
10-2
100
err
ors
Trapezoidal, Corrected trapezoidal and Richardson T-extrapolation
Trapezoidal
corrected Trapezoidal
Richardson extrapolation
100
101
102
n
10-16
10-14
10-12
10-10
10-8
10-6
10-4
10-2
err
ors
Simpson, corrected Simpson and Richardson S-extrapolation
Simpson
corrected Simpson
Richardson extrapolation
100
101
102
n
10-13
10-12
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
err
ors
Corrected midpoint/Trapezoidal, and Richardson-extrapolated midpoint/Trapezoidal vs Simpson
corrected midpoint
corrected Trapezoidal
Richardson-extrapolated midpoint
Richardson-extrapolated Trapezoidal
Simpson
FIG. 12.1. .
>> Example_5_1_3_1st
Midpoint =
0.778800783071405
0.754597943772199
0.748747131891009
73
0.747303578730748
0.746943912516367
0.746854072623361
0.746831617445408
0.746826003950687
0.746824600595743
Trapezoidal =
0.683939720585721
0.731370251828563
0.742984097800381
0.745865614845695
0.746584596788222
0.746764254652294
0.746809163637828
0.746820390541618
0.746823197246153
Trapezoidal calculations for int_0^1 exp(-x^2) dx:
1st <the computed errors>: I-I_{1,m}
2nd <the estimated errors using the Richardson‘s estimate: (1/3)*(I_{1,2m}-I_{1,m})>
3rd <the errors in Corrected Trapezoidal>: I-CI_{1,m}
4th <the errors in Richardson‘s extrapolation of Trapezoidal>: I-RI_{1,m}
Tabular =
85 table
n Error RichardsonErrorEstim ErrorCorrected ErrorRichardson
___ __________ ____________________ ______________ _______________
2 0.015454 0.01581 0.00012557 -0.0003563
4 0.00384 0.0038713 7.9575e-06 -3.1247e-05
8 0.00095852 0.00096051 4.9859e-07 -1.9877e-06
16 0.00023954 0.00023966 3.118e-08 -1.2462e-07
32 5.9878e-05 5.9886e-05 1.949e-09 -7.7946e-09
64 1.4969e-05 1.497e-05 1.2182e-10 -4.8725e-10
128 3.7423e-06 3.7423e-06 7.6136e-12 -3.0454e-11
256 9.3557e-07 9.3557e-07 4.7562e-13 -1.9038e-12
5th <the order of convergence errors>: p=log(I-I_{1,m})/log(2)
6th <the ratios of differences>: (I_{1,2m}-I_{1,m})/(I_{1,4m}-I_{1,2m})
7th <the order with ratios>: p=log((I_{1,2m}-I_{1,m})/(I_{1,4m}-I_{1,2m}))/log(2)
Tabular =
86 table
74
n OrderConvExact ApproxRatio OrderWithRatios OrderConvCorrected orderTRichardson
___ ______________ ___________ _______________ __________________ ________________
2 2.0247 0 -Inf 3.6453 3.9745
4 2.0088 0 2.03 3.98 3.9955
8 2.0022 4.084 2.0109 3.9964 3.999
16 2.0006 4.0305 2.0028 3.9992 3.9997
32 2.0001 4.0078 2.0007 3.9998 3.9999
64 2 4.002 2.0002 4 3.9997
128 2 4.0005 2 4 -Inf
256 2 4.0001 2 4.0007 -Inf
Simpson calculations for int_0^1 exp(-x^2) dx:
1st <the computed errors>: I-I_{2,m}
2nd <the estimated errors using the Richardson‘s estimate: (1/15)*(I_{2,2m}-I_{2,m})>
3rd <the errors in Corrected Simpson>: I-CI_{2,m}
4th <the errors in Richardson‘s extrapolation of Simpson>: I-RI_{2,m}
Tabular =
89 table
n Error RichardsonErrorEstim ErrorCorrected ErrorRichardson OrderConvExact OrderConvCorrected orderTRichardson ApproxRatio
___ ___________ ____________________ _______________ _______________ ______________ __________________ ________________ ___________
2 -0.0003563 0.019415 0.00015465 -0.019771 9.6731 10.917 8.0121 0
4 -3.1247e-05 -2.167e-05 6.87e-07 -9.577e-06 3.5113 7.8145 6.4745 0
8 -1.9877e-06 -1.9506e-06 8.1587e-09 -3.7097e-08 3.9745 6.3958 6.1256 -895.93
16 -1.2462e-07 -1.2421e-07 1.188e-10 -4.1719e-10 3.9955 6.1017 6.0346 11.109
32 -7.7946e-09 -7.7886e-09 1.8234e-12 -5.9752e-12 3.999 6.0258 5.9808 15.705
64 -4.8725e-10 -4.8715e-10 2.8533e-14 -9.1149e-14 3.9997 5.9979 Inf 15.947
128 -3.0454e-11 -3.0453e-11 4.4409e-16 -1.4433e-15 3.9999 6.0056 -Inf 15.988
256 -1.9035e-12 -1.9034e-12 -1.1102e-16 0 3.9999 2 -Inf 15.997
Exercises 2, 3 and 9, page 200. Compute the integral∫π
0ex cos(4x)dx =
eπ −117
using the (composite) midpoint, trapezoidal, Simpson rule. Find the errors and the ratios by which the errors de-creases (rates of convergence).Solution:% Exercise 5.1.2, 3 and 9 (a):
% use the composite-trapezoidal rule,
% Simpson rule and the midpoint rule
% to approximate
% I = int_0^pi exp(x)*cos(4*x) dx ==*== (exp(pi)-1)/17
% with n=2^j, j=1:9
% Write the values of the errors and the ratio:
% (I-In)/(I-I2n)
clear all
75
101
102
103
n
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
101
err
ors
Trapezoidal, Simpson and midpoint for 0 ex cos (4x) dx
Trapezoidal
Simpson
midpoint
FIG. 12.2. Log-log plot of errors for the midpoint, trapezoidal and Simpson rules.
% Use the function "trapezoidal(a,b,n0,index_f)" to compute the integral
n0=2;
a=0;b=pi;
[integralT,differenceT,ratioT]=trapezoidal(a,b,n0,12);
% Evaluate errors trapezoidal
ellT=length(integralT);
errorsTrapezoidal = (exp(pi)-1)/17 - integralT;
ratioTexact(2:ellT)=errorsTrapezoidal(1:ellT-1)./errorsTrapezoidal(2:ellT);
ratioT;
orderTexact=log(abs(ratioTexact(2:ellT)))/log(2);
format long
n = n0*2.^linspace(0,ellT-1,ellT);
H=((b-a)./n)’;
[integralS,differenceS,ratioS]=simpson(a,b,n0,12);
% Evaluate errors Simpson
ellS=length(integralS);
errorsSimpson = (exp(pi)-1)/17 - integralS;
ratioSexact(2:ellS)=errorsSimpson(1:ellS-1)./errorsSimpson(2:ellS);
ratioS;
orderSexact=log(abs(ratioSexact(2:ellS)))/log(2);
76
[integralM,differenceM,ratioM]=midpoint(a,b,n0,12);
% Evaluate errors Midpoint
ellM=length(integralM);
errorsmidpoint = (exp(pi)-1)/17 - integralM;
ratioMexact(2:ellM)=errorsmidpoint(1:ellM-1)./errorsmidpoint(2:ellM);
ratioM;
orderMexact=log(abs(ratioMexact(2:ellM)))/log(2);
format short
fprintf(’Trapezoidal calculations for int_0^pi e^x cos(4x) dx: \n’)
Tabular = table(n(2:end)’,integralT(2:end),errorsTrapezoidal(2:end),...
orderTexact’,’VariableNames’,{’n’ ’Trapezoidal’ ’Error in trapezoidal’ ’OrderConvExact’})
Tabular = table(n(2:end)’,integralS(2:end),errorsSimpson(2:end),...
orderSexact’,’VariableNames’,{’n’ ’Simpson’ ’Error in Simpson’ ’OrderConvExact’})
fprintf(’midpoint calculations for int_0^pi e^x cos(4x) dx: \n’)
Tabular = table(n(2:end)’,integralM(2:end),errorsmidpoint(2:end),...
orderMexact’,’VariableNames’,{’n’ ’Midpoint’ ’Error in midpoint’ ’OrderConvExact’})
figure(1)
%hold all
loglog(n(2:end),abs(errorsTrapezoidal(2:end)),’-sr’,n(2:end),abs(errorsSimpson(2:end)),’-*b’,...
n(2:end),abs(errorsmidpoint(2:end)),’-og’);
grid on
xlabel(’n’)
ylabel(’errors’)
legend(’Trapezoidal’,’Simpson’,’midpoint’,’location’,’best’)
title(’Trapezoidal, Simpson and midpoint for \int_0^\pi e^x cos (4x) dx’)
>> Exercise_5_1_2and3and9_a
Trapezoidal calculations for int_0^pi e^x cos(4x) dx:
Tabular =
84 table
n Trapezoidal Error in trapezoidal OrderConvExact
___ ___________ ____________________ ______________
4 3.2491 -1.9467 3.6952
8 1.6245 -0.32213 2.5953
16 1.3757 -0.073329 2.1352
32 1.3203 -0.017918 2.033
64 1.3068 -0.0044542 2.0082
128 1.3035 -0.001112 2.002
256 1.3027 -0.0002779 2.0005
77
512 1.3025 -6.9468e-05 2.0001
Tabular =
84 table
n Simpson Error in Simpson OrderConvExact
___ _______ ________________ ______________
4 -4.5067 5.8091 1.8821
8 1.083 0.21938 4.7268
16 1.2928 0.0096054 4.5134
32 1.3018 0.00055202 4.1211
64 1.3024 3.3796e-05 4.0298
128 1.3024 2.1015e-06 4.0074
256 1.3024 1.3117e-07 4.0019
512 1.3024 8.1956e-09 4.0005
midpoint calculations for int_0^pi e^x cos(4x) dx:
Tabular =
84 table
n Midpoint Error in midpoint OrderConvExact
___ ___________ _________________ ______________
4 -3.9445e-15 1.3024 4.033
8 1.1269 0.17547 2.8918
16 1.2649 0.037492 2.2266
32 1.2934 0.0090098 2.057
64 1.3002 0.0022303 2.0143
128 1.3018 0.00055618 2.0036
256 1.3023 0.00013896 2.0009
512 1.3024 3.4735e-05 2.0002
Exercise 11 (a), page 202. Show that the midpoint rule
I0( f ) = (b−a) f(a+b
2
)for approximating I( f ) =
∫ ba f (x)dx has the degree of precision (exactness) 1.
(Hint: Consider only I( f ) =∫ b
0 f (x)dx for suitable b and f (x) = 1,x,x2, · · · )Solution:
I( f ) :=∫ b
0dx = b, I0( f ) := b ·1 ( f (x) = 1 : r = 0)
I( f ) :=∫ b
0xdx =
b2
2, I0( f ) := b · b
2. ( f (x) = x : r = 1)
78
I( f ) :=∫ b
0x2dx =
b3
3, I0( f ) := b ·
(b2
)2. ( f (x) = x2 :���r = 2)
Since for f (x) = x2 the integral and the quadrature do not coincide, we conclude that the midpoint rule has D.O.E.=1.
Exercise 12, page 203. Determine the degree of precision of the approximation∫ 1
0f (x)dx≈ 1
4f (0)+
34
f(2
3
).
Solution:
I( f ) :=∫ 1
0dx = 1, I0( f ) :=
14·1+ 3
4·1≡ 1 ( f (x) = 1 : r = 0)
I( f ) :=∫ 1
0xdx =
12, I0( f ) :=
14·0+ 3
4· 2
3≡ 1
2. ( f (x) = x : r = 1)
I( f ) :=∫ 1
0x2dx =
13, I0( f ) :=
14·02 +
34·(2
3)2 ≡ 1
3. ( f (x) = x2 : r = 2)
I( f ) :=∫ 1
0x3dx =
14, I0( f ) :=
14·03 +
34·(2
3)3 ≡ 2
9. ( f (x) = x3 :���r = 3)
Since for f (x)= x3 the integral and the quadrature do not coincide, we conclude that the midpoint rule has D.O.E. =2.
Exercise 13, page 203. The degree of precision of a quadrature rule is defined as follows: If the formula haszero error when integrating any polynomial of degree ≤ r, and if the error is nonzero for some polynomial of degreer+1, then we say the formula has degree of precision equal to r.Let
Ih =3h4[ f (0)+3 f (2h)] .
What is the degree of precision of the approximation Ih ≈∫ 3h
0 f (x)dx?(Hint: Consider f (x) = 1,x,x2,x3,etc.)Solution:
• f = 1→∫ 3h
0 1dx = 3h≈ 3h4 (1+3) = 3h.
• f = x→∫ 3h
0 xdx = x2
2
∣∣∣3h
0= 9h2
2 ≈3h4 (0+3(2h)) = 3h = 9h2
2 .
• f = x2→∫ 3h
0 x2dx = x3
3
∣∣∣3h
0= 33h3
3 = 32h3 ≈ 3h4
(02 +3(2h)2
)= 9h3.
• f = x3→∫ 3h
0 x3dx = x4
4
∣∣∣3h
0= 34h4
4 6=3h4
(03 +3(2h)3
)= 36h4.
The degree of precision is 2.
Exercise 14. Approximate I( f ) =∫ 1−1 f (x)dx by replacing f (x) with P1(x), the linear interpolant to f (x) at
nodes x0 = − 13 and x1 = 1
3 . Give the resulting numerical integration formula. What is the degree of precision(exactness)?Solution: Using the Lagrange form of the interpolating polynomial, the interpolant is
P1(x) = f (x0)x− x1
x0− x1+ f (x1)
x− x0
x1− x0= f(− 1
3
) x− 13
− 13 −
13
+ f(1
3
) x+ 13
13 +
13
79
=−32
f(− 1
3
)(x− 1
3
)+
32
f(1
3
)(x+
13
),
and the quadrature rule is
Ih(x) =−32
f(− 1
3
)∫ 1
−1
(x− 1
3
)dx+
32
f(1
3
)∫ 1
−1
(x+
13
)dx
=−34
f(− 1
3
)(x− 1
3
)2 ∣∣∣1−1
+34
f(1
3
)(x+
13
)2∣∣∣1−1
=−34
f(− 1
3
)(49− 16
9
)+
34
f(1
3
)(169− 4
9
)= f(− 1
3
)+ f(1
3
).
To compute the degree of exactness we evaluate
I( f ) :=∫ 1
−1dx = 2, Ih( f ) := 1+1≡ 2 ( f (x) = 1 : r = 0)
I( f ) :=∫ 1
−1xdx = 0, Ih( f ) :=−1
3+
13≡ 0 ( f (x) = x : r = 1)
I( f ) :=∫ 1
−1x2dx =
23, Ih( f ) :=
(− 1
3
)2+(1
3
)2≡ 2
9. ( f (x) = x2 :���r = 2)
Since for f (x) = x2 the integral and the quadrature do not coincide, we conclude that the quadrature rule has D.O.E.=1.
Exercise 15, page 203. Consider the approximation
I( f ) =∫ 1
−1f (x)dx≈ f (−β )+ f (β )
for some β satisfying 0 < β ≤ 1. Show it has degree of precision greater than or equal to 1 for any such choice of β .Choose β to obtain a formula with degree of precision greater than 1. What is the degree of precision of thisformula?Solution: To compute the degree of exactness we evaluate
I( f ) :=∫ 1
−1dx = 2, Ih( f ) := 1+1≡ 2 ( f (x) = 1 : r = 0)
I( f ) :=∫ 1
−1xdx = 0, Ih( f ) :=−β +β ≡ 0 ( f (x) = x : r = 1)
I( f ) :=∫ 1
−1x2dx =
23, Ih( f ) := (−β )2 +(β )2 ≡ 2β
2. ( f (x) = x2 : r = 2)
I( f ) :=∫ 1
−1x3dx = 0, Ih( f ) := (−β )3 +(β )3 ≡ 0. ( f (x) = x3 : r = 3)
I( f ) :=∫ 1
−1x4dx =
25, Ih( f ) := (−β )4 +(β )4 ≡ 2β
4. ( f (x) = x4 :���r = 4)
For β =√
13 , the D.O.E. = 3, otherwise 1.
Exercise 16, page 203. Approximate I( f ) =∫ 2h
0 f (x)dx by replacing f (x) with P1(x), the linear interpolant tof (x) at x0 = 0 and x1 = h.
80
Give the resulting numerical integration formula. What is the degree of precision (exactness)?Solution: Using the Lagrange form of the interpolating polynomial, the interpolant is
P1(x) = f (x0)x− x1
x0− x1+ f (x1)
x− x0
x1− x0= f (0)
x−h−h
+ f (h)xh,
and the quadrature rule is
Ih(x) = f (0)∫ 2h
0
x−h−h
dx+ f (h)∫ 2h
0
xh
dx =− f (0)2h
(x−h)2∣∣2h0 dx+
f (h)2h
x2∣∣2h0
= 2h f (h).
To compute the degree of exactness we evaluate
I( f ) :=∫ 2h
0dx = 2h, Ih( f ) := 2h ·1 ( f (x) = 1 : r = 0)
I( f ) :=∫ 2h
0xdx = 2h2, Ih( f ) := 2h ·h ( f (x) = x : r = 1)
I( f ) :=∫ 2h
0x2dx =
8h3
3, Ih( f ) := 2h ·h2. ( f (x) = x2 :���r = 2)
Since for f (x) = x2 the integral and the quadrature do not coincide, we conclude that the quadrature rule has D.O.E.=1.
Exercise 1, page 215. Using the (composite trapezoidal error) formula , bound the error in the trapezoidal ruleI1,m applied to the following integrals ∫
π/2
0cosxdx, (a)∫ 1
0e−x2
dx, (b)∫ √π
0cos(x2)dx. (c)
Solution: (b) To evaluate the error in the composite trapezoidal rule for∫ 1
0e−x2
dx (b)
we compute f (x) = e−x2, f ′(x) =−2xe−x2
, f ′′(x) = (4x2−2)e−x2, and bound
maxx∈[0,1]
|(4x2−2)e−x2 |= 2.
Therefore
|E1,m( f )|=∣∣∣− b−a
12H2 f ′′(ξ )
∣∣∣≤ 16
H2,
where H = 1m , and m is the number of subintervals.
(c) To evaluate the error in the composite trapezoidal rule for∫ √π
0cos(x2)dx (c)
81
we compute f (x) =−2xsin(x2), f ′(x) =−2xe−x2, f ′′(x) =−2sin(x2)−4x2 cos(x2), and bound
maxx∈[0,
√π]|2sin(x2)+4x2 cos(x2)|= |2sin(π)+4π cos(π)|= 4π.
Therefore
|E1,m( f )|=∣∣∣− b−a
12H2 f ′′(ξ )
∣∣∣≤ √π
12H24π =
π√
π
3H2,
where H = 1m , and m is the number of subintervals.
Exercise 5, page 215. Using the asymptotic formula (asymptotic trapezoidal error formula) for the trapezoidalrule, estimate the number of m subdivisions to evaluate the following integrals to the given accuracy ε:
(a) I =∫ 3
1ln(x)dx, ε = 10−8
(b) I =∫ 2
1
ex− e−x
2dx, ε = 10−10
Solution:(a) By the (asymptotic trapezoidal error formula) we have that
E1,m( f )≈−H2
12(
f ′(b)− f ′(a))≡−H2
12(1
b− 1
a
)=
4m2
118≤ 10−8 ( f ′(x) = 1
x ,H = 2m )
m2 ≥ 29∗108, m≥ 4715.
Exercise 6, page 215. Using the asymptotic formula (asymptotic Simpson error formula) for the Simpson rule,estimate the number of m subdivisions to evaluate the following integrals to the given accuracy ε:
(a) I =∫ 3
1ln(x)dx, ε = 10−8
(b) I =∫ 2
1
ex− e−x
2dx, ε = 10−10
Solution:(a) By the (asymptotic Simpson error formula) we have that
E2,m( f )≈− 1180
(H2
)4(f (3)(b)− f (3)(a)
)≡− 1
180
(H2
)4( 2b3 −
2a3
)=− 1
1801
m4−2 ·26
27( f ′′′(x) = 2
x3 ,H = 2m )
=13
12151
m4≤ 10−8
m4 ≥ 131215
∗108, m≥ 33.
Exercise 8, page 216.(a) Consider using the trapezoidal rule to estimate the integral
I =∫ 1
−1
dx2+ x
≡ ln(3)≈ 1.098612288668110.
Give both a rigorous error bound found for I− I1,m and an asymptotic error estimate I− I1,m . Using the rigorouserror bound, determine how large m should be in order that |I− I1,m| ≤ 5×10−8.
(b) Repeat with Simpson’s rule.82
Solution:(a) The rigorous error bound (composite trapezoidal error) for the trapezoidal rule gives
E1,m =−b−a12
H2 f ′′(ξ )≡− 212
4m2
2(2+ξ )3 , (H = 2
m , f ′′(x) = 2(2+x)3 )
|E1,m| ≤43
1m2≤5×10−8, m > 5163.
The (asymptotic trapezoidal error formula) gives
E1,m( f )≈−H2
12(
f ′(b)− f ′(a)).≡− 1
124
m289=− 8
271
m2 , (H = 2m , f ′(x) =− 1
(2+x)2 )
|E1,m|=827
1m2≤5×10−8, m > 2434.
Indeed, using the composite trapezoidal rule with m = 2436 intervals the error is
|E1,2436|= |−0.000000049931155| ≈ 5×10−8.
(b) The rigorous error bound (composite Simpson error) for the Simpson rule gives
E2,m( f ) =−b−a180
(H2
)4f (4)(ξ ) =− 2
1801
m424
(2+ξ )5 , ( f (4)(x) = 24(2+x)5 ,H = 2
m )∣∣E2,m( f )∣∣= 4
151
m4≤ 5×10−8
m4 ≥ 475∗108, m≥ 49.
By the (asymptotic Simpson error formula) we have that
E2,m( f )≈− 1180
(H2
)4(f (3)(b)− f (3)(a)
)( f ′′′(x) =− 6
(2+x)4 ,H = 2m )
≡− 1180
(H2
)4( 2b3 −
2a3
)=
1180
1m4
16027
=8
2431
m4≤ 5×10−8
m4 ≥ 81215
∗108, m≥ 29.
Indeed, using the composite Simpson rule with m = 29 intervals the error is
|E2,29|= |−4.641445316977411e−08| ≈ 5×10−8.
Exercise 13, page 217.(a) From (error form)
I− Im ≈c
mp ,
derive
I− Im
I− I2m≈ 2p
for all m for which the approximation is valid.(Hint: Consider (error form) for m and 2m.)
83
(b) Also derive the computable estimate
I2m− Im
I4m− I2m≈ 2p.
This gives a practical means of checking the value of the value of p (order of convergence), using three consec-utive values Im, I2m, I4m. Using the log function, we get
p = log
(I2m− Im
I4m− I2m
)/log(2). (12.1)
(Hint: Write I2m− In = (I− Im)− (I− I2m), and do the same for the denominator.)Solution:
Exercise 16.(a) Following is a table of numerical integrals Im for an integral whose true value is I = 0.3. Assuming that the error
has an asymptotic formula of the form
I− Im ≈c
mp ,
for some p > 0 and c, estimate the order of convergence p.Estimate c. Estimate the size of m in order to have I− Im ≤ 10−10.
m Im8 0.2993331765
16 0.299789913932 0.299933823964 0.2999791556
128 0.2999934344256 0.2999979320
(b) Assuming I is not known (as is usually the case), estimate p.Solution:
(a)
m Im I− ImI−ImI−I2m
I2m− ImI2m−ImI4m−I2m
8 0.2993331765 0.6668 e-0316 0.2997899139 0.2101 e-03 3.1740 4.5674e-0432 0.2999338239 0.0662 e-03 3.1747 1.4391e-04 3.173864 0.2999791556 0.0208 e-03 3.1748 4.5332e-05 3.1746
128 0.2999934344 0.0066 e-03 3.1748 1.4279e-05 3.1747256 0.2999979320 0.0021 e-03 3.1749 4.4976e-06 3.1748
We have that p is approximately
p≈ log(3.1749)/ log(2) = 1.6667,
and
c≈ mp(I− Im)≡ 2561.6667(0.3− I256) = 2561.6667(0.3−0.2999979320) = 0.0213. (m = 256)
Then for the error to be less than 10−10
I− Im ≈c
mp ≡0.0213m1.6667 ≤ 10−10, m≥ 99295.
84
(b) Using the approximation of the ratios we have that p is approximately
p≈ log(3.1748)/ log(2) = 1.6667.
Exercise 19, page 218. In the following table of numerical integrals and their differences, give the likely valueof p if we assume the errors behave like I− Im ≈ c/mp. Also, estimate the error in I64.
m Im Im− Im/22 0.7028773964 0.781978959 0.079108 0.804500932 0.02252
16 0.810303086 0.00580232 0.811764354 0.00146164 0.812130341 0.0003660
(Hint: Use (12.1) to evaluate p and (Richardson’s error estimate in I2m) to estimate the error in I64) .Solution:
p≈ log(0.001461/0.0003660)/ log(2) = 1.9970,
I− I64 ≈1
21.9970−10.0003660 = 1.2234e−04.
We shall solve using the information for I16 and I32.
p≈ log(0.005802/0.001461)/ log(2) = 1.989594117394499≈ 1.9895, (order of convergence)
I− I32 ≈1
21.989594117394499−10.001461≈ 4.9171e−04. (Richardson’s estimated error in I32)
85
13. Section 5.3 GAUSSIAN NUMERICAL INTEGRATION.The (orthogonal polynomials) play an important role in devising quadrature formulae with maximal degree of exact-ness.
Let x0, · · · ,xn be n+1 distinct point in [−1,1]. To approximate the weighted integral
Iw( f ) =∫ 1
−1f (x)w(x)dx
where f ∈C[−1,1], we consider quadrature rules of the following type
In,w( f ) = α0 f (x0)+ · · ·+αn f (xn), (Gaussian quadrature formula)
where αi are to be suitably determined, in order to maximize the degree of exactness (w.r.t. weight w(x)).THEOREM 13.1. The (Gaussian quadrature formula) has degree of exactness n+m if and only if is of interpo-
latory type and the (nodal polynomial) ωn+1(x) is orthogonal on all polynomials of degree m−1:∫ 1
−1ωn+1(x)p(x)w(x)dx = 0, ∀p ∈ Pm−1. (13.1)
COROLLARY 13.2. The maximum degree of exactness of the (Gaussian quadrature formula) is 2n+1.Proof. By contradiction, assume m≥ n+2 (i.e. the D.O.E. > 2n+2) and therefore we could choose p = ωn+1
in (13.1), which yields to the absurd result ωn+1 ≡ 0.Setting m = n+1 gives the result.
13.1. Gauss-Legendre quadrature. We shall use the orthogonal (Legendre polynomials) (the weight func-tion w(x) = 1) to approximate the (Gaussian quadrature formula). The quadrature nodes x j are then the zeros of(Legendre polynomials) Ln+1 (see also the examples (11.7)) , and the quadrature weights are
α j =2
(1− x j)∣∣Ln+1′(x)
∣∣2 , j = 0, · · · ,n. (Gaussian quadrature weights)
REMARK 34 (Integration over an arbitrary interval). The integral
I( f ) :=∫ b
af (x)dx =
b−a2
∫ 1
−1f(
b−a2
y+a+b
2
)dy
has the following Gaussian quadrature formula
In( f ) =b−a
2α0 f
(b−a
2x0 +
a+b2
)+ · · ·+ b−a
2αi f(
b−a2
xi +a+b
2
)+ · · ·+ b−a
2αn f
(b−a
2xn +
a+b2
).
(Gauss-Legendre quadrature)
Exercise 2. (d) Apply I2, I3 to the integral
∫π
−π
ecosxdx≈ 7.95492652101284.
86
Calculate the errors and compare them with the earlier results for the trapezoidal and Simpson rules.Solution: First, using formula (5.62) for an arbitrary interval we have
I( f ) =∫ b
af (x)dx≡
∫π
−π
ecosxdx=b−a
2
∫ 1
−1f(b+a+ t(b−a)
2
)dt ≡ π
∫ 1
−1f (πt)dt.
Secondly, we give the value computed with 20 Gaussian nodes
I20 = 7.954926520986633,E20 = 7.95492652101284−7.954926520986633 = 2.620748063009160e−11. (Gaussian error)
We report also the errors using 20 nodes for the midpoint, trapezoidal and Simpson errors:
I( f )− I0,20( f ) = 7.95492652101284−7.954926521012845 =−4.440892098500626e−15, (midpoint error)I( f )− I1,20( f ) = 7.95492652101284−7.954926521012844 =−3.552713678800501e−15, (trapezoidal error)I( f )− I2,20( f ) = 7.95492652101284−7.954926519859693 = 1.153147799470844e−09 (Simpson error)
We note that the midpoint and trapezoidal rules are ”super-convergent” (f(x) is periodic on [a,b])!!!
>> midpoint(-pi,pi,20,11)
ans = 7.954926521012845
>> trapezoidal(-pi,pi,20,11)
ans = 7.954926521012844
>> simpson(-pi,pi,20,11)
ans = 7.954926519859693
Now we apply formulae (5.50) or (5.53), with the values for the nodes and weights from Table 5.7, to obtain
I2( f ) = π(w1 f (π ∗ x1)+w2 f (π ∗ x2)
)≡ π
(ecos(−0.577350269189626·π)+ ecos(−0.577350269189626·π))
= 4.939472538370615,
[val,bp,wf]=gaussint(-pi,pi,2,11)
val =
4.939472538370615
bp =
-0.577350269189626
0.577350269189626
wf =
1.000000000000000
1.000000000000000
and respectively
I3( f ) = π(w1 f (π ∗ x1)+w2 f (π ∗ x2)+w3 f (π ∗ x3)
)87
≡ π(0.555555555555556 · ecos(−0.774596669241484·π)+0.888888888888889 · ecos(0∗π)
+0.555555555555556 · ecos0.774596669241484·π)= 9.224020450374859.
>> [val,bp,wf]=gaussint(-pi,pi,3,11)
val =
9.224020450374855
bp =
-0.774596669241484
0
0.774596669241483
wf =
0.555555555555556
0.888888888888889
0.555555555555556
Exercise 10, page 230.(b) To find a formula
I( f ) :=∫ 1
0f (x) log
(1x
)dx≈ w1 f (x1)+w2 f (x2) =: I2( f )
which is exact for all polynomials of degree ≤ 3, set up a system of four equations with unknowns w1,w2,x1,x2. Donot attempt to solve the system; instead show that
x1 =15−
√106
42, x2 =
15+√
10642
,
w1 =21√106
(x2−
14
), w2 = 1−w2
is a solution of the system.Solution: Taking f (x) = 1,x,x2,x3 and enforcing I( f ) = I2( f ) we obtain∫ 1
0log(1
x
)dx =1 = w1 +w2, ( f = 1)∫ 1
0x log
(1x
)dx =
14= w1x1 +w2x2, ( f = x)∫ 1
0x2 log
(1x
)dx =
19= w1x2
1 +w2x22, ( f = x2)∫ 1
0x3 log
(1x
)dx =
116
= w1x31 +w2x3
2, ( f = x3)
which is easy to check that hold for the given values.88
14. Section 5.4 NUMERICAL DIFFERENTIATION. APPROXIMATION OF FUNCTION DERIVATIVES.
The main goal of this section is to approximate the derivative of a given function f at the n+ 1 distinct nodes{xk,k = 0, · · ·n} in [a,b], namely
x0 = a,xn = b,
xk+1 = xk +h, k = 0, · · ·n−1, h =b−a
n.
The efficient way of computing/approximating the values of f ′(xi)≈ ui using the nodal values of f (xk) is to usethe following compact formula
hk=m
∑k=−m
αkui−k =k=m′
∑k=−m′
βk f (xi−k), (compact finite differences)
where {αk},{βk} ∈ R are 2(m+m′+ 1) coefficients, to be determined, and uk are the desired approximations off ′(xk).
DEFINITION 14.1 (stencil). The set of nodes involved in constructing the derivative of f at a certain node iscalled stencil.
14.1. Classical Finite Differences method.
The simplest way to generate a formula like (compact finite differences) is by using the definition of the deriva-tive
f ′(xi) = limh→0+
f (xi +h)− f (xi)
h.
(I) (forward finite difference) Replacing the limit with the incremental ratio, with h finite, gives
uFDi =
f (xi +h)− f (xi)
h, i = 0, · · · ,n−1. (forward Finite Difference)
REMARK 35.(a) Note that (forward Finite Difference) is a special instance of (compact finite differences), with m =
0,α0 = 1,m′ = 1,β−1 = 1,β0 =−1,β1 = 0.(b) The approximation uFD
i to f ′(xi) given by (forward Finite Difference) is the slope of the line passingthrough the points
(xi, f (xi)
)and
(xi+1, f (xi+1)
).
(c) Assuming that f ∈C2[a,b], and using Taylor’s expansion
f (xi+1) = f (xi)+h f ′(xi)+h2
2f ′′(ξi), with ξi ∈ (xi,xi+1),
we obtain that the error made by the approximation is
f ′(xi)−uFDi =−h
2f ′′(ξi). (14.1)
(II) (centered finite difference) Using a centered incremental ratio, with h finite, gives
uCDi =
f (xi +h)− f (xi−h)2h
, i = 1, · · · ,n−1. (centered Finite Difference)
REMARK 36.89
(a) Note that (centered Finite Difference) is also a special instance of (compact finite differences), with m =0,α0 = 1,m′ = 1,β−1 = 1/2,β0 = 0,β1 =−1/2.
(b) The approximation uCDi to f ′(xi) given by (forward Finite Difference) is the slope of the line passing
through the points(xi−1, f (xi−1)
)and
(xi+1, f (xi+1)
).
(c) Assuming that f ∈C2[a,b], and using Taylor’s expansion
f (xi+1) = f (xi)+h f ′(xi)+h2
2f ′′(ξi)+
h3
6f ′′′(ξ+
i ), with ξi ∈ (xi,xi+1),
f (xi−1) = f (xi)−h f ′(xi)+h2
2f ′′(ξi)−
h3
6f ′′′(ξ−i ), with ξi ∈ (xi−1,xi),
we obtain that the error made by the approximation is
f ′(xi)−uCDi =−h2
6f ′′′(ξi), ξi ∈ (xi−1,xi+1). (14.2)
(III) (backward finite difference) Replacing the limit with the incremental ratio, with h finite, gives
uBDi =
f (xi)− f (xi−h)h
, i = 1, · · · ,n. (backward Finite Difference)
REMARK 37.(a) Note that (backward Finite Difference) is also special instance of (compact finite differences).(b) Assuming that f ∈C2[a,b], and using Taylor’s expansion we obtain that the error made by the approxi-
mation is
f ′(xi)−uBDi =
h2
f ′′(ξi). (14.3)
14.2. Differentiation using interpolation.
Since we can use interpolation to approximate f (x) by polynomials defined using nodal values of f (xi), i.e.,f (x)≈ (Πn f )(x), it is only natural to ask if we could not approximate f ′(x)≈ (Πn f )(x)?
THEOREM 14.2. Assume f ∈ Cn+2[a,b], and x0,x1,xn are n + 1 distinct interpolatory nodes in [a,b], andx ∈ [a,b] is an arbitrary value. Then the error in approximation f ′(x)≈ (Πn f )(x) is
f ′(x)− (Πn f )′(x) = ωn+1(x)f (n+2)(c1)
(n+1)!+ω
′n+1(x)
f (n+1)(c2)
(n+1)!,
where c1,c2 are arbitrary points in the interval determined by the nodes {x0, · · · ,xn}.Proof. Recall the (interpolation error / Newton form) formula
En(x) := f (x)− (Πn f )(x) =f (n+1)(cx)
(n+1)!ωn+1(x).
Then the result yields after differentiation.
14.3. The Method of Undetermined Coefficients. Higher-order derivatives.
DEFINITION 14.3 (Central finite difference formula).Assume f ∈C4[a,b]. Then the approximate of f ′′(xi) denoted by u′′i or D2 f (xi) is
u′′i =f (xi+1)−2 f (xi)+ f (xi−1)
h2 , ∀i = 1, · · ·n−1, (central finite difference formula)
90
with the error
f ′′(xi)−u′′i =− h2
24(
f (4)(xi +θih)+ f (4)(xi−ωih))≈− h2
12f (4)(xi +ξih), (14.4)
where θi,ωi ∈ [0,1).The error formula (14.4) is a consequence of Taylor expansions
f (xi+1) = f (xi)+h f ′(xi)+h2
2f ′′(xi)+
h3
6f ′′′(xi)+
h4
24f (4)(ξi +θih),
f (xi−1) = f (xi)−h f ′(xi)+h2
2f ′′(xi)−
h3
6f ′′′(xi)+
h4
24f (4)(ξi−ωih),
−2 f (xi) =−2 f (xi),
hence
u′′i = f ′′(xi)+h2
4!(
f (4)(ξi +θih)+h4
24f (4)(ξi−ωih)
),
and finally using the Mean Value Theorem we obtain (14.4).Let us now derive (central finite difference formula) by a more general technique.We start by asking the question of finding an approximation to f ′′(x) by a linear combination of nodal values
f (x+h), f (x), f (x−h), with the smallest error term:
f ′′(x)≈ A f (x+h)+B f (x)+C f (x−h).
Therefore we seek the undetermined coefficients A,B,C so that the error made by the approximation is smallest.Proceeding as above, we write the Taylor expansions
f (x+h) = f (x)+h f ′(x)+h2
2f ′′(xi)+
h3
6f ′′′(x)+
h4
24f (4)(x)+O(h5), (·A)
f (x) = f (x), (·B)
f (x−h) = f (x)−h f ′(x)+h2
2f ′′(x)− h3
6f ′′′(x)+
h4
24f (4)(x)+O(h5), (·C)
A f (x+h)+B f (x)+C f (x−h) = (A+B+C) f (x) (≡ 0 · f (x))+h(A−C) f ′(x) (≡ 1 · f ′(x))
+h2
2(A+C) f ′′(x) (≡ 0 · f ′′(x))
+h3
6(A−C) f ′′′(x)
+h4
24(A+C) f (4)(x)+O(h5).
Imposing
A+B+C = 0, h(A−C) = 0,h2
2(A+C) = 1,
we obtain
A =C =1h2 , B =− 2
h2 ,
which confirms the (central finite difference formula) and the error (14.4).91
14.4. Effects of error in function values. We shall look now at the effect of small (measurements or round-off)errors in the functions values into numerical differentiation. Let say that the true values f (xi) are approximated byf (xi), i.e., f (xi)≈ f (xi). Then using the (central finite difference formula) as an example, the approximation D2 f (xi)of the second derivative f ′′(xi), which uses instead of f (xi) the approximate values f (xi) = f (xi)+ εi would write
D2 f (xi) =1h2
(f (x+h)+ ε1−2 f (x)−2ε2 + f (x−h)+ ε3
)= D2 f (xi)+
ε1−2ε2 + ε3
h2 .
Therefore the true error would satisfy
∣∣ f ′′(x)−D2 f (x)∣∣≤ ∣∣ f ′′(x)− D2 f (x)
∣∣+ ∣∣D2 f (x)−D2 f (x)∣∣≤ h2
12| f (4)(x)|+ 4ε
h2 . (14.5)
This suggests that as h↘ 0, the last term in the right hand side will increase.An optimal h should then satisfy h2 ≈ ε
h2 .For ε ≈ 10−16 this means h = ε1/4 = 10−4.
Exercise 1,2,3 page 241. (b) Find the numerical (forward, backward and centered finite differences) derivativesD1 f (x) at x = 1 for f (x) = arctan(x2− x+ 1), using h = 0.1,0.05,0.025,0.0125,0.00625; calculate the error, theratio and estimate the error by using x = 1 instead of the intermediate point.Solution: First, the exact value for the derivative is
f ′(x) =2x−1
1+(x2− x+1)2 , f ′(1) =2x−1
1+(x2− x+1)2
∣∣∣∣x=1
=12,
f ′′(ξ )≈ f ′′(1) =12, f ′′′(ξ )≈ f ′′′(1) =−2.45.
Using the (forward Finite Difference) formula we obtain:
h D1 f Error Ratio Error estimate≈− 14 h Order
0.1 0.5209 −0.0209 - 0.02500.05 0.5115 −0.0115 1.8199 - 0.0125 0.8638540.025 0.5060 −0.0060 1.9132 -0.0063 0.9360250.0125 0.5031 −0.0031 1.9575 - 0.0031 0.9689920.00625 0.5015 −0.0016 1.9790 - 0.0016 0.984736
Using the (centered Finite Difference) formula we obtain:
h D1 f Error Ratio Error estimate≈ −2.456 h2 Order
0.1 0.495855700471995 0.004144299528005 0.0040833333333330.05 0.498959737494895 0.001040262505105 3.9839 0.001020833333333 1.994180.025 0.499739671191277 0.000260328808723 3.9960 0.000255208333333 1.998540.0125 0.499934901325987 0.000065098674013 3.9990 0.000063802083333 1.999630.00625 0.499983724301654 0.000016275698346 3.9997 0.000015950520833 1.99991
(SEE FIGURE 14.1 FOR THE EFFECT OF ROUNDOFF ERROR IN FUNCTION VALUES !!!
Exercise 4, page 241. Let h> 0,x j = x0+ jh for j = 0,1,2, · · · ,n, and let (Πn f )(x) be the degree n interpolatingpolynomial of f (x) at x0,x1, · · · ,xn. Use this polynomial to estimate f ′(x0). Produce the actual formulae involvingf (x0), f (x1), · · · , f (xn) for n = 1,2,3,4. Also produce their error formulae.
92
x0 f0
x1 f1f1− f0
hx2 f2
f2− f1h
f2−2 f1+ f02h2
x3 f3f3− f2
hf3−2 f2+ f1
2h2f3−3 f2+3 f1− f0
6h3
Solution: For n = 3, the Newton divided differences
give the interpolating polynomial
(Π4 f )(x) = f0 +f1− f0
h(x− x0)+
f2−2 f1 + f0
2h2 (x− x0)(x− x1)+f3−3 f2 +3 f1− f0
6h3 (x− x0)(x− x1)(x− x2)
hence
(Π4 f )′(x0) =f1− f0
h+
f2−2 f1 + f0
2h2 (x0− x1)+f3−3 f2 +3 f1− f0
6h3 (x0− x1)(x0− x2)
=1h
(13
f3−32
f2 +3 f1−116
f0
).
The error term is
14!
f (4)(ξ )(x− x0)(x− x1)(x− x2)(x− x3)︸ ︷︷ ︸≈−6h3
≈−h3
4f (4)(ξ ), ξ ∈ [x0,x3].
Exercise 9, page 242. Use the (central finite difference formula) formula to estimate f ′′(x) for the functionf (x) = arctan(x2− x+ 1), at x = 1, using h = 0.1,0.05,0.025,0.0125,0.00625,0.003125, · · · ; calculate the error,and the ratios by which they decrease.Solution: Using the (central finite difference formula) formula we obtain:
h D2 f Error Ratio Order0.1 0.4999958333958 0.00000416660420.05 0.4999997395836 0.0000002604164 15.9997 4.000010.025 0.4999999837240 0.0000000162760 16.0000 4.013560.0125 0.4999999989828 0.0000000010172 1.9132 7.113390.00625 0.4999999999370 0.0000000000630 1.9575 -6.953840.003125 0.4999999999995 0.0000000000005 1.9790 -0.505240.0015625 0.5000000000564 −0.0000000000564 1.9790 -2.540570.00078125 0.4999999999200 0.0000000000800 1.9790 0
(SEE FIGURE 14.1 FOR THE EFFECT OF ROUNDOFF ERROR IN FUNCTION VALUES !!!)
Exercise 6, page 241. The error formulae (14.1) for the (forward Finite Difference), (14.2) for the (centered Finite Difference),and (14.4) for the (central finite difference formula), are nearly proportional to a power of h. These justify the useof Richardson’s extrapolation as in Section 12.1.4 (see (Richardson extrapolation) for the quadrature rules). For the(forward Finite Difference), derive the extrapolation formula
f ′(x)≈ 2Dh f (x)−D2h f (x) (Richardson extrapolation of forward finite difference)
and show its error convergences to zero more rapidly than does the error for (forward Finite Difference).Derive corresponding error formulae for (centered Finite Difference) and (central finite difference formula), based
93
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
10-14
10-12
10-10
10-8
10-6
10-4
10-2
100
102
104
forward difference
central difference
second derivative
FIG. 14.1. Errors in numerical differentiation: the forward and central finite difference approximation of f ′, and the centered finitedifference approximation of f ′′, at x = 1, with f (x) = arctan(x2− x+ 1). Notice the effects of errors in function values for small values of themesh size h� 1.
on (14.2) and (14.4).Solution: Using the error formula (14.1) for the with (forward Finite Difference) h and 2h we have
f ′(x)−uFDh ≈−h
2f ′′(x)+O(h2),
f ′(x)−uFD2h ≈−
�2h
�2f ′′(x)+O(h2),
hence
f ′(x)≈ 2uFDh −uFD
2h +O(h2).
Similarly, using the error formula (14.2) for the with (centered Finite Difference) h and 2h we have
f ′(x)−uCDh ≈−
h2
6f ′′′(x)+O(h3),
f ′(x)−uCD2h ≈−
4h2
6f ′′′(x)+O(h3),
hence
f ′(x)≈ 13(4uCD
h −uCD2h)+O(h3).
Exercise 7 (or similar). Use the extrapolation formula (Richardson extrapolation of forward finite difference)to improve the answers given in table in Exercise 1,2,3.
94
Solution: The extrapolation formula gives the values in the table (notice that the extrapolated values are moreaccurate than the values given by the (forward Finite Difference) in Exercise 1,2,3):
h D1 f Error Order Extrapolated values Error in extrapolated Order Xtrp0.1 0.5209 −0.0209 0.502063969827183 -0.0020639698271830.05 0.5115 −0.0115 0.8638 0.500519610991171 -5.196109911710378e-04 1.98990.025 0.5060 −0.0060 0.9360 0.500130131651430 -1.301316514300543e-04 1.99740.0125 0.5031 −0.0031 0.9689 0.500032547283285 -3.254728328494494e-05 1.99930.00625 0.5015 −0.0016 0.9847 0.500008137720619 -8.137720619028066e-06 1.9998
Exercise 14, page 242. Using the following table of rounded values of f (x), estimate f ′′(0.5) numerically withstepsizes h = 0.2,0.1.
x f (x)0.3 7.38910.4 7.46330.5 7.53830.6 7.61410.7 7.6906
Also estimate the possible size of that part of the error in your answer that is due to the rounding errors in the tableentries.Is this a serious source of error in this case?Solution: Using h = 0.2 (and h = 0.1 respectively) and the (central finite difference formula) to approximate f ′′(0.5)we have
f ′′(0.5)≈ 7.3891−2 ·7.5383+7.69060.04
= 0.077500000000019, (h = 0.2)
≈ 7.4633−2 ·7.5383+7.61410.01
= 0.080000000000080. (h = 0.1)
The rounding error in the table (only four significant digits) is of order
ε = 0.00005,
hence using a reasoning similar to (14.5), the error due to rounding values is of the form
4ε
h2 ≈ 0.005 (h = 0.2)
≈ 0.02. (h = 0.1)
Comparing these error values with the estimated values of f ′′(0.5), we conclude that when using h = 0.2, the round-ing error has less significant influence than when using h = 0.1.
95
15. Section 3. ROOTFINDING FOR NONLINEAR EQUATIONS.
MOTIVATION 1. Let’s consider solving the following Initial Value Problem (IVP)
{u′(t) = F(u), t ∈ (0,T ],u(0) = u0
(IVP)
i.e., we are looking to find the solution u(t) : [0,T ]→R, a continuously differentiable function satisfying the equationand the initial condition. For a general right-hand side term, this could be very difficult, so let’s again settle with anapproximate solution, namely a sequence {un} of approximate values
un ≈ u(tn), tn = t0 +n∆t, ∆t =TN,n = 0, · · · ,N.
These values could be then computed iteratively. Using a numerical approximation of the derivative, for example the(backward Finite Difference), we can approximate (IVP) by the (backward Euler) formula:
un+1−un
∆t= F(un+1), n = 0, · · · ,N−1,
u(0) = u0.
(backward Euler)
We now point out that (for each n = 0, · · · ,N−1) un+1 is a root of the function
f (x) = 0,
where f (x) = x−∆t F(x)−un.
DEFINITION 15.1. Let f : I = [a,b]⊂R→R be a scalar function. The complex number α ∈C is called a rootor zero of f if f (α) = 0.
REMARK 38. The methods for the numerical approximation of a zero α of f are usually iterative, i.e.,(given an initial guess) generate a sequence of values {x(k)}k≥0 such that
limk→∞
x(k) = α. (15.1)
-6 -4 -2 0 2 4 6
-8
-6
-4
-2
0
2
4
6
8
FIG. 15.1. f (x) = 0.1− x+ sin(x)
DEFINITION 15.2 (Order of convergence).The sequence {x(k)} generated by a numerical method converges to α with order p≥ 1
96
• if there exists C > 0 such that
|x(k+1)−α||x(k)−α|p
≤C, ∀k ≥ k0, (where k0 is a suitable constant) (15.2)
• or
limk→∞
x(k+1)−α
(x(k)−α)p=C, (asymptotic error constant). (15.3)
REMARK 39. Unlike the case of linear systems, the convergence of iterative methods for rootfinding of nonlinearequations depends in general on the choice of (the initial ”guess”) x(0). In such a case we say that the method islocally convergent: x(0) ∈ V (α).
DEFINITION 15.3 (Global convergence).The iterative methods which converge for all x(0) are called globally convergent.
REMARK 40. If p = 1 (linear convergence), from (15.2) we see that in order to have convergence x(k)→ α , it isnecessary to have the convergence factor C < 1:
|x(k+1)−α| ≤C|x(k)−α| ≤C2|x(k−1)−α| ≤ · · · ≤Ck+1|x(0)−α| → 0 if and only if C < 1.
DEFINITION 15.4 (Efficiency factor). Consider an iterative method with the q≥ 1 order of convergence. If eachiteration requires m units of ”work” (work involved in computing a function value or the value of its derivatives),then
E := q1m (efficiency index)
is the efficiency index of the iteration.
REMARK 41.• The methods which converge linearly all have the efficiency index Elinear = 1. (Indeed, E = 1m.)• We shall see that Newton’s method has ENewton = 2
12 , as q= 2 and m= 2 (each iteration involves evaluations
of f and f ′).REMARK 42 (Errors in rootfinding). Let us consider the fourth-order polynomial
p4(x) = (x−1)4,
and its round-off error approximation
p4(x) = (x−1)4− ε,
where ε = 10−16. It is easy to see that the roots of each polynomials are
αi = 1, i = 1,2,3,4,
αi = 1± 4√
ε, 1± i 4√
ε, i = 1,2,3,4.
So, to an ε = 10−16 error in the function evaluation it corresponds a much (four orders of magnitude) larger relativeerror in the roots:
Relative error(α1) =1− (1− 4
√ε)
1= 4√
ε = 10−4.
97
15.1. Subsection 3.1 THE BISECTION METHOD. A GEOMETRIC APPROACH TO ROOTFINDING.
Let us recall first the Intermediate Value Theorem.THEOREM 15.5 (Intermediate Value Theorem).
Given f ∈C([a,b];R) such that f (a) · f (b)< 0, there exists α ∈ (a,b) such that f (α) = 0.
The theorem states that a continuous function which has opposite sign values on an interval, also has one rootin that interval; and this is the basic fact on which we can construct an iterative algorithm to get as close as possibleto the root:
I0 := [a,b]; generate a sequence Ik = [a(k),b(k)] such that Ik ⊂ Ik−1 with f (a(k)) · f (a(k))< 0.Result: Bisection Method
initialization: set a(0) = a, b(0) = b, x(0) =a(0)+b(0)
2;
for k ≥ 0 doa(k+1) = a(k), b(k+1) = x(k) if f (x(k)) · f (a(k))< 0 ;
a(k+1) = x(k), b(k+1) = b(k) otherwise ;
x(k+1) =a(k+1)+b(k+1)
2Terminate when tolerance is reached: |b(k)−a(k)|< ε or k ≥ max iterate.
endREMARK 43.• At step m we have that the error is
|α− x(m)|< |Im|2
• Speed of convergence:
|I0|= |b−a|, · · · |Ik|=b−a
2k
• The absolute error satisfies
|e(k)|= |α− x(k)| ≤ b−a2k+1
k↗∞−−−→ 0 (global convergence of bisection)
Hence the bisection method is globally convergent!!!
QUESTION 1.(1) How many iterates m are necessary to obtain tolerance ε?
Using (global convergence of bisection) we have that
e(m) = |α− x(m)| ≤ b−a2m+1≤ε,
b−aε≤ 2m+1,
log( b−a
ε
)log(2)
≤ m+1,
m≥−1+log( b−a
ε
)log(2)
. (15.4)
98
(2) How many iterates m are necessary to obtain one extra significant digit?Similarly
���b−a2k+1 ≈ |α− x(k)|= |α− x( j)|
10≈���b−a2 j+1
110
,
2k+�1− j−�1 ≈ 10,
k− j ≈ log(10)log(2)
≈ 3.3219. (≈ 3.3 bisection iterates for one significant digit)
REMARK 44.• (global convergence of bisection): The bisection method is globally convergent!!!• (≈ 3.3 bisection iterates for one significant digit) makes the bisection method slow!!!!
The fact that is slow but globally convergent makes the bisection method a perfect approaching techinique!The fzero function of MATLAB uses a combination of bisection, secant and inverse quadratic interpolation methods:fzero(inline(-4+x^2),1)
EXAMPLE 5 (COUNTEREXAMPLE - f DISCONTINUOUS). Let f (x) = 5+(3.5x+ 1) ·(
1+ 1(3.5x+1)2 +
1x2
),
which has discontinuities at x =− 13.5 ≈−0.285714285714286 and at x = 0,
-0.4 -0.35 -0.3 -0.25 -0.2 -0.15
-1.5
-1
-0.5
0
0.5
1
1.510
4
so the assumption of Intermediate Value Theorem 15.5 does not hold.Hence we should doubt the output of the bisection algorithm, which does not know of this fault!
Indeed, if we run it on [a,b]≡ [−5,5], with tolerance ε = 10−10 and max iterate = 100 we obtainbisect(-5,5,1.e-10,100,10)
root =
-0.285714285782888
and no indication of any errors!But if we check now if this α =−0.285714285782888 is indeed a root, by evaluating f (α) and expecting f (α) = 0,we are in for a big surprise:
f (α) =−4.1648e+09 6= 0!!!!
99
REMARK 45. The errors in the approximation of the root by the bisection method does not decay monotonically,and therefore is not of order 1:
|e(k+1)|� |e(k)|
|α− x(k+1)|� |α− x(k)|
EXAMPLE 6. For the case f (x) = x2−4,α = 2 we see that indeed the errors do not decay monotonically:
k x(k) α− x(k)
1 2.250000 -0.25002 1.125000 0.87503 1.687500 0.31254 1.968750 0.03125 2.109375 -0.10946 2.039062 -0.03917 2.003906 -0.00398 1.986328 0.01379 1.995117 0.0049
1 2 3 4 5 6 7 8 9
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
FIG. 15.2. The error in the bisection method does not decay monotonically.
function [root,error_bound,it_count]=bisect(a0,b0,ep,max_iterate,index_f)
>> bisect(0,4.5,1.e-2,100,12)
Iteration number: 1
Approximation of the root: 2.250000
Iteration number: 2
Approximation of the root: 1.125000
Iteration number: 3
100
Approximation of the root: 1.687500
Iteration number: 4
Approximation of the root: 1.968750
Iteration number: 5
Approximation of the root: 2.109375
Iteration number: 6
Approximation of the root: 2.039062
Iteration number: 7
Approximation of the root: 2.003906
Iteration number: 8
Approximation of the root: 1.986328
Iteration number: 9
Approximation of the root: 1.995117
root =
1.995117187500000
Exercise 1, page 77. Use the bisection method with a hand calculator or computer to find the indicated roots of thefollowing equations. Use a tolerance of ε = 10−4.(a) The real root of x3− x2− x−1 = 0, f1(x) = x3− x2− x−1.(b) The root of x = 1+0.3cosx, f2(x) := x−1−0.3cos(x).(c) The smallest positive root of cos(x) = 1
2 + sinx, f3(x) := cos(x)− 12 − sin(x).
Solution:(a) With tolerance 10−15 it takes 52 iterations to obtain the root = 1.839286755214161;
f1(1.839286755214161) =−4.4409e−16.With tolerance 10−4:bisect(0,3,1.e-4,100,16)
Iteration number: 1
Approximation of the root: 1.500000
The error estimate : 1.500000
Iteration number: 2
Approximation of the root: 2.250000
The error estimate : 0.750000
Iteration number: 3
Approximation of the root: 1.875000
The error estimate : 0.375000
101
Iteration number: 4
Approximation of the root: 1.687500
The error estimate : 0.187500
Iteration number: 5
Approximation of the root: 1.781250
The error estimate : 0.093750
Iteration number: 6
Approximation of the root: 1.828125
The error estimate : 0.046875
Iteration number: 7
Approximation of the root: 1.851562
The error estimate : 0.023438
Iteration number: 8
Approximation of the root: 1.839844
The error estimate : 0.011719
Iteration number: 9
Approximation of the root: 1.833984
The error estimate : 0.005859
Iteration number: 10
Approximation of the root: 1.836914
The error estimate : 0.002930
Iteration number: 11
Approximation of the root: 1.838379
The error estimate : 0.001465
Iteration number: 12
Approximation of the root: 1.839111
The error estimate : 0.000732
Iteration number: 13
Approximation of the root: 1.839478
The error estimate : 0.000366
Iteration number: 14
Approximation of the root: 1.839294
The error estimate : 0.000183
Iteration number: 15
Approximation of the root: 1.839203
The error estimate : 0.000092
(b) With tolerance 10−15 it takes 52 iterations to obtain the root = 1.128425092992224.f2(1.128425092992224) =−8.8818e−16.
102
With tolerance 10−4:>> bisect(0,3,1.e-4,300,17)
Iteration number: 1
Approximation of the root: 1.500000
The error estimate : 1.500000
Iteration number: 2
Approximation of the root: 0.750000
The error estimate : 0.750000
Iteration number: 3
Approximation of the root: 1.125000
The error estimate : 0.375000
Iteration number: 4
Approximation of the root: 1.312500
The error estimate : 0.187500
Iteration number: 5
Approximation of the root: 1.218750
The error estimate : 0.093750
Iteration number: 6
Approximation of the root: 1.171875
The error estimate : 0.046875
Iteration number: 7
Approximation of the root: 1.148438
The error estimate : 0.023438
Iteration number: 8
Approximation of the root: 1.136719
The error estimate : 0.011719
Iteration number: 9
Approximation of the root: 1.130859
The error estimate : 0.005859
Iteration number: 10
Approximation of the root: 1.127930
The error estimate : 0.002930
Iteration number: 11
Approximation of the root: 1.129395
The error estimate : 0.001465
Iteration number: 12
Approximation of the root: 1.128662
The error estimate : 0.000732
103
Iteration number: 13
Approximation of the root: 1.128296
The error estimate : 0.000366
Iteration number: 14
Approximation of the root: 1.128479
The error estimate : 0.000183
Iteration number: 15
Approximation of the root: 1.128387
The error estimate : 0.000092
(c) With tolerance 10−15 it takes 52 iterations to obtain the root = 0.424031039490741.f3(0.424031039490741) =−6.6613e−16.With tolerance 10−4:>> bisect(0,3,1.e-4,300,18)
Iteration number: 1
Approximation of the root: 1.500000
The error estimate : 1.500000
Iteration number: 2
Approximation of the root: 0.750000
The error estimate : 0.750000
Iteration number: 3
Approximation of the root: 0.375000
The error estimate : 0.375000
Iteration number: 4
Approximation of the root: 0.562500
The error estimate : 0.187500
Iteration number: 5
Approximation of the root: 0.468750
The error estimate : 0.093750
Iteration number: 6
Approximation of the root: 0.421875
The error estimate : 0.046875
Iteration number: 7
Approximation of the root: 0.445312
The error estimate : 0.023438
Iteration number: 8
Approximation of the root: 0.433594
The error estimate : 0.011719
Iteration number: 9
104
Approximation of the root: 0.427734
The error estimate : 0.005859
Iteration number: 10
Approximation of the root: 0.424805
The error estimate : 0.002930
Iteration number: 11
Approximation of the root: 0.423340
The error estimate : 0.001465
Iteration number: 12
Approximation of the root: 0.424072
The error estimate : 0.000732
Iteration number: 13
Approximation of the root: 0.423706
The error estimate : 0.000366
Iteration number: 14
Approximation of the root: 0.423889
The error estimate : 0.000183
Iteration number: 15
Approximation of the root: 0.423981
The error estimate : 0.000092
root =
0.423980712890625
Exercise 2, page 77. To help determine the roots of x = tan(x), graph both y = x and y = tan(x), and look at theintersection points of the two curves.
-2 -1 0 1 2 3 4 5 6 7 8
-10
-8
-6
-4
-2
0
2
4
6
8
10
105
(a) Find the smallest nonzero positive root of x = tan(x), with an accuracy of ε = 10−4.Note: The desired root is greater than π/2.
(b) Solve x = tan(x) for the root that is closest to x = 100.Solution:(a) From the graph we see that the first zero of the function f (x)= x− tan(x) is between [π/2,3π/2]. With ε = 10−15,
in 52 iterates we get α = 4.493409457909064.>> bisect(pi/2+0.01,3*pi/2-0.01,1.e-4,100,19)
Iteration number: 15
Approximation of the root: 4.493476239621677
The error estimate : 0.000095
(b) >> bisect(31.5*pi,32.5*pi,1.e-4,100,19)
Iteration number: 15
Approximation of the root: 102.0918862403462
The error estimate : 0.000096
Exercise 4, page 77. Show that for any real constants c and d, the equation x =−c−d cos(x) has at least one root.
-10 -8 -6 -4 -2 0 2 4 6 8 10-15
-10
-5
0
5
10
f(x)=x-c-d*cos(x)
FIG. 15.3. f (x) = x− c−d cos(x), with c = 2,d =−3.
Hint: Find an interval [a,b] on which f (x) = x− c−d cos(x) changes sign.Solution:
Exercise 6, page 77. Use the bisection method and a graph of f (x), to find all roots of
f (x) = 32x2−48x4 +18x2−1.
The true roots are
cos((2 j−1)
π
12
), j = 1,2, · · · ,6.
Solution:>> bisect(-1,-0.8,1.e-6,100,20)
Iteration number: 18
Approximation of the root: -0.965926361083984
>> bisect(-0.8,-0.6,1.e-6,100,20)
Iteration number: 18
Approximation of the root: -0.707106781005859
The error estimate : 0.000001
106
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
FIG. 15.4. f (x) = 32x2−48x4 +18x2−1.
>> bisect(-0.4,-0.2,1.e-6,100,20)
Iteration number: 18
Approximation of the root: -0.258818817138672
>> bisect(0.2,0.4,1.e-6,100,20)
Iteration number: 18
Approximation of the root: 0.258818817138672
>> bisect(0.6,0.8,1.e-6,100,20)
Iteration number: 18
Approximation of the root: 0.707106781005859
>> bisect(0.8,1.0,1.e-6,100,20)
Iteration number: 18
Approximation of the root: 0.965926361083984
Exercise 7, page 78. Using the program of Problem 5, solve the equation
f (x)≡ x3−3x2 +3x−1 = 0 ( f (x) = (x−1)3)
with an accuracy of ε = 10−6.Experiment with different ways of evaluating f (x); for example, use (i) the given form, (ii) reverse its order, and (iii)the nested form
f (x) =−1+ x(3+ x(−3+ x)).
Try various initial intervals [a,b], for example [0,1.5], [0.5,2.0], and [0.5,1.1].Explain the results.(Note that α = 1 is the only root of f (x) = 0.)Solution:
[a,b] x3−3x2 +3x−1 −1+3x−3x2 + x3 −1+ x(3+ x(−3+ x))[0.0,1.5] 1.000006914138794 (18 iter) 1.000005483627319 (21iter) 1.000005483627319 (21 iter)[0.5,2.0] 1.000007390975952 (21 iter) 1.000004529953003 (21 iter) 1.000004529953003 (21 iter)[0.5,1.1] 1.000006675720215 (20 iter) 1.000005531311035 (20 iter) 1.000004386901856 (20 iter)
107
Exercise 8, page 78. The polynomial
f (x) = x4−5.4x3 +10.56x2−8.954x+2.7951
has a root α in [1,1.2]. Repeat Problem 7, but vary the intervals [a,b] to reflect the location of the root α .Solution: With ε = 10−15, starting from [1.08,1.15], in 46 iterations, the nested form gives α = 1.100006916588192.With ε = 10−6:
[a,b] x4−5.4x3 +10.56x2−8.95x+2.7951 2.7951+ x(−8.95+ x(10.56+ x(−5.4+ x)))[1.00,1.20] 1.100002288818359 (18 iter) 1.099996185302734 (18 iter)[1.08,1.15] 1.100002059936524 (17 iter) 1.100007400512696 (17 iter)
Exercise 9, page 78. Let the initial interval used in the bisection method have length b−a = 3. Find the numberof midpoints cn that must be calculated with the bisection method to obtain an approximate root within an errortolerance of ε = 10−9.Solution: Using estimate (15.4)
≡−1+log( 3
10−9
)log(2)
= 30.4823,
hence 31 iterates.
Exercise 10, page 78. Consider the equation e−x = sinx. Find an interval [a,b] that contains the smallestpositive root. Estimate the number of midpoints c needed to obtain an approximate root that is accurate within anerror tolerance of 10−10.
0 1 2 3 4 5 6 7-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
e-x -sin(x)
(root,f(root))
Solution: The function
f (x) = e−x− sinx
has the smallest root on the interval [0,2]. With ε = 10−10, the bisection method gives>> bisect(0.0,2.0,1.e-10,100,8)
Iteration number: 34
Approximation of the root: 0.588532744033728
Using estimate (15.4)
m≥−1+log( b−a
ε
)log(2)
≡−1+log( 2
10−10
)log(2)
= 33.2193,
108
hence 34 iterates.
Exercise 11, page 78. Let α be the smallest positive root of
f (x)≡ 1− x+ sinx = 0
Find an interval [a,b] containing α and for which the bisection method will converge to α .
-8 -6 -4 -2 0 2 4 6 8-6
-4
-2
0
2
4
6
8
f(x)=1-x-sin(x)
(root,f(root))
FIG. 15.5. Exercise 11: f (x)≡ 1− x+ sinx = 0.
Then estimate the number of iterates needed to find α within an accuracy of 5×10−8.Solution: With ε = 10−15 the bisection method gives α = 1.934563210752025 in 53 iterations. With varepsilon =5×10−8 we obtain:>> bisect(0,2*pi,5.e-8,100,9)
Iteration number: 26
Approximation of the root: 1.934563195716542
Using estimate (15.4)
m≥−1+log( b−a
ε
)log(2)
≡−1+log( 2π
5∗10−8
)log(2)
= 25.9050,
hence 26 iterates.
Exercise 12, page 78. Let α be the unique root of
x =3
1+ x4 .
Find an interval [a,b] containing α and for which the bisection method will converge to α .Then estimate the number of iterates needed to find α within an accuracy of 5×10−8.Solution: With ε = 10−15 the bisection method gives α = 1.132997565885066 in 50 iterations. With ε = 5× 10−8
we obtain:>> bisect(0,2,5.e-8,100,21)
Iteration number: 25
Approximation of the root: 1.132997542619705
Using estimate (15.4)
m≥−1+log( b−a
ε
)log(2)
≡−1+log( 2
5∗10−8
)log(2)
= 24.2535,
109
-3 -2 -1 0 1 2 3-4
-3
-2
-1
0
1
2
3
x-3/(1+x 4 )
(root,f(root))
FIG. 15.6. Exercise 12: x− 31+x4 = 0.
hence 25 iterates.
Exercise 13 page 79. Let α be the largest root of
f (x)≡ ex− x−2 = 0.
-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
ex-x-2
(root,f(root))
FIG. 15.7. Exercise 13: ex− x−2 = 0.
Find an interval [a,b] containing α and for which the bisection method will converge to α .Then estimate the number of iterates needed to find α within an accuracy of 5×10−8.Solution: With ε = 10−15 the bisection method gives α = 1.146193220620583 in 51 iterations. With ε = 5× 10−8
we obtain:>> bisect(-1.5,1.5,5.e-8,100,22)
Iteration number: 25
Approximation of the root: 1.146193251013756
Using estimate (15.4)
m≥−1+log( b−a
ε
)log(2)
≡−1+log( 3
5∗10−8
)log(2)
= 24.8385,
hence 25 iterates.
110
15.2. Subsection 3.2 NEWTON’S METHOD.The methods of chord, secant and NewtonGOAL: for better convergence - include values of both f and f ′, or suitable approximations.
Using Taylor series of f about the root α:
0 = f (α) = f (x)+ f ′(ξ ) · (α− x), ξ between x and α
equivalently
α = x− f (x)f ′(ξ )
, if x is close to α, and f is C1 near α.
Result: Algorithm “iterative method”for k ≥ 0, given x(k) do
x(k+1) = x(k)− f (x(k))q(k)
where q(k) ≈ f ′(x(k)) ;
endAs particular cases:
q(k) ≡ q :=f (b)− f (a)
b−a⇒ x(k+1) = x(k)− f (x(k))
f (b)− f (a)(b−a) (chord method)
q(k) ≡ f (x(k))− f (x(k−1))
x(k)− x(k−1) ⇒ x(k+1) = x(k)− f (x(k))f (x(k))− f (x(k−1))
(x(k)− x(k−1)) (secant method, x(0),x(1))
q(k) ≡ f ′(x(k)) ⇒ x(k+1) = x(k)− f (x(k))f ′(x(k))
(Newton’s method)
THEOREM 15.6 (Newton’s method - quadratic convergence).If α is a root of multiplicity one, and the initial guess x(0) is sufficiently close to α so that the Newton method
converges x(k)k↗∞−−−→ α , then Newton’s method has order of convergence two:
limk→∞
α− x(k+1)
(α− x(k))2=− f ′′(α)
2 f ′(α). (Newton’s method - quadratic convergence)
Proof. Using the Taylor expansion about x(k):
0 = f (α) = f (x(k))+(α− x(k)) f ′(x(k))+12(α− x(k))2 f ′′(ξ (k)).
Dividing by f ′(x(k)) and using the Newton’s method (x(k+1) = x(k)− f (x(k))f ′(x(k))
):
0 =f (x(k))f ′(x(k))︸ ︷︷ ︸
=x(k)−x(k+1)
+α− x(k)+12(α− x(k))2 f ′′(ξ (k))
f ′(x(k))
111
=��x(k)− x(k+1)+α−��x(k)+12(α− x(k))2 f ′′(ξ (k))
f ′(x(k)).
Equivalently we have
α− x(k+1) = (α− x(k))2(− f ′′(ξ (k))
2 f ′(x(k))
),
α− x(k+1)
(α− x(k))2=− f ′′(ξ (k))
2 f ′(x(k))
which gives (Newton’s method - quadratic convergence).
REMARK 46 (Newton’s method is locally convergent).Convergence of Newton’s method is guaranteed provided the initial guess x(0) is close enough to the root α:
|α− x(0)|<∣∣∣∣2 f ′(α)
f ′′(α)
∣∣∣∣. (3.21 - Newton’s method - local convergence)
Proof. Indeed form the previous proposition we have:
|α− x(k+1)|∣∣∣∣ f ′′(α)
2 f ′(α)
∣∣∣∣≈ (|α− x(k)|∣∣∣∣ f ′′(α)
2 f ′(α)
∣∣∣∣)21
≈(|α− x(k−1)|
∣∣∣∣ f ′′(α)
2 f ′(α)
∣∣∣∣)22
≈ ·· ·
≈(|α− x(0)|
∣∣∣∣ f ′′(α)
2 f ′(α)
∣∣∣∣)2k+1
,
hence for convergence
|α− x(0)|∣∣∣∣ f ′′(α)
2 f ′(α)
∣∣∣∣< 1,
which concludes the argument.COROLLARY 15.7 (Error estimation). Using Taylor expansion:
α− x(k) =− f (x(k))f ′(ξ (k))
(Taylor expansion: f (x(k)) =���f (α)+ f ′(ξ (k))(x(k)−α))
≈− f (x(k))f ′(x(k))
(using the Newton iteration: x(k+1) = x(k)− f (x(k))f ′(x(k))
)
= x(k+1)− x(k).
Exercise 1, page 88. Carry out the Newton iteration
x(n+1) = x(n)−x6(n)− x(n)−1
6x5(n)−1
, n≥ 0 (3.12)
with the two initial guesses x0 = 1.5 and x0 = 2.0.Solution:
112
n x(n) f (x(n)) x(n)− x(n−1)0 1.5 8.8906e+00 -1.9951e-011 1.300490883590 2.5373e+00 -1.1901e-012 1.181480416403 5.3846e-01 -4.2025e-023 1.139455590276 4.9235e-02 -4.6780e-034 1.134777625237 5.5032e-04 -5.3480e-055 1.13472414531 7.1136e-08 -6.9147e-096 1.134724138402 1.5543e-15 -2.2204e-16n x(n) f (x(n)) x(n)− x(n−1)0 2.0 6.1000e+01 -3.1937e-011 1.680628272251 1.9853e+01 -2.4989e-012 1.430738988239 6.1468e+00 -1.7577e-013 1.254970956109 1.6517e+00 -9.3433e-024 1.161538432773 2.9431e-01 -2.5185e-025 1.136353274171 1.6826e-02 -1.6227e-036 1.134730528344 6.5738e-05 -6.3898e-067 1.134724138500 1.0154e-09 -9.8702e-118 1.134724138402 -8.8818e-16 0
Exercise 2, page page 88.Using Newton’s method, find the roots of the equations. Use an error tolerance of ε = 10−6.(a) The real root of x3− x2− x−1 = 0.(b) The root of x = 1+0.3cosx; f (x) := x−1−0.3cosx.(c) The smallest positive root of cosx = 1
2 + sinx, f (x) := cosx− 12 − sinx.
Solution:
-1 -0.5 0 0.5 1 1.5 2 2.5 3
-2
0
2
4
6
8
10
12
14
x3-x
2-x-1
root
(a) With ε = 10−15 the root approximation is 1.839286755214161.With ε = 10−6 we obtain:>> [root,error_bd,it_count] = newton(0,1.e-6,100,3)
Iteration number: 13
Approximation of the root: 1.839286755214161
The error estimate: -0.000000007291
(b) With ε = 10−15 the root approximation is 1.128425092992225.With ε = 10−6 we obtain:>> [root,error_bd,it_count] = newton(0,1.e-6,100,6)
Iteration number: 3
Approximation of the root: 1.128425092992225
113
-1 -0.5 0 0.5 1 1.5 2 2.5 3-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
x-1-0.3*cos(x)
root
The error estimate: -0.000000061308
-1 -0.5 0 0.5 1 1.5 2 2.5 3-2
-1.5
-1
-0.5
0
0.5
1
cos(x)-0.5-sin(x)
root
(c) With ε = 10−15 the root approximation is 0.424031039490741 in 5 iterations.With ε = 10−6 we obtain:>> [root,error_bd,it_count] = newton(0,1.e-6,100,4)
Iteration number: 3
Approximation of the root: 0.424031039490745
The error estimate: -1.5940e-07
Exercise 4, page 88.Give Newton’s method for finding m
√a, with a > 0 and m a positive integer. Apply it to finding m
√2 for m =
3,4,5,6,7,8, say, to six significant digits.Hint: Solve xm−a = 0.Solution:
m ‘root’ f (x(m)) iterates3 1.259921050018 5.8526e-10 44 1.189207115676 4.5301e-09 45 1.148698356620 1.4128e-08 46 1.122462051041 2.9198e-08 47 1.104089517486 4.8335e-08 48 1.090507737437 7.0019e-08 4
Defining
f (x) := xm−a, f ′(x) = mxm−1,
114
the Newton’s iteration writes
x(n+1) = x(n)−f (x(n))f ′(x(n))
≡ x(n)−xm(n)−a
mxm−1(n)
=(m−1)xm
(n)+a
mxm−1(n)
.
With tolerance ε = 5×10−7, · · · (see the Table).Exercise 5, page 88.
To help determine the roots of x = tan(x), graph both y = x and y = tan(x), and look at the intersection points of thetwo curves.
-2 -1 0 1 2 3 4 5 6 7 8
-10
-8
-6
-4
-2
0
2
4
6
8
10
(a) Find the smallest nonzero positive root of x= tan(x), using Newton’s method, with an error tolerance of ε = 10−6.Note: The desired root is greater than π/2.
(b) Solve x = tan(x) for the root that is closest to x = 100. The root near 100 will be difficult to find using Newton’smethod. To explain this, compute the quantity M of (3.21 - Newton’s method - local convergence):
M :=− f ′′(α)
2 f ′(α),
and use it in the condition
|α− x(0)|<1M
=
∣∣∣∣2 f ′(α)
f ′′(α)
∣∣∣∣for x(0).Solution:
f (x) = x− tan(x), f ′(x) = 1− 1cos2 x
, f ′′(x) =−2sinx
cos3 x,
x(n+1) = x(n)−1
sinx(n) cos3 x(n)(Newton iteration)
M :=−2 sinα
cos3 α
2 1cos2 α
=− sinα
cosα,
withsin(4.28)cos(4.28)
≈ 98.9501⇒ |α− x(0)|<1
98.9501≈ 0.0101,
115
and withsin(102.1)cos(102.1)
≈ 567.7807⇒ |α− x(0)|<1
567.7807≈ 0.0018,
>> [root,error_bd,it_count] = newton(4.288,1.e-6,200,31)
Iteration number: 13
Approximation of the root: 4.493409630214
The error estimate: -1.7231e-07
>> [root,error_bd,it_count] = newton(102.1,1.e-6,200,31)
Iteration number: 6
Approximation of the root: 102.091966495223
The error estimate: -3.0315e-08
Exercise 6, page 88. The equation
f (x)≡ x+ e−Bx2cosx = 0, B > 0
has a uniques root, and it is in the interval (−1,0).
-1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
B=1
B=5
B=10
B=25
B=50
Use Newton’s method to find it as accurately as possible. Use values of B = 1,5,10,25,50. Among your choices ofx0, choose x0 = 0, and explain the behavior observed in the iterates for the larger values of B.Solution:
f (x) = x+ e−Bx2cosx, f ′(x) = 1−2Bxe−Bx2
cosx− e−Bx2sinx;
f (0) = 1, f ′(0) = 1,
f (x)≈ x, f ′(x) = 1, (for B� 1)f (−1)≈−1, f ′(−1)≈ 1 (for B� 1)
xn+1 = xn−f (xn)
f ′(xn)≈ xn−
xn
1≈ 0 (for B� 1)
xn+2 = xn+1−f (xn+1)
f ′(xn+1)≈ 0− f (0)
f ′(0)≈ 0− 1
1=−1 (for B� 1)
xn+3 = xn+2−f (xn+2)
f ′(xn+2)≈−1− f (−1)
f ′(−1)≈−1− −1
1= 0≡ xn+1 (for B� 1)
xn+4 = xn+2.
116
B “Root” iterates f (“root”) α
1 -0.588401776501 5 -1.1102e-16 -0.58845 -0.404911548209 8 5.5511e-17 -0.404910 -0.9845988999155 200 9.8385e-01 -0.326425 -0.999999979887 200 1.0000e+00 -0.237450 -1.000000000000 200 1 -0.1399
Exercise 10, page 89.Solve the equation
x3−3x2 +3x−1 = 0 (multiply root: (x−1)3)
on a computer and use Newton’s method.Experiment with the choice of initial guess x0.Also experiment with different ways of evaluating f (x) and f ′(x).Note any unusual behavior in the iteration.Solution:
-100 -80 -60 -40 -20 0 20 40 60 80 100-1.5
-1
-0.5
0
0.5
110
6
(x-1)3
root: (1,0)
x(0) iterates Root-100 42 0.999994068953-10 36 0.9999970003770 45 0.9999956528100.5 31 0.9999992292331.5 27 1.00000763994610 35 1.000007110518100 41 1.000007209443
Exercise 11, page 89.Solve the equation
x4−5.4x3 +10.56x2−8.954x+2.7951 = 0
on a computer and use Newton’s method. Look for the root α located in [1,1.2].Experiment with the choice of initial guess x0.
117
Solution:
f ′(x) = 4x3−16.4x2 +21.12x−8.954, f ′(1)≈−0.0340, f ′(1.2)≈−0.0260.
1.09 1.095 1.1 1.105 1.11 1.115
-1
-0.5
0
0.5
1
1.510
-6
x4 - 5.4 x
3 + 10.56 x.
2 - 8.954 x + 2.7951
FIG. 15.8. Exercise 11
Using tolerance ε = 10−6:
x(0) iterates Root f(‘root’)-100 86 1.099987461861 1.3323e-15
-10 192 1.099987564173 1.3323e-150 90 1.099987355516
0.5 27 1.0999866269851.5 91 1.099986589433 4.4409e-1610 13 2.100000000021 2.0999e-11
100 21 2.100000013363 1.3363e-08
118
15.3. Subsection 3.2 SECANT METHOD.Recall the general iterative method:
x(k+1) = x(k)− f (x(k))q(k)
, where q(k) ≈ f ′(x(k)).
Using
q(k) :=f (x(k))− f (x(k−1))
x(k)− x(k−1) ,
we obtain the following:
given x(0),x(1) find:
x(k+1) = x(k)− f (x(k))f (x(k))− f (x(k−1))
(x(k)− x(k−1)), ∀k ≥ 1. (secant method)
Recall (Newton’s method - quadratic convergence):
limk→∞
α− x(k+1)Newton(
α− x(k)Newton
)2 =− f ′′(α)
2 f ′(α). (Newton’s method - quadratic convergence)
THEOREM 15.8 (Secant method - superlinear convergence).If α is a root of multiplicity one, and the initial guess x(0) is sufficiently close to α so that the Secant method converges
x(k)k↗∞−−−→ α , then the Secant method has order of convergence p = 1+
√5
2 ≈ 1.618:
limk→∞
|α− x(k+1)Secant |
|α− x(k)Secant |q= max
x,y∈V (α)
∣∣∣∣12 f ′′(y)f ′(x)
∣∣∣∣q−1
. (Secant method: order q = 1+√
52 convergence)
The proof of Theorem 15.8 is based on the following result.PROPOSITION 15.9. The error in the (secant method) method ek := α− x(k)secant satisfies the following recursion
relation:
ek+1 =−f ′′(ξk)
2 f ′(θk)ekek−1.
Proof. Recall the (interpolating polynomial in Newton form)
(Π2 f )(x) = f (x0)+(x− x0) f [x0,x1]+ (x− x0)(x− x1) f [x0,x1,x2],
and choose
x0 = x(k), x1 = x(k−1), x2 = α,
where f (α) = 0 (and consequently (Π2 f )(α)≡ f (α) = 0). Therefore
0 = f (α)≡ (Π2 f )(α)
119
= f (x(k))+(α− x(k)) f [x(k),x(k−1)]+ (α− x(k))(α− x(k−1)) f [x(k),x(k−1),α].(Also using the (secant method): x(k+1)− x(k) =− f (x(k))
f [x(k),x(k−1)]
)0 = f (x(k))+(x(k+1)− x(k)) f [x(k),x(k−1)],
( hence subtracting the above relations we obtain )
0 = (α−��x(k)− x(k+1)+��x(k)) f [x(k),x(k−1)]+ e(k)e(k−1) f [x(k),x(k−1),α].
= e(k+1) f [x(k),x(k−1)]+ e(k)e(k−1) f [x(k),x(k−1),α].
Therefore
e(k+1) =−f [x(k),x(k−1),α]
f [x(k),x(k−1)]e(k)e(k−1).
Finally, using (8.1) in Theorem 8.3 we obtain:
e(k+1) =−f [x(k),x(k−1),α]
f [x(k),x(k−1)]e(k)e(k−1) ≈−
f ′′(ξk)
2 f ′(θk)e(k)e(k−1),
which concludes the proof.
Then Theorem 15.8 is an easy consequence of Proposition 15.9 and the following result.THEOREM 15.10. Let f be a twice continuously differentiable function in a neighborhood of the root α such
that
12
∣∣∣∣ f ′′(y)f ′(x)
∣∣∣∣≤M, ∀x,y in a neighborhood of α.
Let x(0),x(1), · · · ,x(n) denote the (secant method) iterates, en = α− x(n) be the error in the iterates,
and let K := max{
M|e0|,(M|e1|
) 1q}
, where q = 1+√
52 .
(Note that q is a solution of q2−q−1 = 0.)Then
|en| ≤1M
Kqn, n = 0,1, · · · , (15.5)
i.e., the (secant method) converges with order q = 1+√
52 ≈ 1.618, since
en
|en−1|q≈ 1
MKqn ·
(M
Kqn−1
)q
= Mq−1 Kqn
Kqn−1·q = Mq−1 = constant.
Proof.
|en|=∣∣∣∣− 1
2f ′′(ξn)
f ′(ζn)en−1en−2
∣∣∣∣≤Men−1en−2 (using (15.5))
≤��M ·1��M
Kqn−1 · 1M
Kqn−2=
1M
Kqn−2(q+1) (using q2 = q+1)
=1M
Kqn−2q2=
1M
Kqn+1
120
which concludes the argument.
Comparison of efficiency: (Newton’s method) versus (secant method)Recall the order of convergence for Newton’s and Secant methods’:
qN = 2, (q Newton′s method)
qS =1+√
52
≈ 1.618. (Secant method)
Also recall the definition of (efficiency index)
E := q1m , (efficiency index)
where m = units of work required for each iteration.PROPOSITION 15.11. Assume the work to compute f ′(x) is θ times the amount of work required to compute
f (x). Then if θ > 0.44, the secant method is asymptotically more efficient than Newton’s method.Proof. To compare the amount of work done by the Newton and secant method, we have
Newton work: (1+θ)m (Newton iteration: compute f and f ′ ≈ θ f )secant work: m (secant iteration: compute f )
therefore to obtain the same amount work, it takes k Newton iterations while k(1+θ) secant iterations:
k{Newton iteration}m≈ (1+θ)k{secant iteration}m.
In terms of errors this means: ((me(0))
qN)k︸ ︷︷ ︸
Newton
≈((me(0))
qs)k(1+θ)︸ ︷︷ ︸
secant
, (qN = 2,qs = 1.618)
2k = 1.618k(1+θ),
�k log2 = �k(1+θ) log(1.618)
θ =log(2)
log(1.618)−1 = 0.4404
which concludes the argument.
Exercise 1, page page 96.Using the secant method, find the roots of the equations. Use an error tolerance of ε = 10−6.(a) The real root of x3− x2− x−1 = 0.(b) The root of x = 1+0.3cosx; f (x) := x−1−0.3cosx.(c) The smallest positive root of cosx = 1
2 + sinx, f (x) := cosx− 12 − sinx.
Solution: We use as initial guesses x0 = 0,x1 = 2:(a) With ε = 10−15 the root approximation is 1.839286755214161.
With ε = 10−6 we obtain, using secant:>> [root,error_bd,it_count] = secant(0,2,1.e-6,100,3)
Iteration number: 9
Approximation of the root: 1.839286754889028
The error estimate: 3.2513e-10
121
while we recall the Newton’s method results:>> [root,error_bd,it_count] = newton(0,1.e-6,100,3)
Iteration number: 13
Approximation of the root: 1.839286755214161
The error estimate: -0.000000007291
(b) With ε = 10−15 the root approximation is 1.128425092992225.With ε = 10−6 we obtain:>> [root,error_bd,it_count] = secant(0,2,1.e-6,100,7)
Iteration number: 5
Approximation of the root: 1.128425092992225
The error estimate: 1.3526e-10
while we recall the Newton’s method results:>> [root,error_bd,it_count] = newton(0,1.e-6,100,6)
Iteration number: 3
Approximation of the root: 1.128425092992225
The error estimate: -0.000000061308
(c) With ε = 10−15 the root approximation is 0.424031039490741 in 5 iterations.With ε = 10−6 we obtain:>> [root,error_bd,it_count] = secant(0,2,1.e-6,100,4)
Iteration number: 4
Approximation of the root: 0.424031039649515
The error estimate: 9.7260e-07
while we recall the Newton’s method results:>> [root,error_bd,it_count] = newton(0,1.e-6,100,4)
Iteration number: 3
Approximation of the root: 0.424031039490745
The error estimate: -1.5940e-07
122
15.4. Subsection 3.2 FIXED POINT ITERATION.
Exercise 2, page 106.(a) Calculate the first six iterates in the iteration
xn+1 =0.5
1+ x2n
with x0 = 1. Chose another initial guesses x0 and repeat this calculation.(b) Find an interval [a,b] satisfying the hypotheses of Theorem ??.
Hint: For g(x) = 0.51+x , let
a = minx∈(−∞,∞)
g(x), b = maxx∈(−∞,∞)
g(x).
(c) Prepare a table in the same manner as Table 3.5 in the preceding discussion.The true solution is α ≈ 0.423853799069783.Solution:
(a) >> root = fixedpoint(inline(’0.5./(1+x.^2)’),0,5)
The iterations are x=0.0000000000000000
The iterations are x=0.5000000000000000
The iterations are x=0.4000000000000000
The iterations are x=0.4310344827586207
The iterations are x=0.4216595638004512
The errors are e=0.423854
The errors are e=-0.076146
The errors are e=0.023854
The errors are e=-0.007181
The errors are e=0.002194
The ratios are r=0.000000
The ratios are r=-0.179652
The ratios are r=-0.313263
The ratios are r=-0.301029
The ratios are r=-0.305575
The "Picard" approximation of the root,
after
5 iterations
is
0.4216595638004512,
with error 0.00219424
root =
0.421659563800451
123
iteration n x0 = 0 x0 = 1.50 0.0000000000000000 1.50000000000000001 0.5000000000000000 0.15384615384615392 0.4000000000000000 0.48843930635838163 0.4310344827586207 0.40369039028305134 0.4216595638004512 0.42993520840384285 0.4245214498823317 0.4219963812920247
-50 -40 -30 -20 -10 0 10 20 30 40 500
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.5/(1+x2 )
(b) We have
a = minx∈(−∞,∞)
0.51+ x2 = 0, b = max
x∈(−∞,∞)
0.51+ x2 = 0.5.
(c) We have
g′(x) =− x(1+ x2)2 , g′(α) =− α
(1+α2)2 ≈−0.304584803720867.
The computations are
n xn error α− xn ratios rn =α−xn
α−xn−1
0 1.0000000000000000 -5.761462e-011 0.2500000000000000 1.738538e-01 -3.017529e-012 0.4705882352941176 -4.673444e-02 -2.688146e-013 0.4093484419263456 1.450536e-02 -3.103783e-014 0.4282412618114397 -4.387463e-03 -3.024719e-015 0.4225147716747344 1.339027e-03 -3.051940e-016 0.4242613938187147 -4.075947e-04 -3.043961e-017 0.4237296285978768 1.241705e-04 -3.046420e-018 0.4238916173427634 -3.781827e-05 -3.045674e-019 0.4238422799977083 1.151907e-05 -3.045901e-01
Exercise 3, page 106.How many solutions are there to the equation x = e−x?Will the iteration xn+1 = e−xn converge for suitable choices of x0?
124
Calculate the first six iterates when x0 = 0.Solution: There is only one solution (y = x is increasing, while y = e−x is decreasing).
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
1.5
2
2.5
3
y=x
y=e-x
Since
a = minx∈(−∞,∞)
e−x = 0, b = maxx∈(0,∞)
e−x = e0 = 1,
g′(x) =−e−x ∈ (−1,0], ∀x ∈ (0,1]
the iterations will converge if x0 ∈ (0,1].
We used x0 = 0.5:>> fixedpoint(inline(’exp(-x)’),0.5,7)
The iterations are x=0.5000000000000000
The iterations are x=0.6065306597126334
The iterations are x=0.5452392118926050
The iterations are x=0.5797030948780683
The iterations are x=0.5600646279389019
The iterations are x=0.5711721489772151
The iterations are x=0.5648629469803235
while in 60 iterates x60 = 0.5671432904097841.Exercise 6, page 107. Convert the equation x2−5 = 0 to the fixed-point problem
x = x+ c(x2−5)≡ g(x)
with c a nonzero constant.Determine the possible values of c to ensure convergence of
xn+1 = xn + c(x2n−5)
to α =√
5.Solution: We have
g′(x) = 1+2cx
hence for convergence we have to ensure that
|g′(x)|= |1+2cx|< 1, −1 < 1+2cx < 1, −1 < cx < 0, ∀x ∈ [a,b].
Since α =√
5 > 0, we need c < 0.With c < 0 we have that
maxx∈R
(x+ c(x2−5)
)=− 1
2c+ c( 1
4c2 −5)=−1+20c2
4c> 0. (g′(x∗) = 0,x∗ =− 1
2c )
125
Then, since g′(x) = 1+2cx is decreasing (for c < 0), we chose
a =− 14c
, b = max{− 1
2c,−1+20c2
4c
}.
Exercise 7, page 107.What are the solutions α , if any, of the equation x =
√1+ x?
Does the iteration xn+1 =√
1+ xn converge to any of these solutions (assuming x0 is chosen sufficiently close to α)?Solution: There is only one solution. Squaring the equation:
-2 0 2 4 6 8 10-2
0
2
4
6
8
10
y=x
y= 1+x
x2 = 1+ x, x1,2 =1±√
52
,
with one root being negative. Hence
α =1+√
52
.
With g(x) =√
1+ x,g′(x) = 12√
1+x∈ (0,1) for x ∈ [1,2].
126