perfidious polynomials and elusive roots
DESCRIPTION
Perfidious Polynomials and Elusive Roots. Zhonggang Zeng. Northeastern Illinois University. Nov. 2, 2001 at Northern Illinois University. What are the “ best ” functions?. 1. Linear functions 2. Polynomials 3. Exponential functions 4. Trigonometric functions. Weierstrass Theorem: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/1.jpg)
Perfidious Polynomials and Elusive RootsZhonggang Zeng
Northeastern Illinois University
Nov. 2, 2001 at Northern Illinois University
![Page 2: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/2.jpg)
What are the “best” functions?
1. Linear functions2. Polynomials3. Exponential functions4. Trigonometric functions... ...
Weierstrass Theorem: Any continuous function can be approximated to any accuracy by a polynomial of sufficiently high degree.
Why by a
polynomial ?
![Page 3: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/3.jpg)
Polynomial root-finding
xn + a1 xn-1 + a2 xn-2 + ... + an-1x + a0 = 0
has played a key role in the history of mathematics
- Rational/irrational numbers, complex numbers- Fundamental Theorem of Algebra- Abel and Galois Theorem... ...
![Page 4: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/4.jpg)
Can you solve (x-1.0 )100 = 0 Can you solve
x100-100 x99 +4950 x98 - 161700 x97+3921225x96 - ... - 100 x +1 = 0
![Page 5: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/5.jpg)
1 01 1 1 1 ... ... 1 1
0 1 1
A = X X-1
Eigenvalues of
![Page 6: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/6.jpg)
The Wilkinson polynomial
p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ...
Wilkinson wrote in 1984:
Speaking for myself I regard it as the most traumatic experience in my career as a numerical analyst.
![Page 7: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/7.jpg)
Classical textook methods for multiple roots
Newton’s iteration xj+1 = xj - f(xj)/f’(xj), j=0,1,2,...converges locally to a multiple root of f(x) with a linear rate.
The modified Newton’s iteration xj+1 = xj - mf(xj)/f’(xj), j=0,1,2,...converges locally to a m-fold root of f(x) with a quadratic rate.
Newton’s iteration applied to g(x) = f(x)/f’(x)converges locally and quadratically to a root of f(x) regardliss of its multiplicity.
None of them work!
![Page 8: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/8.jpg)
Example: f(x) = (x-2)7(x-3)(x-4) in expanded form.
Modified Newton’s iteration with m = 7 intended for root x = 2:
x1 = 1.9981x2 = 1.7481x3 = 1.9892x4 = 0.4726x5 = 1.8029x6 = 1.9931x7 = 4.2681x8 = 3.3476 ... ...
![Page 9: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/9.jpg)
Myths on polynomial roots:
- multiple roots are ill-conditioned, or even intractable
- extension of machine precision is necessary to calculate multiple roots
- there is an “attainable precision” for multiple roots:
machine precision attainable precision = ----------------------------- multiplicity
Example: for 100-fold root, to get 5 digits right
500 digits in machine precision 5 digits precision = -----------------------------------------
100 in multiplicity
![Page 10: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/10.jpg)
How do we justify the answer?
![Page 11: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/11.jpg)
The condition number
[Forward error] ~ [Condition number] [Backward error]
A large condition number <=> The problem is sensitive or, ill-conditioned
From computational method
From problem
![Page 12: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/12.jpg)
The backward error: 5 x 10-10
The forward error: 5
Conclusion: the problem is “bad”
-- method is good!
-- Ouch! Who’s responsible?
![Page 13: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/13.jpg)
If the answer is highly sensitive to perturbations, you have probably asked the wrong question.
Maxims about numerical mathematics, computers, science and life, L. N. Trefethen. SIAM News
Who is asking a wrong question?
What is the wrong question?
A: “Customer”
B: Numerical analyst
A: The polynomial
B: The computing objective
![Page 14: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/14.jpg)
The question we used to askThe question we used to ask:(Fundamental Theorem of Algebra)
Given a polynomial
p(x) = xn + a1 xn-1+...+an-1 x + an
find z = ( z1, ..., zn ) such that
p(x) = ( x - z1 )( x - z2 ) ... ( x - zn )
Right - or - Wrong ?
![Page 15: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/15.jpg)
Kahan’s pejorative manifolds
xn + a1 xn-1+...+an-1 x + an <=> (a1 , ..., an-1 , an )
All n-polynomials having certain multiplicity structure form a pejorative manifold
Example: ( x-t )2 = x2 + (-2t) x + t2
Pejorative manifold: a1= -2t a2= t2
![Page 16: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/16.jpg)
Pejorative manifolds of 3-polynomials
( x - s )( x - t )2 = x3 + (-s-2t) x2 + (2st+t2) x + (-st2)
( x - s )3 = x3 + (-3s) x2 + (3s2) x + (-s3)
Pejorative manifold of multiplicity structure [1,2]
a1= -s-2ta2= 2st+t2
a3= -st2
Pejorative manifold ofmultiplicity structure [ 3 ]
a1 = -3sa2 = 3s2
a3 = -s3
![Page 17: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/17.jpg)
Pejorative manifolds of 3-polynomials
The wings: a1= -s-2t a2= 2st+t2
a3= -st2
The edge: a1 = -3s a2 = 3s2
a3 = -s3
General form ofpejorative manifolds
u = G(z)
![Page 18: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/18.jpg)
W. Kahan, Conserving confluence curbs ill-condition, 1972
1. Ill-condition occurs when a polynomial is near a pejorative manifold.
2. A small “drift” by a polynomial on that pejorative manifolddoes not cause large forward error to the multiple roots, except
3. If a multiple root is sensitive to small perturbation on thepejorative manifold, then the polynomial is near a pejorativesubmanifold of higher multiplicity.
Ill-condition is caused by solving polynomialequations on a wrong manifold
![Page 19: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/19.jpg)
Pejorative manifolds of 3-polynomials
The wings: a1= -s-2t a2= 2st+t2
a3= -st2
The edge: a1 = -3s a2 = 3s2
a3 = -s3
![Page 20: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/20.jpg)
Given a polynomial p(x) = xn + a1 xn-1+...+an-1 x + an
The wrong question: Find ( z1, ..., zn ) such thatp(x) = ( x - z1 )( x - z2 ) ... ( x - zn )
because you are asking for simple roots!
/ / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / /
The right question:Find distinct z1, ..., zm such thatp(x) = ( x - z1 1 x - z2 )2 ... ( x - zm )m
m = n, m < n
do it on the pejorative manifold!
![Page 21: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/21.jpg)
For ill-conditioned polynomial p(x)= xn + a1 xn-1+...+an-1 x + an ~ a = (a1 , ..., an-1 , an )
The objective: find u*=G(z*) that is nearest to p(x)~a
![Page 22: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/22.jpg)
Let ( x - z1 1 x - z2 )2 ... ( x - zm )m =
xn + g1 ( z1, ..., zm ) xn-1+...+gn-1 ( z1, ..., zm ) x + gn ( z1, ..., zm )
Then, p(x) = ( x - z1 1 x - z2 )2 ... ( x - zm )m <==>
g1 ( z1, ..., zm ) =a1
g2( z1, ..., zm ) =a2
... ... ...
gn ( z1, ..., zm ) =an
I.e. An over determined polynomial system
G(z) = a
(m<n)n
m
![Page 23: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/23.jpg)
tangent plane P0 :
u = G(z0)+J(z
0)(z- z0)
initial iterate
u0 =
G(z
0 )
pejorative root
u* =
G(z
* )
The polynomiala
Project to tangent plane
u 1 = G(z 0
)+J(z 0)(z 1
- z 0)
~
new iterate
u1 =
G(z
1 )
Pejora
tive m
anifo
ld
u = G
( z )
Solve G( z ) = a for nonlinear least squares solution z=z*
Solve G(z0)+J(z0)( z - z0 ) = a for linear least squares solution z = z1
G(z0)+J(z0)( z - z0 ) = aJ(z0)( z - z0 ) = - [G(z0) - a ] z1 = z0 - [J(z0)+] [G(z0) - a]
![Page 24: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/24.jpg)
Theorem: Let u*=G(z*) be nearest to p(x)~a, if1. z*=(z*1, ..., z*m) with z*1, ..., z*m distinct;2. z0 is sufficiently close to z*;3. a is sufficiently close to u*
then the iteration converges with a linear rate.
Further assume that a = u* , then the convergence is quadratic.
Theorem: If z=(z1, ..., zm) with z1, ..., zm distinct, then the Jacobian J(z) of G(z) is of full rank.
zi+1=zi - J(zi )+[ G(zi )-a ], i=0,1,2 ...
![Page 25: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/25.jpg)
Calculation of G(z) and J(z) ( x - z1 1 x - z2 )2 ... ( x - zm )m =
xn + g1 ( z1, ..., zm ) xn-1+...+gn-1 ( z1, ..., zm ) x + gn ( z1, ..., zm )
Explicit polynomail formula gj ( z1, ..., zm ) isneither necessary nor desirable.
G(z) and every column of J(z) are calculated viasimple numerical procedures, as key componentsof the algorithm.
![Page 26: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/26.jpg)
The conventional condition number
Let f(x*) = 0
f(xg(xx(0) = x*
f’(xx’g(xg’(xx’/ / / / / / / / / / / / -----x*
-----x*
x’(0) = - g(x*)/ f’(x*)
xx* - g(x*)/ f’(x*)h.o.t.
| xx* | < | 1 / f’(x*)| .| g(x*) | + h.o.t.
Condition number | 1 / f’(x*)| = infinity at multiple a root
![Page 27: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/27.jpg)
The “pejorative” condition number
u = G(y)
v = G(z)
|| u - v ||2 = backward error|| y - z ||2 = forward error
u - v = G(y) - G(z) = J(z) (y - z) + h.o.t.
|| u - v ||2 = || J(z) (y - z) ||2 > y - z ||2
y - z ||2 < (1/) u - v ||2
1/ is the pejorative condition numberwhere is the smallest singular value of J(z) .
![Page 28: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/28.jpg)
Example 1 (x-1.0 )100 = 0
To make the problem interesting:round the coefficients to 5 digits
Step iterates ... ... ... ... 43 1.007 44 1.001 45 1.0001 46 1.0000003 47 .999999998
conventional condition: infinity pejorative condition: 0.0017
Is it ill-conditioned?
![Page 29: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/29.jpg)
Example 2 (x-0.9)18(x-1.0)10(x-1.1)16 = 0 Step z1 z2 z3
--------------------------------------------------------------------0 .92 .95 1.121 .87 1.05 1.102 .92 .95 1.113 .88 1.01 1.104 .90 .97 1.125 .901 .992 1.1016 .89993 .9998 1.10027 .9000003 .999998 1.10000078 .899999999997 .999999999991 1.1000000000099 .900000000000006 .99999999999997 1.10000000000001
forward error: 6 x 10-15
backward error: 8 x 10-16
Pejorative condition: 58
Even clustered multiple roots are pejoratively well conditioned
![Page 30: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/30.jpg)
Multiplicity pejorative roots backward error pejorative condition structure------------------------------------------------------------------------------------------------------------- [1,1,...1] erratic .0000000000000006 1390704851032436. [18,10,16] (.9000, 1.0000, 1.1000) .0000000000000008 58.2 [17,11,16] (.8980, .9934, 1.1006) .0000005 51.9 [14,16,14] (.8890, .9892, 1.1090) .000006 29.0 [10,24,10] (.8711, .9906, 1.1316) .00001 25.8 [ 2,40, 2] (.7393, .9917, 1.3282) .0001 23.6 [ 1,43] (.6559, 1.0053 ) .004 12.9 [44] ( .9925 ) .04 .0058
If you are on a wrong manifold ...
![Page 31: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/31.jpg)
Example 3 (x-.3-.6i)100 (x-.1-.7i) 200 (x - .7-.5i) 300 (x-.3-.4i) 400 =0
Scary enough? Round coefficients to 6 digits.
Z1 z2 z3 z4
.289 +.601i .100 +.702i .702 +.498i .301 +.399i
.309 +.602i .097 +.698i .698 +.499i .299 +.401i
.293 +.596i .101 +.7003i .7002 +.5005i .3007 +.4003i
.300005 +.600006i .099998 +.6999992i .69999992+.4999993i .2999992 +.3999992i
.3000002+.60000005i .09999995+.69999998i .69999997+.49999998i .29999997+.400000002i
Roots are correct up to 7 digits!
Pejorative condition: 0.58
![Page 32: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/32.jpg)
Example 4: The Wilkinson polynomial
p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ...
There are 605 manifolds in total. It is near some manifolds, but which ones?
Multiplicity backward error condition Estimated structure number error------------------------------------------------------------------------[1,1,1,1,1,1,1,1,1...,1] .000000000000003 550195997640164 1.6[1,1,1,1,2,2,2,4,2,2,2] .000000003 29269411 .09[1,1,1,2,3,4,5,3] .0000001 33563 .003[1,1,2,3,4,6,3] .000001 6546 .007[1,1,2,5,7,4] .000005 812 .004[1,2,5,7,5] .00004 198 .008[1,3,8,8] .0002 25 .005[2,8,10] .003 6 .02[5,15] .04 1 .04[20] .9 .2 .2
What are the roots of the Wilkinson polynomial?
Choose your poison!
![Page 33: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/33.jpg)
If the estimated error is the sole criterion...
Roots multiplicity---------------------------- .9999012 * 2.0012667 * 2.9758883 * 4.3848008 ** 6.9548807 ***10.3585762 ****15.0276741 *****19.2719863 ***
The polynomial with these roots matches W. Polynomial for 6 digitsThe condition 33563 is manageable.
![Page 34: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/34.jpg)
Summary of the simultaneous method:- Given a polynomial p(x) ~ a, decide the structure ( x - z1 1 x - z2 )2 ... ( x - zm )m
implicitly define the coefficient operator G(z) and its Jacobian J(z)
- Have an initial guess z0 to the “pejorative root”
- Start the nonlinear least squares iteration zi+1=zi - J(zi )+[ G(zi )-a ], i=0,1,2 ...
- If it converges verify (i) the backward error || G(z* )-a || (ii) the pejorative condition 1/
- Accept the pejorative root if both are tiny
![Page 35: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/35.jpg)
Isolated multiple roots:
The polynomial p(x) has an m-fold root p(x) = ( x-z* )m . q(x-z*)
The clue:p(z*) = p’(z*) = p”(z*) = ... = p(m-1)(z*) = 0 p(m)(z*) =/= 0
F
t1t2
...tm-1
x
=
p(x) - t1 p(m)(x)
p’(x) - t2 p(m)(x)
... ... ...
p(m-1)( x) - tm-1 p(m)(x)
p(m-1)( x)
![Page 36: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/36.jpg)
F
t1t2
...tm-1
x
=
p(x) - t1 p(m)(x)
p’(x) - t2 p(m)(x)
... ... ...
p(m-1)( x) - tm-1 p(m)(x)
p(m-1)( x)
DF
t1t2
...tm-1
x
= diag(-p(m)(x), -p(m)(x),..., -p(m)(x), p(m)(x) )
z* is an m-fold root of p <==> (0,0,..., z* ) is a simple zero of F
![Page 37: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/37.jpg)
Theorem: Let p(x) = [( x-z* )m . q(x-z*)] + [m-1(x-z* )m-1 + ... + 1(x-z* ) + 0] Then F has a simple zero
t2
...tm-1
z
t1
=
0...0
z*
0
+ [1/q(0)]
1
...m-2
m-1/m
0
z = z* + [1/(mq(0))] [m-1]
![Page 38: Perfidious Polynomials and Elusive Roots](https://reader035.vdocuments.us/reader035/viewer/2022070401/568136df550346895d9e7a8e/html5/thumbnails/38.jpg)
Conclusion
1. Ill-condition is cause by a wrong “identity”
4. To calculate multiple roots, one has to figure out the multiplicity structure (how?)
2. Multiple roots are pejoratively well conditioned, thereby tractable.
3. Extension of machine precision is not needed,a change in computing concept is.
Question: Can Jordan Canonical form be computed?