![Page 1: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/1.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Nonlinear Systems
CS 205A:Mathematical Methods for Robotics, Vision, and Graphics
Justin Solomon
CS 205A: Mathematical Methods Nonlinear Systems 1 / 24
![Page 2: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/2.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Part III: Nonlinear Problems
Not all numerical problemscan be solved with \ in
Matlab.
CS 205A: Mathematical Methods Nonlinear Systems 2 / 24
![Page 3: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/3.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Question
Have we already seen anonlinear problem?
minimize ‖A~x‖2such that ‖~x‖2 = 1←− nonlinear!
CS 205A: Mathematical Methods Nonlinear Systems 3 / 24
![Page 4: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/4.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Question
Have we already seen anonlinear problem?
minimize ‖A~x‖2such that ‖~x‖2 = 1←− nonlinear!
CS 205A: Mathematical Methods Nonlinear Systems 3 / 24
![Page 5: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/5.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Root-Finding Problem
Given: f : Rn→ Rm
Find: ~x∗ with f (~x∗) = ~0
CS 205A: Mathematical Methods Nonlinear Systems 4 / 24
![Page 6: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/6.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Issue: Regularizing Assumptions
f (x) =
{−1 x ≤ 01 x > 0
f (x) =
{−1 x ∈ Q1 otherwise
CS 205A: Mathematical Methods Nonlinear Systems 5 / 24
![Page 7: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/7.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Issue: Regularizing Assumptions
f (x) =
{−1 x ≤ 01 x > 0
f (x) =
{−1 x ∈ Q1 otherwise
CS 205A: Mathematical Methods Nonlinear Systems 5 / 24
![Page 8: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/8.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Typical Regularizing AssumptionsContinuous
f(~x)→ f(~y) as ~x→ ~y
Lipschitz
‖f(~x)− f(~y)‖ ≤ C‖~x− ~y‖
Differentiable
Df(~x) exists for all ~x
Ck
k derivatives exist and are continuous
CS 205A: Mathematical Methods Nonlinear Systems 6 / 24
![Page 9: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/9.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Typical Regularizing AssumptionsContinuous
f(~x)→ f(~y) as ~x→ ~y
Lipschitz
‖f(~x)− f(~y)‖ ≤ C‖~x− ~y‖
Differentiable
Df(~x) exists for all ~x
Ck
k derivatives exist and are continuous
CS 205A: Mathematical Methods Nonlinear Systems 6 / 24
![Page 10: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/10.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Typical Regularizing AssumptionsContinuous
f(~x)→ f(~y) as ~x→ ~y
Lipschitz
‖f(~x)− f(~y)‖ ≤ C‖~x− ~y‖
Differentiable
Df(~x) exists for all ~x
Ck
k derivatives exist and are continuous
CS 205A: Mathematical Methods Nonlinear Systems 6 / 24
![Page 11: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/11.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Typical Regularizing AssumptionsContinuous
f(~x)→ f(~y) as ~x→ ~y
Lipschitz
‖f(~x)− f(~y)‖ ≤ C‖~x− ~y‖
Differentiable
Df(~x) exists for all ~x
Ck
k derivatives exist and are continuous
CS 205A: Mathematical Methods Nonlinear Systems 6 / 24
![Page 12: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/12.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Today
f : R→ RCS 205A: Mathematical Methods Nonlinear Systems 7 / 24
![Page 13: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/13.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Property of Continuous Functions
Intermediate Value TheoremSuppose f : [a, b]→ R is continuous. Suppose
f (x) < u < f (y). Then, there exists z between
x and y such that f (z) = u.
Used in Homework 1!
CS 205A: Mathematical Methods Nonlinear Systems 8 / 24
![Page 14: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/14.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Property of Continuous Functions
Intermediate Value TheoremSuppose f : [a, b]→ R is continuous. Suppose
f (x) < u < f (y). Then, there exists z between
x and y such that f (z) = u.
Used in Homework 1!
CS 205A: Mathematical Methods Nonlinear Systems 8 / 24
![Page 15: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/15.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Reasonable Starting Point
f (`) · f (r) < 0
CS 205A: Mathematical Methods Nonlinear Systems 9 / 24
![Page 16: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/16.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Bisection Algorithm
1. Compute c = `+r/2.
2. If f (c) = 0, return x∗ = c.
3. If f (`) · f (c) < 0, take r ← c. Otherwise
take `← c.
4. Return to step ?? until |r − `| < ε; then
return c.
CS 205A: Mathematical Methods Nonlinear Systems 10 / 24
![Page 17: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/17.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Two Important Questions
1. Does it converge?
Yes! Unconditionally.
2. How quickly?
CS 205A: Mathematical Methods Nonlinear Systems 11 / 24
![Page 18: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/18.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Two Important Questions
1. Does it converge?Yes! Unconditionally.
2. How quickly?
CS 205A: Mathematical Methods Nonlinear Systems 11 / 24
![Page 19: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/19.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Two Important Questions
1. Does it converge?Yes! Unconditionally.
2. How quickly?
CS 205A: Mathematical Methods Nonlinear Systems 11 / 24
![Page 20: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/20.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence Analysis
Examine Ek with|xk − x∗| < Ek.
CS 205A: Mathematical Methods Nonlinear Systems 12 / 24
![Page 21: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/21.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Bisection: Linear Convergence
Ek+1 ≤ 12Ek
for Ek ≡ |rk − `k|
CS 205A: Mathematical Methods Nonlinear Systems 13 / 24
![Page 22: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/22.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Fixed Points
g(x∗) = x∗
Question:Same as root-finding?
CS 205A: Mathematical Methods Nonlinear Systems 14 / 24
![Page 23: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/23.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Fixed Points
g(x∗) = x∗
Question:Same as root-finding?
CS 205A: Mathematical Methods Nonlinear Systems 14 / 24
![Page 24: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/24.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Simple Strategy
xk+1 = g(xk)
CS 205A: Mathematical Methods Nonlinear Systems 15 / 24
![Page 25: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/25.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence Criterion
Ek ≡ |xk − x∗|= |g(xk−1)− g(x∗)|
≤ C|xk−1 − x∗|if g is Lipschitz
= CEk−1
CS 205A: Mathematical Methods Nonlinear Systems 16 / 24
![Page 26: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/26.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence Criterion
Ek ≡ |xk − x∗|= |g(xk−1)− g(x∗)|≤ C|xk−1 − x∗|
if g is Lipschitz
= CEk−1CS 205A: Mathematical Methods Nonlinear Systems 16 / 24
![Page 27: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/27.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Alternative Criterion
Lipschitz near x∗ with goodstarting point.
e.g. C1 with |g′(x∗)| < 1
CS 205A: Mathematical Methods Nonlinear Systems 17 / 24
![Page 28: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/28.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Alternative Criterion
Lipschitz near x∗ with goodstarting point.
e.g. C1 with |g′(x∗)| < 1
CS 205A: Mathematical Methods Nonlinear Systems 17 / 24
![Page 29: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/29.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence Rate of Fixed Point
When it converges...Always linear (why?)
Often quadratic!
CS 205A: Mathematical Methods Nonlinear Systems 18 / 24
![Page 30: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/30.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence Rate of Fixed Point
When it converges...Always linear (why?)
Often quadratic!
CS 205A: Mathematical Methods Nonlinear Systems 18 / 24
![Page 31: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/31.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Newton’s Method
xk+1 = xk −f (xk)
f ′(xk)
Fixed point iteration on
g(x) ≡ x− f (x)
f ′(x)
CS 205A: Mathematical Methods Nonlinear Systems 19 / 24
![Page 32: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/32.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Newton’s Method
xk+1 = xk −f (xk)
f ′(xk)
Fixed point iteration on
g(x) ≡ x− f (x)
f ′(x)CS 205A: Mathematical Methods Nonlinear Systems 19 / 24
![Page 33: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/33.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence of Newton
Simple Root
A root x∗ with f ′(x∗) 6= 0.
Quadratic convergence in this case!
Higher-order approximations?
CS 205A: Mathematical Methods Nonlinear Systems 20 / 24
![Page 34: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/34.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence of Newton
Simple Root
A root x∗ with f ′(x∗) 6= 0.
Quadratic convergence in this case!
Higher-order approximations?
CS 205A: Mathematical Methods Nonlinear Systems 20 / 24
![Page 35: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/35.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Convergence of Newton
Simple Root
A root x∗ with f ′(x∗) 6= 0.
Quadratic convergence in this case!
Higher-order approximations?
CS 205A: Mathematical Methods Nonlinear Systems 20 / 24
![Page 36: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/36.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Issue
Differentiation is hard!
CS 205A: Mathematical Methods Nonlinear Systems 21 / 24
![Page 37: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/37.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Secant Method
xk+1 = xk −f (xk)(xk − xk−1)
f (xk)− f (xk−1)
Trivia:Converges at rate 1+
√5
2 ≈ 1.6180339887 . . .
(“Golden Ratio”)
CS 205A: Mathematical Methods Nonlinear Systems 22 / 24
![Page 38: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/38.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Secant Method
xk+1 = xk −f (xk)(xk − xk−1)
f (xk)− f (xk−1)
Trivia:Converges at rate 1+
√5
2 ≈ 1.6180339887 . . .
(“Golden Ratio”)
CS 205A: Mathematical Methods Nonlinear Systems 22 / 24
![Page 39: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/39.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Secant Method
xk+1 = xk −f (xk)(xk − xk−1)
f (xk)− f (xk−1)
Trivia:Converges at rate 1+
√5
2 ≈ 1.6180339887 . . .
(“Golden Ratio”)
CS 205A: Mathematical Methods Nonlinear Systems 22 / 24
![Page 40: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/40.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Hybrid Methods
Want: Convergence rate of secant/Newton with
convergence guarantees of bisection
e.g. Dekker’s Method: Take secant step if it is
in the bracket, bisection step otherwise
CS 205A: Mathematical Methods Nonlinear Systems 23 / 24
![Page 41: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/41.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Hybrid Methods
Want: Convergence rate of secant/Newton with
convergence guarantees of bisection
e.g. Dekker’s Method: Take secant step if it is
in the bracket, bisection step otherwise
CS 205A: Mathematical Methods Nonlinear Systems 23 / 24
![Page 42: Nonlinear Systems - Computer Graphics · CS 205A: Mathematical Methods Nonlinear Systems 8 / 24 Nonlinearity Root- nding Bisection Fixed Point Iteration Newton’s Method Secant Method](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec50922e9f9bc484151f5f4/html5/thumbnails/42.jpg)
Nonlinearity Root-finding Bisection Fixed Point Iteration Newton’s Method Secant Method Conclusion
Single-Variable Conclusion
I Unlikely to solve exactly, so we settle for
iterative methodsI Must check that method converges at allI Convergence rates:
I Linear: Ek+1 ≤ CEk for some 0 ≤ C < 1I Superlinear: Ek+1 ≤ CEr
k for some r > 1I Quadratic: r = 2I Cubic: r = 3
I Time per iteration also important
Next
CS 205A: Mathematical Methods Nonlinear Systems 24 / 24