ordinary differential equations i - computer...
TRANSCRIPT
Introduction Initial Value Problems Theory Model Equations Simple Integration
Ordinary Differential Equations I
CS 205A:Mathematical Methods for Robotics, Vision, and Graphics
Justin Solomon
CS 205A: Mathematical Methods Ordinary Differential Equations I 1 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Theme of Last Few Weeks
The unknown is an entirefunction f .
CS 205A: Mathematical Methods Ordinary Differential Equations I 2 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
New Twist
So far:f (or its derivative/integral) known
at isolated points
Instead:Optimize properties of f
CS 205A: Mathematical Methods Ordinary Differential Equations I 3 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
New Twist
So far:f (or its derivative/integral) known
at isolated points
Instead:Optimize properties of f
CS 205A: Mathematical Methods Ordinary Differential Equations I 3 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Example Problems
I Approximate f0 with f but make it smoother
(or sharper!)
I Simulate some dynamical or physical
relationship as f (t) where t is time
I Approximate f0 with f but transfer
properties of g0
CS 205A: Mathematical Methods Ordinary Differential Equations I 4 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Example Problems
I Approximate f0 with f but make it smoother
(or sharper!)
I Simulate some dynamical or physical
relationship as f (t) where t is time
I Approximate f0 with f but transfer
properties of g0
CS 205A: Mathematical Methods Ordinary Differential Equations I 4 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Example Problems
I Approximate f0 with f but make it smoother
(or sharper!)
I Simulate some dynamical or physical
relationship as f (t) where t is time
I Approximate f0 with f but transfer
properties of g0
CS 205A: Mathematical Methods Ordinary Differential Equations I 4 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Today: Initial Value Problems
Find f (t) : R→ Rn
Satisfying F [t, f (t), f ′(t), f ′′(t), . . . , f (k)(t)] = 0
Given f (0), f ′(0), f ′′(0), . . . , f (k−1)(0)
Example of canonical form (on board):
y′′ = ty′ cos y
CS 205A: Mathematical Methods Ordinary Differential Equations I 5 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Today: Initial Value Problems
Find f (t) : R→ Rn
Satisfying F [t, f (t), f ′(t), f ′′(t), . . . , f (k)(t)] = 0
Given f (0), f ′(0), f ′′(0), . . . , f (k−1)(0)
Example of canonical form (on board):
y′′ = ty′ cos y
CS 205A: Mathematical Methods Ordinary Differential Equations I 5 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Most Famous Example
~F = m~aNewton’s second law
~F (t, ~x, ~x′) usual expression of force
n particles =⇒ simulation in R3n
CS 205A: Mathematical Methods Ordinary Differential Equations I 6 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Most Famous Example
~F = m~aNewton’s second law
~F (t, ~x, ~x′) usual expression of force
n particles =⇒ simulation in R3n
CS 205A: Mathematical Methods Ordinary Differential Equations I 6 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Most Famous Example
~F = m~aNewton’s second law
~F (t, ~x, ~x′) usual expression of force
n particles =⇒ simulation in R3n
CS 205A: Mathematical Methods Ordinary Differential Equations I 6 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Protein Folding
http://www.sciencedaily.com/releases/2012/11/121122152910.htm
CS 205A: Mathematical Methods Ordinary Differential Equations I 7 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Gradient Descent
min~xE(~x)
=⇒ ~xi+1 = ~xi − h∇E(~xi)
=⇒h→0
d~x
dt= −∇E(~x)
CS 205A: Mathematical Methods Ordinary Differential Equations I 8 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Gradient Descent
min~xE(~x)
=⇒ ~xi+1 = ~xi − h∇E(~xi)
=⇒h→0
d~x
dt= −∇E(~x)
CS 205A: Mathematical Methods Ordinary Differential Equations I 8 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Gradient Descent
min~xE(~x)
=⇒ ~xi+1 = ~xi − h∇E(~xi)
=⇒h→0
d~x
dt= −∇E(~x)
CS 205A: Mathematical Methods Ordinary Differential Equations I 8 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Crowd Simulation
http://video.wired.com/watch/building-a-better-zombie-wwz-exclusive
http://gamma.cs.unc.edu/DenseCrowds/
CS 205A: Mathematical Methods Ordinary Differential Equations I 9 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Examples of ODEs
I y′ = 1 + cos t: solved by integrating both sides
I y′ = ay: linear in y, no dependence on t
I y′ = ay + et: time and value-dependent
I y′′ + 3y′ − y = t: multiple derivatives of y
I y′′ sin y = ety′: nonlinear in y and t
CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Examples of ODEs
I y′ = 1 + cos t: solved by integrating both sides
I y′ = ay: linear in y, no dependence on t
I y′ = ay + et: time and value-dependent
I y′′ + 3y′ − y = t: multiple derivatives of y
I y′′ sin y = ety′: nonlinear in y and t
CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Examples of ODEs
I y′ = 1 + cos t: solved by integrating both sides
I y′ = ay: linear in y, no dependence on t
I y′ = ay + et: time and value-dependent
I y′′ + 3y′ − y = t: multiple derivatives of y
I y′′ sin y = ety′: nonlinear in y and t
CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Examples of ODEs
I y′ = 1 + cos t: solved by integrating both sides
I y′ = ay: linear in y, no dependence on t
I y′ = ay + et: time and value-dependent
I y′′ + 3y′ − y = t: multiple derivatives of y
I y′′ sin y = ety′: nonlinear in y and t
CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Examples of ODEs
I y′ = 1 + cos t: solved by integrating both sides
I y′ = ay: linear in y, no dependence on t
I y′ = ay + et: time and value-dependent
I y′′ + 3y′ − y = t: multiple derivatives of y
I y′′ sin y = ety′: nonlinear in y and t
CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Reasonable Assumption
Explicit ODEAn ODE is explicit if can be written in the form
f (k)(t) = F [t, f (t), f ′(t), f ′′(t), . . . , f (k−1)(t)].
Otherwise need to do root-finding!
CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Reasonable Assumption
Explicit ODEAn ODE is explicit if can be written in the form
f (k)(t) = F [t, f (t), f ′(t), f ′′(t), . . . , f (k−1)(t)].
Otherwise need to do root-finding!
CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Reduction to First Order
f (k)(t) = F [t, f(t), f ′(t), f ′′(t), . . . , f (k−1)(t)]
d
dt
f0(t)f1(t)
...fk−2(t)fk−1(t)
=
f1(t)f2(t)
...fk−1(t)
F [t, f0(t), f1(t), . . . , fk−1(t)]
CS 205A: Mathematical Methods Ordinary Differential Equations I 12 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Reduction to First Order
f (k)(t) = F [t, f(t), f ′(t), f ′′(t), . . . , f (k−1)(t)]
d
dt
f0(t)f1(t)
...fk−2(t)fk−1(t)
=
f1(t)f2(t)
...fk−1(t)
F [t, f0(t), f1(t), . . . , fk−1(t)]
CS 205A: Mathematical Methods Ordinary Differential Equations I 12 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Example
y′′′ = 3y′′ − 2y′ + y
d
dt
y
z
w
=
0 1 0
0 0 1
1 −2 3
y
z
w
CS 205A: Mathematical Methods Ordinary Differential Equations I 13 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Example
y′′′ = 3y′′ − 2y′ + y
d
dt
y
z
w
=
0 1 0
0 0 1
1 −2 3
y
z
w
CS 205A: Mathematical Methods Ordinary Differential Equations I 13 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Time Dependence
Visualization: Slope field
CS 205A: Mathematical Methods Ordinary Differential Equations I 14 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Time Dependence
Visualization: Slope field
CS 205A: Mathematical Methods Ordinary Differential Equations I 14 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Autonomous ODE
~y′ = F [~y]
No dependence of F on t
g′(t) =
(f ′(t)
g′(t)
)=
(F [f (t), g(t)]
1
)
CS 205A: Mathematical Methods Ordinary Differential Equations I 15 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Autonomous ODE
~y′ = F [~y]
No dependence of F on t
g′(t) =
(f ′(t)
g′(t)
)=
(F [f (t), g(t)]
1
)
CS 205A: Mathematical Methods Ordinary Differential Equations I 15 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Visualization: Phase Space
θ′′ = − sin θCS 205A: Mathematical Methods Ordinary Differential Equations I 16 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Visualization: Traces
x′ = σ(y − x), y = x(ρ− z)− y, z′ = xy − βz
CS 205A: Mathematical Methods Ordinary Differential Equations I 17 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Existence and Uniqueness
dy
dt=
2y
t
Two cases:y(0) = 0, y(0) 6= 0
CS 205A: Mathematical Methods Ordinary Differential Equations I 18 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Existence and Uniqueness
Theorem: Local existence and uniqueness
Suppose F is continuous and Lipschitz, that is,‖F [~y]− F [~x]‖2 ≤ L‖~y − ~x‖2 for some fixed L ≥ 0.Then, the ODE f ′(t) = F [f(t)] admits exactly onesolution for all t ≥ 0 regardless of initial conditions.
CS 205A: Mathematical Methods Ordinary Differential Equations I 19 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Linearization of 1D ODEs
y′ = F [y]
−→ y′ = ay + b
−→ y′ = ay
CS 205A: Mathematical Methods Ordinary Differential Equations I 20 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Linearization of 1D ODEs
y′ = F [y]
−→ y′ = ay + b
−→ y′ = ay
CS 205A: Mathematical Methods Ordinary Differential Equations I 20 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Linearization of 1D ODEs
y′ = F [y]
−→ y′ = ay + b
−→ y′ = ayCS 205A: Mathematical Methods Ordinary Differential Equations I 20 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay
=⇒ y(t) = Ceat
CS 205A: Mathematical Methods Ordinary Differential Equations I 21 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay
=⇒ y(t) = Ceat
CS 205A: Mathematical Methods Ordinary Differential Equations I 21 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Stability: Visualization
y′ = ay
CS 205A: Mathematical Methods Ordinary Differential Equations I 22 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Three Cases
y′ = ay, y(t) = Ceat
1. a = 0: Stable
2. a < 0: Stable; solutions get closer
3. a > 0: Unstable; mistakes in initial data
amplified
CS 205A: Mathematical Methods Ordinary Differential Equations I 23 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Three Cases
y′ = ay, y(t) = Ceat
1. a = 0: Stable
2. a < 0: Stable; solutions get closer
3. a > 0: Unstable; mistakes in initial data
amplified
CS 205A: Mathematical Methods Ordinary Differential Equations I 23 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Three Cases
y′ = ay, y(t) = Ceat
1. a = 0: Stable
2. a < 0: Stable; solutions get closer
3. a > 0: Unstable; mistakes in initial data
amplified
CS 205A: Mathematical Methods Ordinary Differential Equations I 23 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Intuition for Stability
An unstable ODE magnifies mistakes in the
initial conditions y(0).
CS 205A: Mathematical Methods Ordinary Differential Equations I 24 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Multidimensional Case
~y′ = A~y,A~yi = λi~yi
~y(0) =∑i
ci~yi
=⇒ ~y(t) =∑i
cieλit~yi
Stability depends on maxi λi.
CS 205A: Mathematical Methods Ordinary Differential Equations I 25 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Multidimensional Case
~y′ = A~y,A~yi = λi~yi
~y(0) =∑i
ci~yi
=⇒ ~y(t) =∑i
cieλit~yi
Stability depends on maxi λi.
CS 205A: Mathematical Methods Ordinary Differential Equations I 25 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Multidimensional Case
~y′ = A~y,A~yi = λi~yi
~y(0) =∑i
ci~yi
=⇒ ~y(t) =∑i
cieλit~yi
Stability depends on maxi λi.
CS 205A: Mathematical Methods Ordinary Differential Equations I 25 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Integration Strategies
Given ~yk at time tk, generate ~yk+1 assuming
~y′ = F [~y].
CS 205A: Mathematical Methods Ordinary Differential Equations I 26 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Forward Euler
~yk+1 = ~yk + hF [~yk]
I Explicit method
I O(h2) localized truncation error
I O(h) global truncation error;
“first order accurate”
CS 205A: Mathematical Methods Ordinary Differential Equations I 27 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Forward Euler: Stability
CS 205A: Mathematical Methods Ordinary Differential Equations I 28 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay −→ yk+1 = (1 + ah)yk
For a < 0, stable when h < 2|a|.
CS 205A: Mathematical Methods Ordinary Differential Equations I 29 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Backward Euler
~yk+1 = ~yk + hF [~yk+1]
I Implicit method
I O(h2) localized truncation error
I O(h) global truncation error;
“first order accurate”
CS 205A: Mathematical Methods Ordinary Differential Equations I 30 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Forward Euler: Stability
CS 205A: Mathematical Methods Ordinary Differential Equations I 31 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay −→ yk+1 =1
1− ahyk
Unconditionally stable!
But this has nothing to do with accuracy.
Good for stiff equations.
CS 205A: Mathematical Methods Ordinary Differential Equations I 32 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay −→ yk+1 =1
1− ahyk
Unconditionally stable!
But this has nothing to do with accuracy.
Good for stiff equations.
CS 205A: Mathematical Methods Ordinary Differential Equations I 32 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Model Equation
y′ = ay −→ yk+1 =1
1− ahyk
Unconditionally stable!
But this has nothing to do with accuracy.
Good for stiff equations.
CS 205A: Mathematical Methods Ordinary Differential Equations I 32 / 32
Introduction Initial Value Problems Theory Model Equations Simple Integration
Forward and Backward Euleron Linear ODE
~y′ = A~y
I Forward Euler: ~yk+1 = (I + hA)~ykI Backward Euler: ~yk+1 = (I − hA)−1~yk
Next
CS 205A: Mathematical Methods Ordinary Differential Equations I 33 / 32