optimal control and maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · optimal control and...
TRANSCRIPT
![Page 1: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/1.jpg)
Optimal control and Maximum principle
Daniel Wachsmuth, RICAM Linz
EMS school Bedlewo Bedlewo, 12.10.2010
![Page 2: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/2.jpg)
Content
1. Model problem
2. Adjoint equation
3. Maximum principle
4. Numerical algorithms
5. Flow control
EMS school Bedlewo Daniel Wachsmuth
![Page 3: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/3.jpg)
Rocket car 1
Objective: Reach target as fast as possible.
Control: Acceleration.
Constraints: Stop at target, control constraints.
EMS school Bedlewo Daniel Wachsmuth
![Page 4: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/4.jpg)
THE model 2
Quantities: t .. time, x(t) ∈ R position, u(t) ∈ [−1,+1] control,
m > 0 mass, x0 6= 0 initial position
Equation of motion:
m x ′′(t) = u(t)
x(0) = x0
EMS school Bedlewo Daniel Wachsmuth
![Page 5: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/5.jpg)
THE model 2
Quantities: t .. time, x(t) ∈ R position, u(t) ∈ [−1,+1] control,
m > 0 mass, x0 6= 0 initial position
Equation of motion:
m x ′′(t) = u(t)
x(0) = x0
Optimal control problem:
Minimize T subject to
• Equation of motion
• x(T ) = 0, x ′(T ) = 0
• |u(t)| ≤ 1
EMS school Bedlewo Daniel Wachsmuth
![Page 6: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/6.jpg)
THE model 2
Quantities: t .. time, x(t) ∈ R position, u(t) ∈ [−1,+1] control,
m > 0 mass, x0 6= 0 initial position
Equation of motion:
m x ′′(t) = u(t)
x(0) = x0
Optimal control problem:
Minimize T subject to
• Equation of motion
• x(T ) = 0, x ′(T ) = 0
• |u(t)| ≤ 1
Problem: Find a solution!
EMS school Bedlewo Daniel Wachsmuth
![Page 7: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/7.jpg)
Optimal control problem 3
Features:
• optimization problem
• some optimization variable are functions → infinite-dimensional
optimization
• differential equations (ode / pde)
• inequalities
EMS school Bedlewo Daniel Wachsmuth
![Page 8: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/8.jpg)
Generic optimal control problem 4
Minimize functional J given by
J(x, u, T ) :=
∫ T
0
f0(x(t), u(t), t) dt
subject to the ODE
x ′(t) = f (x(t), u(t)) a.e. on (0, T ),
initial and terminal conditions
x(0) = x0, x(T ) = z
control constraints
u(t) ⊂ U.
Setting f0 = 1 the case of the time-optimal problem is contained as
special case.
Unknowns: measurable functions u, x with u(t) ∈ Rm, x(t) ∈ Rn.
Problem: Find a solution!
EMS school Bedlewo Daniel Wachsmuth
![Page 9: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/9.jpg)
Characterization of optimal controls 5
Consider the simpler problem of minimizing f : R→ R,
min f (x), x ∈ R.
If f is differentiable, then every solution x satisfies
f ′(x) = 0.
Solve this equation to find candidates for solutions!
EMS school Bedlewo Daniel Wachsmuth
![Page 10: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/10.jpg)
Characterization of optimal controls 5
Consider the simpler problem of minimizing f : R→ R,
min f (x), x ∈ R.
If f is differentiable, then every solution x satisfies
f ′(x) = 0.
Solve this equation to find candidates for solutions!
What is f ′ in the optimal control problem??
EMS school Bedlewo Daniel Wachsmuth
![Page 11: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/11.jpg)
Content
1. Model problem
2. Adjoint equation
3. Maximum principle
4. Numerical algorithms
5. Flow control
EMS school Bedlewo Daniel Wachsmuth
![Page 12: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/12.jpg)
Directional derivative 6
Assumption: For every control function u there exists a uniquely
determined solution of the ODE x = x(u).
Simplification: No terminal constraint.
Task: Compute directional derivative Given control u0, x0, direction
h
J ′(x0, u0)h ≈1
ǫ(J(x(u0 + ǫh), u0 + ǫh)− J ′(x0, u0))
Disadvantages:
• Choice of ǫ
• Each evaluation of one difference quotient requires one (nonlinear)
ODE solve
EMS school Bedlewo Daniel Wachsmuth
![Page 13: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/13.jpg)
Derivatives 7
Chain rule:
d
duJ(x0, u0) = Jx(x
0, u0)d
dux(u0) + Ju(x
0, u0)
(Total) Directional derivative
d
duJ(x0, u0)h = Jx(x
0, u0)d
dux(u0)h + Ju(x
0, u0)h
The quantity z := d
dux(u0)h is the solution of the linearized ode
z ′ = fx(x0, u0)z + fu(x
0, u0)h, z(0) = 0
The quantity Jx(x0, u0) d
dux(u0)h is a dual product:
Jx(x0, u0)
d
dux(u0)h = 〈Jx(x
0, u0),d
dux(u0)h〉
EMS school Bedlewo Daniel Wachsmuth
![Page 14: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/14.jpg)
Derivatives 7
Chain rule:
d
duJ(x0, u0) = Jx(x
0, u0)d
dux(u0) + Ju(x
0, u0)
(Total) Directional derivative
d
duJ(x0, u0)h = Jx(x
0, u0)d
dux(u0)h + Ju(x
0, u0)h
The quantity z := d
dux(u0)h is the solution of the linearized ode
z ′ = fx(x0, u0)z + fu(x
0, u0)h, z(0) = 0
The quantity Jx(x0, u0) d
dux(u0)h is a dual product:
Jx(x0, u0)
d
dux(u0)h = 〈Jx(x
0, u0),d
dux(u0)h〉 =
⟨(
d
dux(u0)
)
∗
Jx(x0, u0), h
EMS school Bedlewo Daniel Wachsmuth
![Page 15: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/15.jpg)
Adjoint equation 8
How can we characterize
q :=
(
d
dux(u0)
)
∗
Jx(x0, u0)?
It turns out that
q = fu(x0, u0)T p,
where p solves the linear ODE [Recall J =R
f0(x, u)]
−p′(t) = fx (x0, u0)T p + f0,x(x
0, u0)T , p(T ) = 0.
Conclusion: Given (x0, u0) and p0. Then
d
duJ(x0, u0)h =
∫ T
0
pT fu(x0, u0)h + f0,u(x
0, u0)h
Advantage: One linear ODE needed to evaluate many directional
derivatives.
EMS school Bedlewo Daniel Wachsmuth
![Page 16: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/16.jpg)
Universal applicability 9
Adjoint equation:
−p′(t) = fx (x0, u0)T p + f0,x(x
0, u0)T , p(T ) = 0.
Properties:
• Linear in p
• Operator in the equation is the linearized and transposed (adjoint)
operator of the state equation
• Inhomogeneities originate from objective functional
• Nonzero data only where observation takes place
Advantage:
• One linear ODE solve to obtain all derivative information
• Works for many problems: PDE, shape optimization, etc
EMS school Bedlewo Daniel Wachsmuth
![Page 17: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/17.jpg)
10
We only wanted to evaluate f ′ ...
EMS school Bedlewo Daniel Wachsmuth
![Page 18: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/18.jpg)
Content
1. Model problem
2. Adjoint equation
3. Maximum principle
4. Numerical algorithms
5. Flow control
EMS school Bedlewo Daniel Wachsmuth
![Page 19: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/19.jpg)
Recall optimal control problem 11
Minimize functional J given by
J(x, u, T ) :=
∫ T
0
f0(x(t), u(t), t) dt
subject to the ODE
x ′(t) = f (x(t), u(t)) a.e. on (0, T ),
initial and terminal conditions
x(0) = x0, x(T ) = z
control constraints
u(t) ⊂ U.
Optimal control: (x∗, u∗, T ∗) is optimal if
J(x∗, u∗, T ∗) ≤ J(x, u, T )
for all admissible (x, u, T ).
EMS school Bedlewo Daniel Wachsmuth
![Page 20: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/20.jpg)
Pontryagin maximum principle 12
Define Hamilton-Function:
H(t, x, u, p, λ0) = pT f (x, u)− λ0f0(x, u, t)
Theorem: Let the functions f , f0 be continuous wrt (x, u, t) and
continuously differentiable wrt (t, u). Let U ⊂ Rm be given.
Let (x∗, u∗, T ∗) be optimal.
Then there exists pT ∗ ∈ Rn, λ0 ∈ R with (λ0, pT ∗) 6= 0, λ0 ≥ 0 such
that the following conditions are satisfied
• Adjoint equation
−p′(t) = fx (x∗(t), u∗(t), t)T p(t)− λ0f0,x(x
∗(t), u∗(t), t)T
p(T ∗) = pT ∗
• Maximum condition
H(t, x∗(t), u∗(t), p(t), λ0) = maxv∈UH(t, x∗(t), v , p(t), λ0) a.e. on (0, T ∗)
EMS school Bedlewo Daniel Wachsmuth
![Page 21: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/21.jpg)
Pontryagin maximum principle 13
• Maximum function
maxv∈UH(t, x∗(t), v , p(t), λ0)
is continuous on [0, T ∗] and satisfies at T ∗
maxv∈UH(T ∗, x∗(T ∗), v , p(T ∗), λ0) = 0.
In particular, the maximum condition is satisfied in all points of
left/right-continuity of u∗.
Message: The maximum principle generalizes the equation f ′(x) = 0.
Solve the system given by PMP to obtain solution candidates.
EMS school Bedlewo Daniel Wachsmuth
![Page 22: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/22.jpg)
Comments 14
• PMP is a necessary optimality condition: sometimes sufficient
(convex problems)
• Comparison to Kuhn-Tucker-type optimality conditions: Here no
derivatives wrt u needed!
• Role of λ0: Indicates (non-)degeneracy of constraints.
If one knows λ0 > 0 a-priori, the PMP-system can be scaled such
that λ0 = 1.
The point (λ0, pT ∗) = 0 is a solution of the PMP-system.
There is no equation to determine λ0 -in computations set λ0 = 1.
EMS school Bedlewo Daniel Wachsmuth
![Page 23: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/23.jpg)
Rocket car 15
Here: non-degenerate case λ0 > 0 if x0 6= 0.
Control u∗ is bang-bang
u∗(t) =
− sign(x0) if t ∈ (0, T ∗/2)
sign(x0) if t ∈ (T ∗/2, T ∗)
EMS school Bedlewo Daniel Wachsmuth
![Page 24: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/24.jpg)
Content
1. Model problem
2. Adjoint equation
3. Maximum principle
4. Numerical algorithms
5. Flow control
EMS school Bedlewo Daniel Wachsmuth
![Page 25: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/25.jpg)
PMP as boundary value problem 16
Suppose that f is continuously differentiable wrt u, and U = Rm (no
control constraints). Then the maximum condition implies
Hu(t, x∗(t), u∗(t), p(t), λ0) = 0,
which gives
−p(t)T fu(x∗(t), u∗(t), t) + f0,u(x
∗(t), u∗(t), t) = 0
If we can solve this equation for u∗ = u∗(p), then we can replace the
control u by the function u(p) in the state equation, and obtain a
boundary value problem for (x∗, p).
Solve with ODE-integration methods.
EMS school Bedlewo Daniel Wachsmuth
![Page 26: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/26.jpg)
Discretize-then-optimize 17
1. Discretize ODE by some discretization method (e.g. finite
differences)
2. Obtain finite-dimensional optimization problem
3. Use optimization software
EMS school Bedlewo Daniel Wachsmuth
![Page 27: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/27.jpg)
Optimize-then-discretize 18
1. Derive formulas for derivatives (gradient, Hessian) of optimal
control problem
2. Use (infinite-dimensional) optimization algorithm
3. Discretize and run the algorithm in finite-dimensional space
EMS school Bedlewo Daniel Wachsmuth
![Page 28: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/28.jpg)
Content
1. Model problem
2. Adjoint equation
3. Maximum principle
4. Numerical algorithms
5. Flow control
EMS school Bedlewo Daniel Wachsmuth
![Page 29: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/29.jpg)
High-lift configuration: Lift maximization 19
Maximize Lift
under the constraints
Navier-Stokes equations
Maximal drag
Control constraints
EMS school Bedlewo Daniel Wachsmuth
![Page 30: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/30.jpg)
Adjoint equation 20
Navier-Stokes equations:
yt − ν∆y + (y · ∇)y +∇p = 0
div y = 0
y |Γ = u
y(0) = y0.
Adjoint equations:
−λt − ν∆λ+ (∇y)Tλ− (y · ∇)λ+∇π = 0
divλ = 0
λ|Γ = ~e
λ(T ) = 0.
d
duJ(y 0, p0, u0)h =
∫
(0,T )×Γ
−(ν∂λ
∂n− πn)h
EMS school Bedlewo Daniel Wachsmuth
![Page 31: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/31.jpg)
Results 21
Snapshots of vorticity: uncontrolled / controlled
Adjoint velocity field: Large near wing and near stagnation point
streamline
EMS school Bedlewo Daniel Wachsmuth
![Page 32: Optimal control and Maximum principle - mimuwapalczew/wachsmuth_lecture.pdf · Optimal control and Maximum principle Daniel Wachsmuth, RICAM Linz ... Introduction to optimal control](https://reader031.vdocuments.us/reader031/viewer/2022022608/5b89fbf27f8b9a287e8d7a5f/html5/thumbnails/32.jpg)
References 22
Optimal control of ODEs
• L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko.
The mathematical theory of optimal processes. The Macmillan Co., New York,
1964.
• A. D. Ioffe and V. M. Tihomirov. Theory of extremal problems. North-Holland
Publishing Co., Amsterdam, 1979.
• Jack W. Macki and Aaron Strauss. Introduction to optimal control theory.
Springer-Verlag, New York, 1982.
Numerical methods
• J. Stoer and R. Bulirsch. Introduction to numerical analysis. Springer-Verlag,
New York, third edition, 2002.
Optimal control of PDEs
• M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich. Optimization with PDE
constraints. Springer, New York, 2009.
• F. Troltzsch. Optimal control of partial differential equations. AMS,
Providence, RI, 2010.
EMS school Bedlewo Daniel Wachsmuth