presentazione di powerpoint - uniroma1.itiacoviel/materiale/lecture... · 2018. 10. 31. · prof....
TRANSCRIPT
![Page 1: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/1.jpg)
Optimal Control
Prof. Daniela Iacoviello
Lecture 5
![Page 2: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/2.jpg)
31/10/2018 Pagina 2
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM:YOU MUST STUDY ON THE BOOKS
Part of the slides has been taken from the References indicated below
Pagina 231/10/2018Prof. D.Iacoviello - Optimal Control
![Page 3: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/3.jpg)
31/10/2018 Pagina 3Prof.Daniela Iacoviello- Optimal Control
Course outline
• Introduction to optimal control
• Nonlinear optimization
• Dynamic programming
• Calculus of variations
• Calculus of variations and optimal control
• LQ problem
• Minimum time problem
Pagina 331/10/2018Prof. D.Iacoviello - Optimal Control
![Page 4: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/4.jpg)
31/10/2018 Pagina 4Prof. D.Iacoviello - Optimal Control
![Page 5: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/5.jpg)
• Hamilton-Jacobi- Bellman equation
Prof. D.Iacoviello - Optimal Control 531/10/2018
![Page 6: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/6.jpg)
William Rowan Hamilton
(4 August 1805 – 2 September 1865)
He was an Irish physicist, astronomer, and mathematician, who
made important contributions to classical mechanics,
optics and algebra.
His studies of mechanical and optical systems led
him to discover new mathematical concepts and techniques.
His greatest contribution is perhaps the reformulation of
Newtonian mechanics, now called Hamiltonian mechanics.
This work has proven central to the modern study of classical
field theories such as electromagnetism and to the development
of quantum mechanics. In mathematics, he is perhaps best
known as the inventor of quaternions.
.
Prof. D.Iacoviello - Optimal Control 631/10/2018
![Page 7: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/7.jpg)
Jacobi (Posdam 1804- Berlin 1851)
He was still only 12 years old yet he had reached the necessary
standard to enter University in 1817
Around 1825 Jacobi changed from the Jewish faith to
become a Christian which now made university
teaching possible for him.
By the academic year 1825-26 he was teaching at the University
of Berlin.
Jacobi carried out important research in partial differential
equations of the first order and applied them to the differential
equations of dynamics.
He also worked on determinants and studied the functional
determinant now called the Jacobian.
Prof. D.Iacoviello - Optimal Control 731/10/2018
![Page 8: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/8.jpg)
Richard Bellman (1920- 1984, USA)
He was interested in dynamic programming, with
applications in biology and medicine.
Prof. D.Iacoviello - Optimal Control 831/10/2018
![Page 9: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/9.jpg)
The Hamilton-Jacobi equation
It is an approach in some sense alternative to the Minimum
Principle of Pontryagin, combined with the Euler Lagrange
equation.
Actually the Hamilton – Jacobi equation has so far rarely
proved useful except for linear regulator problems, to which
it seems particularly well suited.
Prof. D.Iacoviello - Optimal Control 931/10/2018
![Page 10: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/10.jpg)
The Hamilton-Jacobi equation
The Hamilton-Jacobi (H-J) equation is satisfied by the optimal
performance index under suitable differentiability and
continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance index.
Prof. D.Iacoviello - Optimal Control 1031/10/2018
![Page 11: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/11.jpg)
The Hamilton-Jacobi equation
- Such a solution need not exist
- Not every optimal performance index satisfies the Hamilton –
Jacobi equation
The HJ equation represents only a sufficient condition on the
optimal performance index
Prof. D.Iacoviello - Optimal Control 1131/10/2018
![Page 12: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/12.jpg)
Principle of optimality
Minimizing over [t, tf] is equivalent to minimizing over [t, t1]
and [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 1231/10/2018
![Page 13: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/13.jpg)
The Hamilton-Jacobi equation
Principle of optimality:
OR:
( )
( )
( )
++
= )((),(),(min),(),(min),(
1
1
1
1 ,,
*f
t
tttu
t
tttu
txGduxLduxLttxJ
f
f
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
Prof. D.Iacoviello - Optimal Control 1331/10/2018
![Page 14: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/14.jpg)
The Hamilton-Jacobi equation
Principle of optimality:
The optimal cost for trajectories commencing at t and
finishing at tf is incurred by minimizing the sum of the
cost in transiting to x1(t1) and the optimal cost from there
onwards that is:
and
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
( )1
),(),(
t
t
duxL ( )ttxJ ),(*
Prof. D.Iacoviello - Optimal Control 1431/10/2018
![Page 15: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/15.jpg)
Principle of optimality
A control policy optimal over the interval [t, tf] is
optimal over all subintervals [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 1531/10/2018
![Page 16: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/16.jpg)
The Hamilton-Jacobi equation has so far rarely proved
useful except for linear regulator problems
The Hamilton-Jacobi equation
Prof. D.Iacoviello - Optimal Control 1631/10/2018
![Page 17: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/17.jpg)
The Hamilton-Jacobi equationAssume the Hamiltonian regular: there exists a unique minimum
with respect to u(t).
The Hamilton-Jacobi equation:
Prof. D.Iacoviello - Optimal Control 1731/10/2018
0),(
,,),(
),(),(),(
=
+
t
x
txVt
x
txVtxutxH
t
txVTT
0,,),(
),(),(),(
,),(
,,),(
),(),(),(
=
+
+
ttx
txVtxutxf
x
txV
tx
txVt
x
txVtxutxL
t
txV
TT
TT
![Page 18: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/18.jpg)
The Hamilton-Jacobi equation
Problem:
Consider the system:
Find the optimal control
that minimizes the cost:
( ) giventxtuxfx )(,,, 0=
fttttu ,),( 0*
( ) ( ) ( ))(),(),(),(),(
0
f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 1831/10/2018
![Page 19: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/19.jpg)
Define the Hamiltonian:
Assume:
➢ f, L and G are continuously differentiable in all
arguments
➢ The Hamiltonian regular: there exists a unique minimum
with respect to u(t),
Prof. D.Iacoviello - Optimal Control 19
( ) ( ) ( )tuxftuxLtuxH T ,,,,,,, +=
31/10/2018
![Page 20: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/20.jpg)
Derivation of the Hamilton-Jacobi equation
Let’s consider the system:
with the cost index:
Notation:
Initial instant
initial state
freeorfixedttxfixedxuxfx ff ),(,),,( 0=
+= ft
tftxGdtttutxLJ
0
))(()),(),((minmin
+=
f
f
t
tf
ttttu
txGdtttutxLxtJ0
0
))(()),(),((min),(
,)(
00
Prof. D.Iacoviello - Optimal Control 2031/10/2018
![Page 21: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/21.jpg)
++=
++=
+=
f
f
f
f
f
f
t
tf
t
tttt
tu
t
tf
t
tttt
tu
t
tf
ttttu
txGdtttutxLdtttutxL
txGdtttutxLdtttutxL
txGdtttutxLxtJ
1
1
0
0
1
1
0
0
0
0
))(()),(),(()),(),((min
))(()),(),(()),(),((min
))(()),(),((min),(
,)(
,)(
,)(
00
Prof. D.Iacoviello - Optimal Control 2131/10/2018
![Page 22: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/22.jpg)
( ) ),,,(,,,),()),(),((min
))(()),(),(()),(),((min
],[00110111*
,)(
,)(
10
1
0
0
1
1
0
0
ttf
t
tttt
tu
t
tf
t
tttt
tu
uxttxxtttxtJdtttutxL
txGdtttutxLdtttutxL
f
f
f
=
+=
++=
Principle of optimality
Idea: dttt += 01
,)(,()),(),((min
))(()),(),((min),(
*
],[
,)(
*
+++=
++=
+
+dttxdttJdtttutxL
txGdtttutxLxtJ
dtt
tdttt
f
t
ttttu
f
f
Prof. D.Iacoviello - Optimal Control 2231/10/2018
![Page 23: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/23.jpg)
smalldt
],[)()( dttttuu +
dtttutxLduxLdtt
t)),(),(()),(),((
+
dtd
Jdx
x
JtxtJdttxdttJ
+
+++
**** ))(,())(,(
+
++=
+++= +
+
dtt
xtJdx
x
xtJxtJdtttutxL
dttxdttJdtttutxLxtJdtt
tdttt
),(),(),()),(),((min
)(,()),(),((min),(
***
*
],[
*
Prof. D.Iacoviello - Optimal Control 2331/10/2018
![Page 24: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/24.jpg)
+=
− dx
x
xtJdtttutxLdt
t
xtJ
tu
),()),(),((min
),( *
)(
*
Divide for dt and consider 0
lim→t
+=
− ),(
),()),(),((min
),( **
uxfx
xtJttutxL
t
xtJ
Prof. D.Iacoviello - Optimal Control 2431/10/2018
![Page 25: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/25.jpg)
J*(x(t),t) is a solution of the Hamilton-Jacobi (H-J) equation:
with boundary conditions:
-J*(x(t),t) continuous with respect to its arguments
-the first derivatives of J*(x(t),t) continuous with respect to its
arguments
-
-
−
−=
tttx
Jtxutxf
x
J
ttx
JtxutxL
t
J
,,,),(),('
,,),(),(
**
**
( ) ( ))(),(*fff txGttxJ =
Prof. D.Iacoviello - Optimal Control 25
( ) ( )
= tttx
x
Jtxuttxu
T
o ,),(),(),(*
31/10/2018
![Page 26: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/26.jpg)
is the optimal performance index for
( )( )
( )tutxJttxJfttu
),(),(min),(,
* =
( ) ( ) ( ))(),(),(),(),( f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 26
and the control:
Is the optimal control at time t for the class problems with cost index
( ) ( )
= tttx
x
Jtxuttxuo ,),(),(),(
*
( ) ( ) ( ))(),(),(),(),( f
t
txGduxLuxJ
f
+=
31/10/2018
![Page 27: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/27.jpg)
The Hamilton-Jacobi equation
Example 2 (from Anderson)
Consider the system:
With performance index:
Applying the H-J equation we have:
ux =
( ) 0,2
1)(),0( 0
422
0
=
++= twithdtxxuuxJ
ft
t
Prof. D.Iacoviello - Optimal Control 2731/10/2018
![Page 28: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/28.jpg)
The minimizing control is:
And we have:
With boundary condition:
+++−=
u
x
Jxxu
t
J
tu
*422
)(
*
2
1min
x
Ju
−=
*
2
1
42
2**
2
1
4
1xx
x
J
t
J−−
=
( ) 0),( =ff ttxJ
In actual fact, it is rarely possible to solve a Hamilton-
Jacobi equation
Prof. D.Iacoviello - Optimal Control 2831/10/2018
![Page 29: Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2018. 10. 31. · Prof. D.Iacoviello - Optimal Control 31/10/2018 7. Richard Bellman (1920- 1984, USA) He](https://reader036.vdocuments.us/reader036/viewer/2022071113/5fe9f5240ebfff386d14237a/html5/thumbnails/29.jpg)
The Hamilton-Jacobi equation
Main results
The Hamilton-Jacobi (H-J) equation is satisfied by the
optimal performance index under suitable differentiability
and continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance
index.
Note that the H-J equation represents only a sufficient
condition on the optimal performance index
Prof. D.Iacoviello - Optimal Control 2931/10/2018