presentazione di powerpoint - uniroma1.itiacoviel/materiale/lecture... · 2018. 10. 31. · prof....
Post on 07-Sep-2020
0 Views
Preview:
TRANSCRIPT
Optimal Control
Prof. Daniela Iacoviello
Lecture 5
31/10/2018 Pagina 2
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM:YOU MUST STUDY ON THE BOOKS
Part of the slides has been taken from the References indicated below
Pagina 231/10/2018Prof. D.Iacoviello - Optimal Control
31/10/2018 Pagina 3Prof.Daniela Iacoviello- Optimal Control
Course outline
• Introduction to optimal control
• Nonlinear optimization
• Dynamic programming
• Calculus of variations
• Calculus of variations and optimal control
• LQ problem
• Minimum time problem
Pagina 331/10/2018Prof. D.Iacoviello - Optimal Control
31/10/2018 Pagina 4Prof. D.Iacoviello - Optimal Control
• Hamilton-Jacobi- Bellman equation
Prof. D.Iacoviello - Optimal Control 531/10/2018
William Rowan Hamilton
(4 August 1805 – 2 September 1865)
He was an Irish physicist, astronomer, and mathematician, who
made important contributions to classical mechanics,
optics and algebra.
His studies of mechanical and optical systems led
him to discover new mathematical concepts and techniques.
His greatest contribution is perhaps the reformulation of
Newtonian mechanics, now called Hamiltonian mechanics.
This work has proven central to the modern study of classical
field theories such as electromagnetism and to the development
of quantum mechanics. In mathematics, he is perhaps best
known as the inventor of quaternions.
.
Prof. D.Iacoviello - Optimal Control 631/10/2018
Jacobi (Posdam 1804- Berlin 1851)
He was still only 12 years old yet he had reached the necessary
standard to enter University in 1817
Around 1825 Jacobi changed from the Jewish faith to
become a Christian which now made university
teaching possible for him.
By the academic year 1825-26 he was teaching at the University
of Berlin.
Jacobi carried out important research in partial differential
equations of the first order and applied them to the differential
equations of dynamics.
He also worked on determinants and studied the functional
determinant now called the Jacobian.
Prof. D.Iacoviello - Optimal Control 731/10/2018
Richard Bellman (1920- 1984, USA)
He was interested in dynamic programming, with
applications in biology and medicine.
Prof. D.Iacoviello - Optimal Control 831/10/2018
The Hamilton-Jacobi equation
It is an approach in some sense alternative to the Minimum
Principle of Pontryagin, combined with the Euler Lagrange
equation.
Actually the Hamilton – Jacobi equation has so far rarely
proved useful except for linear regulator problems, to which
it seems particularly well suited.
Prof. D.Iacoviello - Optimal Control 931/10/2018
The Hamilton-Jacobi equation
The Hamilton-Jacobi (H-J) equation is satisfied by the optimal
performance index under suitable differentiability and
continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance index.
Prof. D.Iacoviello - Optimal Control 1031/10/2018
The Hamilton-Jacobi equation
- Such a solution need not exist
- Not every optimal performance index satisfies the Hamilton –
Jacobi equation
The HJ equation represents only a sufficient condition on the
optimal performance index
Prof. D.Iacoviello - Optimal Control 1131/10/2018
Principle of optimality
Minimizing over [t, tf] is equivalent to minimizing over [t, t1]
and [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 1231/10/2018
The Hamilton-Jacobi equation
Principle of optimality:
OR:
( )
( )
( )
++
= )((),(),(min),(),(min),(
1
1
1
1 ,,
*f
t
tttu
t
tttu
txGduxLduxLttxJ
f
f
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
Prof. D.Iacoviello - Optimal Control 1331/10/2018
The Hamilton-Jacobi equation
Principle of optimality:
The optimal cost for trajectories commencing at t and
finishing at tf is incurred by minimizing the sum of the
cost in transiting to x1(t1) and the optimal cost from there
onwards that is:
and
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
( )1
),(),(
t
t
duxL ( )ttxJ ),(*
Prof. D.Iacoviello - Optimal Control 1431/10/2018
Principle of optimality
A control policy optimal over the interval [t, tf] is
optimal over all subintervals [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 1531/10/2018
The Hamilton-Jacobi equation has so far rarely proved
useful except for linear regulator problems
The Hamilton-Jacobi equation
Prof. D.Iacoviello - Optimal Control 1631/10/2018
The Hamilton-Jacobi equationAssume the Hamiltonian regular: there exists a unique minimum
with respect to u(t).
The Hamilton-Jacobi equation:
Prof. D.Iacoviello - Optimal Control 1731/10/2018
0),(
,,),(
),(),(),(
=
+
t
x
txVt
x
txVtxutxH
t
txVTT
0,,),(
),(),(),(
,),(
,,),(
),(),(),(
=
+
+
ttx
txVtxutxf
x
txV
tx
txVt
x
txVtxutxL
t
txV
TT
TT
The Hamilton-Jacobi equation
Problem:
Consider the system:
Find the optimal control
that minimizes the cost:
( ) giventxtuxfx )(,,, 0=
fttttu ,),( 0*
( ) ( ) ( ))(),(),(),(),(
0
f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 1831/10/2018
Define the Hamiltonian:
Assume:
➢ f, L and G are continuously differentiable in all
arguments
➢ The Hamiltonian regular: there exists a unique minimum
with respect to u(t),
Prof. D.Iacoviello - Optimal Control 19
( ) ( ) ( )tuxftuxLtuxH T ,,,,,,, +=
31/10/2018
Derivation of the Hamilton-Jacobi equation
Let’s consider the system:
with the cost index:
Notation:
Initial instant
initial state
freeorfixedttxfixedxuxfx ff ),(,),,( 0=
+= ft
tftxGdtttutxLJ
0
))(()),(),((minmin
+=
f
f
t
tf
ttttu
txGdtttutxLxtJ0
0
))(()),(),((min),(
,)(
00
Prof. D.Iacoviello - Optimal Control 2031/10/2018
++=
++=
+=
f
f
f
f
f
f
t
tf
t
tttt
tu
t
tf
t
tttt
tu
t
tf
ttttu
txGdtttutxLdtttutxL
txGdtttutxLdtttutxL
txGdtttutxLxtJ
1
1
0
0
1
1
0
0
0
0
))(()),(),(()),(),((min
))(()),(),(()),(),((min
))(()),(),((min),(
,)(
,)(
,)(
00
Prof. D.Iacoviello - Optimal Control 2131/10/2018
( ) ),,,(,,,),()),(),((min
))(()),(),(()),(),((min
],[00110111*
,)(
,)(
10
1
0
0
1
1
0
0
ttf
t
tttt
tu
t
tf
t
tttt
tu
uxttxxtttxtJdtttutxL
txGdtttutxLdtttutxL
f
f
f
=
+=
++=
Principle of optimality
Idea: dttt += 01
,)(,()),(),((min
))(()),(),((min),(
*
],[
,)(
*
+++=
++=
+
+dttxdttJdtttutxL
txGdtttutxLxtJ
dtt
tdttt
f
t
ttttu
f
f
Prof. D.Iacoviello - Optimal Control 2231/10/2018
smalldt
],[)()( dttttuu +
dtttutxLduxLdtt
t)),(),(()),(),((
+
dtd
Jdx
x
JtxtJdttxdttJ
+
+++
**** ))(,())(,(
+
++=
+++= +
+
dtt
xtJdx
x
xtJxtJdtttutxL
dttxdttJdtttutxLxtJdtt
tdttt
),(),(),()),(),((min
)(,()),(),((min),(
***
*
],[
*
Prof. D.Iacoviello - Optimal Control 2331/10/2018
+=
− dx
x
xtJdtttutxLdt
t
xtJ
tu
),()),(),((min
),( *
)(
*
Divide for dt and consider 0
lim→t
+=
− ),(
),()),(),((min
),( **
uxfx
xtJttutxL
t
xtJ
Prof. D.Iacoviello - Optimal Control 2431/10/2018
J*(x(t),t) is a solution of the Hamilton-Jacobi (H-J) equation:
with boundary conditions:
-J*(x(t),t) continuous with respect to its arguments
-the first derivatives of J*(x(t),t) continuous with respect to its
arguments
-
-
−
−=
tttx
Jtxutxf
x
J
ttx
JtxutxL
t
J
,,,),(),('
,,),(),(
**
**
( ) ( ))(),(*fff txGttxJ =
Prof. D.Iacoviello - Optimal Control 25
( ) ( )
= tttx
x
Jtxuttxu
T
o ,),(),(),(*
31/10/2018
is the optimal performance index for
( )( )
( )tutxJttxJfttu
),(),(min),(,
* =
( ) ( ) ( ))(),(),(),(),( f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 26
and the control:
Is the optimal control at time t for the class problems with cost index
( ) ( )
= tttx
x
Jtxuttxuo ,),(),(),(
*
( ) ( ) ( ))(),(),(),(),( f
t
txGduxLuxJ
f
+=
31/10/2018
The Hamilton-Jacobi equation
Example 2 (from Anderson)
Consider the system:
With performance index:
Applying the H-J equation we have:
ux =
( ) 0,2
1)(),0( 0
422
0
=
++= twithdtxxuuxJ
ft
t
Prof. D.Iacoviello - Optimal Control 2731/10/2018
The minimizing control is:
And we have:
With boundary condition:
+++−=
u
x
Jxxu
t
J
tu
*422
)(
*
2
1min
x
Ju
−=
*
2
1
42
2**
2
1
4
1xx
x
J
t
J−−
=
( ) 0),( =ff ttxJ
In actual fact, it is rarely possible to solve a Hamilton-
Jacobi equation
Prof. D.Iacoviello - Optimal Control 2831/10/2018
The Hamilton-Jacobi equation
Main results
The Hamilton-Jacobi (H-J) equation is satisfied by the
optimal performance index under suitable differentiability
and continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance
index.
Note that the H-J equation represents only a sufficient
condition on the optimal performance index
Prof. D.Iacoviello - Optimal Control 2931/10/2018
top related