Grading
Project + oral exam
Example of project:
Read a paper on an optimal control problem
1) Study: background, motivations, model, optimal control, solution,
results
2) Simulations
You must give me, al least ten days before the date of the exam:
-A .doc document
-A power point presentation
- Matlab simulation files
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 2
Example of project:
-Read a paper on an optimal control problem (any field will do)
You can choose a topic proposed in the papers I sent you, or from the
examples proposed in the books suggested (i.e. Evans) , or some
specific problem of your interest.
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 3
1) Study: background, motivations, model, optimal control, solution,
results
You have to indicate clearly:
-the background of the problem you are studying by a literature
research, motivations (why the problem is important?),
-the model assumed (indicate variables, parameters, their meaning,
etc),
-the cost index and the optimal control introduced (the meaning, what
they represent, different possible choices, etc).
2) Simulations: You have to implement the model and the control by
matlab
Describe the program, propose the results and discuss them.
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 4
YOU MUST STUDY ON THE BOOKS
THE SLIDES ARE NOT SUFFICIENT
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 5
Schedule
Lecture 1:
Introduction to optimal control: motivations, examples.
Definitions
Unconstrained optimization (first order necessary conditions,
second order necessary conditions)
Weierstrass theorem
Constrained optimization, introduction of Lagrangian function
First order necessary conditions, Second order sufficient
conditions.
Function spaces: First order necessary condition, second
order necessary condition, necessary and sufficient condition
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 6
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a local minimum of over
If
RRf n →::
nRD
•
Dx *f
nRD
)()(
0
*
*
xfxf
xxsatisfyingDxallforthatsuch
−
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 7
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a strict local minimum of over
If
RRf n →::
nRD
•
Dx * fnRD
**
*
),()(
0
xxxfxf
xxsatisfyingDxallforthatsuch
−
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 8
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a global minimum of over
If
RRf n →::
nRD
•
Dx *f
nRD
)()(
0
* xfxf
Dxallforthatsuch
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 9
Unconstrained optimization - first order necessary conditions
All points sufficiently near in are in
Assume and its local minimum. Let
an arbitrary vector.
Being in the unconstrained case:
Let’s consider:
0 is a minimum of g
First order necessary condition for optimality
x
nRd
DnR
1Cf *x
)(:)( * dxfg +=
0* toenoughcloseRDdx +
*x
0)( * = xf
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 10
Unconstrained optimization- second order conditions
Assume and its local minimum. Let
an arbitrary vector.
Second order Taylor expansion of g around
Since
nRd2Cf *x
0=
0)(
lim),()0(''2
1)0(')0()(
2
2
0
2 =+++=→
oogggg
0)0(' =g
0)0('' g
0)( *2 xf Second order necessary condition for
optimality
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 11
Unconstrained optimization- second order conditions
Let and
is a strict local minimum of f
0)( * = xf2Cf 0)( *2 xf
*x
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 12
Definitions- Remarks
A vector is a feasible direction at if
If D in not the entire then D is the constraint set over
which f is being minimized
*xnRd
0* + enoughsmallforDdx
nR
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 13
Global minimum
Weierstrass Theorem
Let f be a continuous function and D a compact set
there exist a global minimum of f over D
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 14
Constrained optimization
Let
Equality constraints
Inequality constraints
Regularity condition:
Lagrangian function
If the stationary point is called normal and we
can assume .
1, CfRD n 1,:,0)( ChRRhxh p →=
1,:,0)( CgRRgxg q →
( )
a
a
a
x
a
qg
ofconstrainactivethearegwhere
qpx
ghrank
dimensionwith
,
*
+=
( ) )()()(,,, 00 xgxhxfxL TT ++=
*x00
10 =Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 15
From now on and therefore the Lagrangian is
If there are only equality constraints the are called
Lagrange multipliers
If there are both equality and inequality constrains we have
Kuhn – Tucker multipliers
Constrained optimization
i
10 =
( ) )()()(,, xgxhxfxL TT ++=
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 16
Constrained optimization
First order necessary conditions for constrained
optimality:
Let and
The necessary conditions for to be a constrained local
minimum are
If the functions f and g are convex and the functions h
are linear these conditions are necessary and
sufficient!!!
i
ixg
x
L
i
ii
T
T
x
=
=
0
,0)(
0
*
*
Dx *1,, Cghf
*x
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 17
Constrained optimization
Second order sufficient conditions for constrained
optimality:
Let and and assume the conditions
is a strict constrained local minimum if
ixgx
Liii
T
T
x
==
0,0)(0 *
*
Dx * 2,, Cghf
*x
piddx
xdhthatsuchdd
x
Ld
x
i
x
T ,...,1,0)(
0**
2
2
==
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 18
Function spaces
Functional
Vector space V,
is a local minimum of J over A if there exists an
RVJ →:
VA
Az *
)()(
0
*
*
zJzJ
zzsatisfyingAzallforthatsuch
−
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 19
Function spaces
Consider function in V of the form
The first variation of J at z is the linear function
such that
First order necessary condition for optimality:
For all admissible perturbation we must have:
RVz + ,,
RVJz
→:
and
)()()()( oJzJzJ z ++=+
0)(* =z
J
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 20
Function spaces
A quadratic form is the second variation
of J at z if
we have:
second order necessary condition for optimality:
If is a local minimum of J over
for all admissible perturbation we must have:
RVJz
→:2
and
)()()()()( 222 oJJzJzJ zz +++=+
0)(*
2 z
J
Az * VA
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 21
Function spaces
The Weierstrass Theorem is still valid
If J is a convex functional and is a convex set
A local minimum is automatically a global one and the first
order condition are necessary and sufficient condition for
a minimum
VA
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 22
Schedule
Lecture 2:
Calculus of variations- Motivations
The Lagrange problem: Euler equation, Weierstrass
Erdmann condition, transversality conditions
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 23
The Lagrange problem
Problem 1
Let us consider the linear space and define the
admissible set:
Introduce the norm:
and consider the cost index:
with L function of C2 class.
Find the global minimum (optimum) for J over D, :
RRRC )(1
( ) ( ) ( ) 111 ),(,),(:)(,, ++ = vf
viiii RDTTzRDttzRRRCTtzD
=T
t
i
i
dtttztzLTtzJ ),(),(),,(
( ) TttztzTtz itt
i +++= )(sup)(sup,,
( )ooi
o Ttz ,,
( ) ( ) DTtzTtzJTtzJ iioo
io ),,(,,,,
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 24
03/12/2018 Pagina 25Prof.Daniela Iacoviello- Optimal Control Pagina 2503/12/2018
SCHEME of the theorems
Theorem (Lagrange). If is a local minimum then
1)
2) In any discontinuity point of
Weierstrass-Erdmann condition
3) Transversality conditions different cases
depending on the nature
of the boundary conditions
( ) Dttz fi *** ,,
fiT ttt
z
L
dt
d
z
L,0
**
=
−
t *z
Euler equation
Theorem (Lagrange). If is a local minimum then
In any discontinuity point of the following conditions are
verified:
( ) DTtz i *** ,,
Tttz
L
dt
d
z
Li
T ,0
**
=
−
Euler equation
t *z
****
+−+−
−=
−
=
tttt
zz
LLz
z
LL
z
L
z
L
Weierstrass-
Erdmann
condition
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 26
Moreover, transversality conditions are satisfied:
• If are open subset we have:
• If are closed subsets defined respectively by
such that
for
fi DD
0000**
**
**
**
===
=
Tt
T
T
T
t
LLz
L
z
L
i
i
fi DD
( ) ( ) 0),(0),( == ffii ttzttz
( ) ( ) fi
ii TTzrg
ttzrg
=
=
**
),(),(
fi RR
**
****
**
**
,
)(,
)(
Tz
z
LL
tz
z
LL
Tzz
L
tzz
L
T
Ti
T
t
T
Ti
T
t
i
i
=
−
=
−
=
=
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 27
• If the sets are defined by the function
affine with respect to such thatfi DandD ))(),(( Tztzw i
)(),( Tztz i
( )=
*
)(),( Tztz
wrg
i
R
Tz
w
z
L
tz
w
z
L T
Ti
T
ti
−=
=
****
)(,
)( **
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 28
03/12/2018 Pagina 29Prof.Daniela Iacoviello- Optimal Control Pagina 2903/12/2018
The Lagrange problem
Problem 2
Consider Problem 1 with
✓ fixed
✓ If are closed sets in defined by the C1
functions
fi tandt
fi DD 1+vR
( )( ) 1dimension of,0),(
1dimension of,0),(
f
i
+=
+=
vTTz
vttz ii
with affine functions and and
( ) ( ) f
o
fi
o
i tzrg
tzrg
=
=
)()(
03/12/2018 Pagina 30Prof.Daniela Iacoviello- Optimal Control Pagina 3003/12/2018
✓ If the sets are defined by the function
with σ components of C1 class affine with respect to
such that
fi DD ))(),(( fi tztzw
)(),( fi tztz
( )=
o
fi tztz
wrg
)(),(
✓ The function L must be convex with respect to
Find the global minimum (optimum) for J over D:
( ))(),( tztz
oz
( ) ( ) DzzJzJ o
Theorem 2. is the optimum if and only if
In any discontinuity point of the following conditions
are verified:
Moreover, transversality conditions are satisfied
…..
Dzo
fiT
oo
tttz
L
dt
d
z
L,0 =
−
Euler equation
t *z
o
t
o
t
o
t
o
t
zz
LLz
z
LL
z
L
z
L
+−+−
−=
−
=
Weierstrass-
Erdmann
condition
Pagina 31Prof.Daniela Iacoviello- Optimal Control 03/12/2018
The Lagrange problem
Problem 1_bis
Let us consider the admissible set:
consider the cost index:
with L function of C2 class.
Find the global minimum (optimum) for J over D
ffiifi ztzztzttCzD === )(,)(:],[1
=T
t
i
i
dtttztzLTtzJ ),(),(),,(
Prof.Daniela Iacoviello- Optimal Control
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 32
The Lagrange problem
Theorem (Legendre)
Necessary condition for to be local minimum is that Dz *
],[0
*
2
2
fi tttz
L
Prof.Daniela Iacoviello- Optimal Control
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 33
The Lagrange problem
Problem 3
Let us consider the linear space and define
the admissible set
of dimension
Consider the cost index:
RRRC )(1
( ) ( )
( ) ( ) ( )
==
=
+
+
kdtttztzhttztzgRDTTz
RDttzRRRCTtzD
T
t
v
f
v
iiii
i
),(),(0),(),(,),(
,),(:)(,,
1
11
g v
=T
t
i
i
dtttztzLTtzJ ),(),(),,(
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 34
Define the augmented lagrangian:
( ) ( )
( ) ( )ttztzhtttztzgt
ttztzLtttztz
TT ),(),()(),(),()(
),(),(),(,,),(),( 00
++
=
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 35
Theorem 3 (Lagrange). Let be such that
If is a local minimum for J over D, then there exist
not simultaneously null in
such that:
❑
❑
with TRANSVERSALITY
CONDITIONS
( ) Dttz fi *** ,,
**
*
,)(
fi ttttz
grank =
****
,0 fiT ttt
zdt
d
z=
−
( )*** ,, fi ttz
RttCR fi **0**0 ],,[, ],[ *
fi tt
03/12/2018 Pagina 36Prof.Daniela Iacoviello- Optimal Control
****
+−+−
−=
−
=
kkkk tttt
zz
zzzz
where are cuspid points forkt
*z
The Lagrange problem
Problem 4 Let us consider the linear space
and define the admissible set
❑ of dimension
❑ ti tf fixed
❑ g and h affine functions in
❑ L C2 function convex with respect to
Consider the cost index:
RRRC )(1
( ) ( )
=== ],[,),(),(,0),(),(,)(,)(,,1fi
t
t
fiiifi tttkdtttztzhttztzgDtzDtzttCzD
f
i
g v
03/12/2018 Pagina 37Prof.Daniela Iacoviello- Optimal Control
],[),(),( fi ttttztz
],[),(),( fi ttttztz
=f
i
t
t
fi dtttztzLttzJ ),(),(),,(
Define the augmented lagrangian:
( ) ( )
( ) ( )ttztzhtttztzgt
ttztzLtttztz
TT ),(),()(),(),()(
),(),(),(,,),(),( 0
++
=
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 38
Theorem 4 (Lagrange). Let such that
is an optimal normal solution
if and only if
➢
➢ in the instants for which
are open we have:
Dzo
fi
o
ttttz
grank ,
)(=
Dzo
fiT ttt
zdt
d
z,0
**
=
−
03/12/2018 Pagina 39Prof.Daniela Iacoviello- Optimal Control
fitt and/or fi
DD and/or
To
z0=
Schedule
Lecture 3:
Calculus of variations and optimal control:
The Hamiltonian function- Necessary conditions- Necessary
and sufficient conditions
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 40
Problem 1
Let us consider the dynamical sistem described by:
f function of C2 class.
),,( tuxfx =
x(t) state vector in Rn
u(t) control vector in Rp
ii xtx =)(known
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 41
1+ nf
( ) 0),( =ff ttx
Vectorial function of C1 class
of dimension
0),,( tuxq
Vectorial function of C2 class
of dimension
Define the cost index
with L function of C2 class
( )=
f
i
t
t
f dttuxLtuxJ ,,),,(
Assume the norm:
( ) ft
t
ttttf ttudutxtxtux
i
++++= )(sup)(sup)(sup)(sup,,
03/12/2018 Pagina 42Prof.Daniela Iacoviello- Optimal Control
AIM: Find (if it exists)
❑ the instant
❑ the control
❑ the state
that satisfy the previous equations and minimize the cost
index
oft
)(0 RCuo
)(1 RCxo
03/12/2018 Pagina 43Prof.Daniela Iacoviello- Optimal Control
DEFINE the scalar function
the Hamiltonian function
),,()(),,(),,,,( 00 tuxfttuxLtuxH T +=
03/12/2018 Pagina 44Prof.Daniela Iacoviello- Optimal Control
Theorem 1
Let be an admissible solution such that )*,*,( *ftux
*
**
,),()),((
fiaactive
fff
ttttu
qrk
ttxrk =
=
03/12/2018 Pagina 45Prof.Daniela Iacoviello- Optimal Control
IF is a local minimum
there exist
not simultaneously null in such that:
)*,*,( *ftux
],[],,[,0 *0**1**0 fifi ttCttC
],[ *fi tt
dimension of
***
*
TT
x
q
x
H
−
−=
03/12/2018 Pagina 46
***
0
TT
u
q
u
H
+
=
,...,2,1,0)(,0)*,*,()( ** == jttuxqt jjj
❑
❑
❑
f
f
Rtx
t
T
tf
f
−= ,
))(()(*
*
*
*
❑
T
tf
t
f
f tH
*
*
*
*
=❑
The discontinuity of may occurr only in the points
where has a discontinuity and
** and t
*u**+− = tt
HH
Prof.Daniela Iacoviello- Optimal Control
Problem 3
Let us consider the dynamical system described by:
f function of C2 class
),,( tuxfx =
x(t) state vector in Rn
u(t) control vector in Rp
ii xtx =)(known
Calculus of variation and optimal control
03/12/2018 Pagina 47Prof.Daniela Iacoviello- Optimal Control
1+ nf
( ) 0),( =ff ttx
Vectorial function of C1 class
of dimension
( ) =
f
i
t
t
Kdtttutxh ),(),(
Vectorial function of C2 class
of dimension
Define the cost index
with L function of C2 class
( )=
f
i
t
t
f dttuxLtuxJ ,,),,(
Assume the norm:
( ) ft
t
ttttf ttudutxtxtux
i
++++= )(sup)(sup)(sup)(sup,,
03/12/2018 Pagina 48Prof.Daniela Iacoviello- Optimal Control
AIM: Find
❑ the instant
❑ the control
❑ the state
that satisfy the previous equations and minimize the cost
index
oft
)(0 RCuo
)(1 RCxo
03/12/2018 Pagina 49Prof.Daniela Iacoviello- Optimal Control
DEFINE the scalar function
the Hamiltonian function
( )ttutxhtuxfttuxLtuxH TT ),(),(),,()(),,(),,,,( 00 ++=
03/12/2018 Pagina 50Prof.Daniela Iacoviello- Optimal Control
Theorem 3
Let be an admissible solution such that )*,*,( *ftux
fff ttx
rk
=
*
)),((
03/12/2018 Pagina 51Prof.Daniela Iacoviello- Optimal Control
IF is a local minimum
there exist not
simultaneously null in such that:
❑
❑
❑
❑
)*,*,( *ftux
RttCR fi **1**0 ],,[,
],[ *fi tt
T
x
H*
*
−=
f
f
Rtx
t
T
tf
f
−= ,
))(()(*
*
*
*
T
u
H*
0
=
T
tf
t
f
f tH
*
*
*
*
=
The discontinuity of may occurr only in the points
where has a discontinuity
and
* kt*u
**+− =
kk ttHH
03/12/2018 Pagina 52Prof.Daniela Iacoviello- Optimal Control
Problem 4:
Consider the linear system
Assume:
❑ fixed
❑
❑
utBxtAx )()( +=
Functions of C2 classfi tt ii xtx =)(
nRor pointfixedabeing)( fff DDtx
0),,( tuxq Vectorial function of C2 class of dimension
CONVEX
03/12/2018 Pagina 53Prof.Daniela Iacoviello- Optimal Control
Define the cost index
))((),,(),( f
t
t
txGdttuxLuxJ
f
i
+=
Functions of C3 class- CONVEX
Functions of C2 class CONVEX
Theorem 4
Let be an admissible solution such that
is a normal
optimal solution
),( oo ux
fia
oactive ttttu
qrk ,),( =
),( oo ux
0)(
,...,2,1
,0),,()(
0
=
=
+
=
−
−=
t
j
tuxqt
u
q
u
H
x
q
x
H
o
ooj
oj
ooToT
ooToT
o
03/12/2018 Pagina 54Prof.Daniela Iacoviello- Optimal Control
nf RD =AND IF
oT
ff
o
tdx
dGt
)()( =
Schedule
Lecture 4:
Pontryagin minimum principle-
Necessary conditions- The convex case- Example
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 55
The Pontryagin principle
Problem 1: Consider the dynamical system:
with:
Assume fixed the initial control instant and the initial and final
values :
Define the performance index :
with
( )uxfx ,=
( ) niURCx
ffRUtuRtx n
i
pn ,...,2,1,,)(,)( 0 =
Pagina 5603/12/2018Prof.Daniela Iacoviello- Optimal Control
ff
ii xtxxtx == )()(
( ) ( ) ))(()(),(,, f
t
t
f txGduxLtuxJ
f
i
+=
( ) 20 ,,...,2,1,, CGniURCx
LL n
i
=
Determine:
▪ the value ,
• the control
• the state
that satisfy:
✓ the dynamical system,
✓ the constraint on the control,
✓ the initial and final conditions
✓ and minimize the cost index
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 5703/12/2018Prof.Daniela Iacoviello- Optimal Control
Hamiltonian function
( ) ( ) ( )uxftuxLuxH T ,)(,,,, 00 +=
Pagina 5803/12/2018Prof.Daniela Iacoviello- Optimal Control
The Pontryagin principle
Theorem 1 (necessary condition):
Assume the admissible solution is a minimum
there exist a constant
and a n-dimensional vector
not simultaneously null such that :
( )*** ,, ftux
00
*1* , fi ttC
T
x
H*
*
−=
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **0
****0
*
Pagina 5903/12/2018Prof.Daniela Iacoviello- Optimal Control
0*=H
The Pontryagin principle
Problem 2
Theorem 2
Problem 3
Theorem 3
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 60
The Pontryagin principle
Problem 4: Consider the dynamical system:
with:
and initial instant and state fixed
( )tuxfx ,,=
( )RURCt
f
x
ffRUtuRtx npn
0,,)(,)(
ii xtx =)(
Pagina 6103/12/2018Prof.Daniela Iacoviello- Optimal Control
For the final values assume: where is a
function of dimension of C1 class.
Assume the constraint with
Define the performance index :
with
( ) 0),( = ff ttx
1+ nf
( ) kduxh
f
i
t
t
= ),(),(
( ) niRURCt
h
tx
hh n ,...,2,1,,
)(, 0 =
( ) ( )=f
i
t
t
f duxLtuxJ ),(),(,,( )RURC
t
L
x
LL n
0,,
Determine
• the value
• the control
• and the state
that satisfy
✓ the dynamical system,
✓ the constraint on the control,
✓ the initial and final conditions
✓ and minimize the cost index .
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 6203/12/2018Prof.Daniela Iacoviello- Optimal Control
Hamiltonian function
( ) ( ) ( ) )),(),((,)(,,,, 00 ttutxhuxftuxLuxH TT ++=
Pagina 6303/12/2018Prof.Daniela Iacoviello- Optimal Control
Theorem 4 (necessary condition):
Consider an admissible solution such that
IF it is a local minimum
there exist a constant , ,
not simultaneously null such that :
( )*** ,, ftux
00 *1* , fi ttC
( ) f
ff ttxrank =
*
),(
Pagina 6403/12/2018Prof.Daniela Iacoviello- Optimal Control
,
**
T
x
H
−=
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **
0
****
0
*
Moreover there exists a vectorsuch that:fR
T
ft
T
f tH
txT
f
*
*
*
**
)()(
−=
=
RkkdH
H
ft
t
=
+ ,
*
**
R*
The discontinuities of may occur only in the instants
in which u has a discontinuity and in these instants the
Hamiltonian is continuous
*
Pagina 6503/12/2018Prof.Daniela Iacoviello- Optimal Control
Remark
If the set U coincides with Rp the minimum
condition reduces to :
0=
u
H
Pagina 6603/12/2018Prof.Daniela Iacoviello- Optimal Control
The Pontryagin principle - convex case
Problem 5: Consider the dynamical linear system:
with A and B of function of C1 class; assume fixed the initial
and final instants and the initial state, and
Assume
where U is a convex set.
utBxtAx )()( +=
nfff Rtxorfixedxtx = )()(
fip tttRUtu ,)(
Pagina 6703/12/2018Prof.Daniela Iacoviello- Optimal Control
Define the performance index :
with
L convex function with respect to x(t), u(t) in
per ogni
G is a scalar function of C2 class and convex with
respect to x(tf).
( ) ( ) ( ))(),(),(, f
t
t
txGduxLuxJ
f
i
+=
( ) nittURCt
L
x
LL fi
n
i
,...,2,1,,,, 0 =
URn
fi ttt ,
Pagina 6803/12/2018Prof.Daniela Iacoviello- Optimal Control
Determine:
the control
and the state
that satisfy
the dynamical system,
the constraint on the control,
the initial and final conditions
and minimize the cost index .
fio ttCu ,0
fio ttCx ,1
Pagina 6903/12/2018Prof.Daniela Iacoviello- Optimal Control
Theorem 5 (necessary and sufficient condition):
Consider an admissible solution such that
It is a minimum normal (i.e.λ0 =1)
if and only if there exists an n-dimensional vector
such that :
( )oo ux ,
fio ttC ,1
( ) f
o
ff ttxrank =
),(
Pagina 7003/12/2018Prof.Daniela Iacoviello- Optimal Control
oTo
x
tuxH
−=
),,,(
( ) ( ) UttutxHttxH ooooo ,)(),(),()(,),(
nf Rtx )(
oT
f
fo
tdx
dGt
)()( =Moreover, if
Schedule
Lecture :
The Hamilton- Jacobi equation- Derivation of the H-J
equation-
Principle of optimality- Example
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 71
Principle of optimality
A control policy optimal over the interval [t, tf] is optimal
over all subintervals [t1, tf]
If the shortest path from Rome to Paris passes through
Milan THEN the OBTAINED path between Milan to Paris
is the shortest one between Milan to Paris
Prof. D.Iacoviello - Optimal Control 7203/12/2018
MILAN
Rome
PARIS
Schedule
Lecture 8:
The optimal regulator problem:
The optimal tracking problem
The optimal regulator problem with null final error
The optimal regulator problem with limited control
Examples
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 73
The regulator problem
)()()(1 tKtBtR T−
r=0+
-
uo
xi
xo ii
o
oTo
xtx
txtKtBtRtBtAtx
=
−= −
)(
)()()()()()()( 1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 74
The regulator problem
Let us consider the following linear system:
with fixed, and fixed
are matrix functions of time with
continuous entries; their dimensions are respectively
)()()(
)()()()()(
txtCty
tutBtxtAtx
=
+=
],[ fi tt
pnmnnn ,,
)(),(),( tCtBtA
ii xtx =)(
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 75
The regulator problem
A linear control law is obtained if we seek to minimize the
quadratic performance index:
where Q(t) and R(t) have continuous entries, symmetric,
nonnegative and positive definite respectively;
F is nonnegative definite matrix simmetric
( )
)()(2
1
)()()()()()(2
1),(
0
ffT
t
t
TT
tFxtx
dttutRtutxtQtxuxJf
+
+=
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 76
The Riccati equation
Theorem: The Riccati equation
admits a unique solution, symmetric, semidefinite positive,
in the control interval.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 77
FtK
tQtKtAtAtKtKtBtRtBtKtK
f
TT
=
−−−= −
)(
)()()()()()()()()()()( 1
Solution of the regulator problem
Theorem: The optimal control for the regulator problem is
given by the linear feedback law:
where:
and
)()()()()( 1 txtKtBtRtu oTo −−=
ii
o
oTo
xtx
txtKtBtRtBtAtx
=
−= −
)(
)()()()()()()( 1
( ) i
i
iToo xtKxuxJ )(2
1, =
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 78
The regulator problem
Theorem: The regulator problem admits a unique solution.
Proof: the uniqueness property is shown by contraddiction
arguments.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 79
Solution of the deterministic linear optimal
regulator problem on
Problem: let us consider the regulator problem over the
infinite time interval .
Assume:
o The matrices A and B are bounded and with elements in
C1 class
o The dynamical system is completely controllable or
exponentially stable
o The matrices Q and R are symmetric, semidefinite and
positive definite respectively, with elements in C1 class
and bounded
),it
),it
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 80
Solution of the deterministic linear optimal
regulator problem on
Find:
the control
and the state
satisfying the dynamical system , the initial condition and
minimizing the cost index
) ,0i
o tCu
) ,1i
o tCx
( )
+=
it
TT dttutRtutxtQtxuxJ )()()()()()(2
1,
),it
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 81
Solution of the deterministic linear optimal
regulator problem on
Theorem: The problem admits a unique optimal solution
that can be expressed as follows:
where is the solution of the Riccati equation
with final condition:
and
ii
ooTo
oTo
xtxtxtKtBtRtBtAtx
txtKtBtRtu
=−=
−=
−
−
)(),()()()()()()(
)()()()()(
1
1
)(tK
)()()()()()()()()()()( 1 tQtKtAtAtKtKtBtRtBtKtK TT −−−= −
0)(lim =→
ft
tKf
( ) ii
iToo xtKxuxJ )(2
1, =
),it
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 82
The steady state solution of the deterministic
linear optimal regulator problem
Theorem: Let us consider the previous problem with the
additive hypotheses :
o The matrices A, B, Q, R are constant
o The matrix Q is positive definite
Then there exists a unique optimal solution:
where:
i
i
oo
r
To
o
r
To
xtxtxKBBRAtx
xKBRtu
=−=
−=
−
−
)(),()(
)(
1
1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 83
Kr is the constant matrix, unique solution definite positive of
the algebraic Riccati equation:
The minimum value for the cost index is:
01 =−−−− QKAAKKBBRK r
T
rr
T
r
( ) i
r
iToo xKxuxJ2
1, =
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 84
Remarks
Consider the system:
It can be defined an optimal regulation problem from the
output y, considering the cost index
)()()(
)()()()()(
txtCty
tutBtxtAtx
=
+=
( )
++
=
f
i
t
t
TT
ffT
dttutRtutytQty
tyFtyuyJ
)()()()()()(2
1
)()(2
1,
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 85
The present problem is similar to the previous one, being
satisfied the hypotheses over the matrices involved.
Anyway the state must be accessible, if one want to
realize the feedback action:
ooo BuAxx +=
KBR T1−
r=0 +
-
uo
xi
xo
Cyo
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 86
The optimal tracking problem
Problem: let us consider the linear system:
where are matrices with elements functions of
time with entries of C1 class.
Consider the reference variable
Determine the optimal control
and the state satisfying the dinamical
constraint and minimizing the cost index:
i
i xtxtutBtxtAtx =+= )(),()()()()(
)(),( tBtA
fi ttCr ,1
fio ttCu ,0
fio ttCx ,1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 87
where:
o Q(t) is a symmetric semi-positive definite matrix
o R(t) is a symmetric positive definite matrix
o The elements of Q(t) and R(t) are functions of C1
class
( ) +−−=
f
i
t
t
TTdttutRtutxtrtQtxtruxJ )()()()()()()()(
2
1,
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 88
The optimal tracking problem
To the optimal tracking problem it can be associated the
Riccati equation
0)(
)()()()()()()()()()()( 1
=
−−−= −
f
TT
tK
tQtKtAtAtKtKtBtRtBtKtK
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 89
Theorem: the optimal tracking problem admits a unique
optimal solution:
i
ioT
oTo
oTo
xtxtgtBtRtB
txtKtBtRtBtAtx
txtKtgtBtRtu
=+
−=
−=
−
−
−
)()()()()(
)()()()()()()(
)()()()()()(
1
1
1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 90
where g is the solution of the differential equation
The minimum value of the cost index is:
where is the solution of the equation:
0)(
)()()()()()()()()( 1
=
−−= −
f
TT
tg
trtQtgtAtBtRtBtKtg
( ) )()()()()()(2
1, iii
oT
i
o
ii
oToo tvtgtxtxtKtxuxJ +−=
v
0)(
)()()(2
1)()()()()(
2
1)( 1
=
−= −
f
TTT
tv
trtQtrtgtBtRtBtgtv
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 91
The optimal regulator problem with null final error
Problem: Let us consider the linear system:
Determine the control
and the state
in order to satisfy the dynamical system, the initial and
final conditions and minimizing the following cost
index:
0)(,)(
),()()()()(
==
+=
fi
i txxtx
tutBtxtAtx
fio ttCu ,0
fio ttCx ,1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 93
( ) +=
ft
TT dttutRtutxtQtxuxJ
0
)()()()()()(2
1,
where:
o Q(t) is symmetric semidefinite positive with elements of C1
class
o R(t) is symmetric definite positive with elements of C1
class
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 94
Theorem: Let us introduce the matrix of dimension 2n×2n
and indicate with
its transition matrix partitioned in submatrices of dimension
n×n.
Assume that the submatrix is not singular .
The optimal regulator problem with null final
error admits a unique optimal solution:
−−
−=
−
)()(
)()()()()(
1
tAtQ
tBtRtBtAt
T
T
( )( ) ( )
( ) ( )
=
,,
,,,
2221
1211
tt
ttt
( )if tt ,12
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 95
( ) ( ) ( ) ( ) iififii
To xtttttttttBtRtu ,,,,)()()( 111
1222211 −− −−=
( ) ( ) ( ) ( ) iififii
o xtttttttttx ,,,,)( 111
121211 −−=
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 96
Schedule
Lecture 9:
The minimum time problem- the minimal time optimal
problem – characterization of the optimal solution-
uniqueness result- Existence- Minimum time problem in case
of steady state system
Switching points
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 97
The linear time-optimal control
Let us consider the problem of optimal control of a linear
system
✓ with fixed initial and final state,
✓ constraints on the control variables and
✓ with cost index equal to the length of the time interval
Prof. D.Iacoviello- Optimal Control 03/12/2018 Pagina 98
SINGULAR SOLUTIONS
Definition:
Let be an optimal solution of the above problem,
and the corresponding multipliers.
The solution is singular if there exists a subinterval
in which the Hamiltonian
is independent from at least one component of in
( )of
oo tux ,,oo 0
( ) ''','',' tttt ( )tttxH ooo ),(,,),( 0
( )'',' tt
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 99
SINGULAR SOLUTIONS
Theorem:
Assume the Hamiltonian of the form
Let be an extremum and the corresponding
multipliers such that is dependent on any
component of in any subinterval of
A necessary condition for to be a singular
extremum is that there exists a subinterval
such that:
( )*** ,, ftux**
0
),,,,(),,,(),,,(),,,,( 002010 tuxNtxHtxHtuxH +=
),,,,( *0
** txN
],[ fi tt
'''],,[]'','[ tttttt fi
]'','[,0),,,( 02 ttttxH =
( )*** ,, ftux
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 100
The linear minimum time optimal control
Let us consider the problem of optimal control of a linear
system
✓ with fixed initial and final state,
✓ constraints on the control variables and
✓ with cost index equal to the length of the time interval
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 101
Problem: Consider the linear dynamical system:
With and the constraint
Matrices A and B have entries with elements of class
Cn-2 and Cn-1 respectively, at least of C1 class .
The initial instant ti is fixed and also:
)()()()()( tutBtxtAtx +=pn RtuRtx )(,)(
Rtpjtu j = ,,...,2,1,1)(
0)(,)( == fi
i txxtx
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 102
The aim is to determine the final instant
the control and the state
that minimize the cost index:
)(RCu oo
Rtof
)(1 RCxo
if
t
t
f ttdttJ
f
i
−== )(
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 103
Theorem: Necessary conditions for
to be an optimal solution are that there exist a constant
and an n-dimensional function ,
not simultaneously null and such that:
Possible discontinuities in can appear only in
the points in which has a discontinuity.
( )*** ,, ftux
0*
0 fi ttC ,1*
pjRBuB
A
j
pTT
T
,...,2,1,1:***
**
=
−=
**u
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 104
Moreover the associated Hamiltonian is a continuous function
with respect to t and it results:
Proof: The Hamiltonian associated to the problem is:
Applying the minimum principle the theorem is proved.
0*
* =ft
H
( ) BuAxtuxH TT ++= 00 ,,,,
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 105
STRONG CONTROLLABILITY
Let us indicate with the j-th column of the matrix B .
The strong controllability is guaranteed by the condition:
where :
)(tb j
( )
i
n
jjjj
ttpj
tbtbtbtG
=
=
,,...,2,1
0)()()(det)(det )()2()1(
nktbtAtbtb
tbtb
k
j
k
j
k
j
jj
,...,3,2)()()()(
)()(
)1()1()(
)1(
=−=
=
−−
Generic column of matrix B(t)
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 106
Remark:
Strong controllability corresponds to the
controllability in
any instant ti,
in any time interval and
by any component of the control vector
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 107
Remark:
In the NON-steady state case the above condition is
sufficient for strong controllability.
In the steady case it is necessary and sufficient and may
be written in the usual way:
( ) pjbAAbb jn
jj ,...,2,10det 1 =−
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 108
Characterization of the optimal solution
Theorem: Let us consider the minimal time optimal problem .
If the strong controllability condition is satisfied and if an
optimal solution exists
it is non singular.
Moreover every component of the optimal control is
piecewise constant assuming only the extreme values
and the number of discontinuity instants is limited.
1
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 109
A control function that assumes only the limit values
is called bang-bang control
and
the instants of discontinuity are called
commutation instants
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 110
Theorem (UNIQUENESS):
If the hypothesis of strong controllability is satisfied, if an
optimal solution exists it is unique.
Proof: the theorem is proved by contradiction arguments.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 111
Remark
This result holds also to the case in which the control is in
the space of measurable function defined on the space of
real number R.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 112
Existence of the optimal solution
Theorem: if the condition of strong controllability is satisfied
and an admissible solution exists, then there exists a
unique optimal solution nonsingular and the control is
bang-bang.
Proof: the existence theorem is proved on the basis of the
following results.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 113
The following result holds:
Theorem: Let assume that the control functions belong to the
space of measurable functions.
If an admissible solution exists then the optimal solution exists.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 114
The results about the characterization of the control may be
extended to the case in which the control belong to the
space of measurable functions:
Therefore it results in:
If the strong controllability condition is satisfied and
if an optimal control in the space of measurable function
exists then this solution is unique,
non singular and the control is bang bang type
Remark
The existence of the optimal solution is guaranteed only
for the couples for which the admissible solution
exists( )ii xt ,
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 115
Existence of the optimal solution
Theorem: if the condition of strong controllability is
satisfied and an admissible solution exists
with the control in the space of measurable functions,
then there exists a unique optimal solution
nonsingular and
the control is bang-bang.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 116
Minimum time problem for steady state system
Let us consider the minimum time problem in case of
steady state system.
In this case it is possible to deduce results about the
number of commutation points
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 117
Problem: Consider the linear dynamical system:
With and the constraint
The initial instant ti is fixed and also:
)()()( tButAxtx +=
pn RtuRtx )(,)(
Rtpjtu j = ,,...,2,1,1)(
0)(,)( == fi
i txxtx
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 118
The aim is to determine the final instant
the control
and the state
that minimize the cost index:
)(RCu oo
Rtof
)(1 RCxo
if
t
t
f ttdttJ
f
i
−== )(
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 119
Theorem:
Let us consider the above time minimum problem and
assume that the control function belong to the space of
measurable function ;
if the system is controllable
there exists a neighbor Ω of the origin such that for any
there exists an optimal solution.ix
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 120
Theorem: Let us consider the above time minimum
problem in the steady state case and assume that the
control function belongs to the space of measurable
function M(R).
If the system is controllable and the eigenvalues of
matrix A have negative real part
there exists an optimal solution
whatever the initial state is.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 121
Theorem: If the controllability condition is satisfied and if
all the eigenvalues of the matrix A are real non positive, then
the number of commutation instants for any components
of control is less or equal than n-1, whatever the inititial
state is.
03/12/2018Prof. D.Iacoviello - Optimal Control Pagina 122