optimality conditions in discrete optimal control problems with state constraints

Numerical Functional Analysis and Optimization, 28(7–8):945–955, 2007Copyright © Taylor & Francis Group, LLCISSN: 0163-0563 print/1532-2467 onlineDOI: 10.1080/01630560701493271

OPTIMALITY CONDITIONS IN DISCRETE OPTIMAL CONTROLPROBLEMS WITH STATE CONSTRAINTS

Boban Marinkovic � Department of Applied Mathematics,Faculty of Mining and Geology, University of Belgrade, Belgrade, Serbia

� We consider nonlinear discrete optimal control problems with variable end points andwith inequality and equality type constraints on trajectories and control. We derive first- andsecond-order necessary optimality conditions that are meaningful without a priori normalityassumptions.

Keywords Discrete optimal control; Normality; Optimality conditions.

AMS Subject Classification 90C46; 49K.

1. INTRODUCTION

Consider the following nonlinear discrete optimal control problem:

N−1∑i=0

fi(xi ,ui) → inf; (1.1)

xi+1 = �i(xi ,ui), i = 0,N − 1, (1.2)

r 1i (ui) ≤ 0, r 2i (ui) = 0, i = 0,N − 1, (1.3)

g 1i (xi) ≤ 0, g 2

i (xi) = 0, i = 0,N , (1.4)

K 1(x0, xN ) ≤ 0, K 2(x0, xN ) = 0, (1.5)

where

fi(x ,u) : Rn × Rr → R , �i(x ,u) : Rn × Rr → Rn ,

r 1i (u) : Rr → Rm1 , r 2i (u) : Rr → Rm2 ,

Address correspondence to Boban Marinkovic, Department of Applied Mathematics, Facultyof Mining and Geology, University of Belgrade, Djušina 7, Belgrade 11000, Serbia; E-mail:[email protected]

945

946 B. Marinkovic

g 1i (x) : Rn → Rs1 , g 2

i (x) : Rn → Rs2 ,

K 1(x0, xN ) : Rn × Rn → Rk1 , K 2(x0, xN ) : Rn × Rn → Rk2

are twice continuously differentiable functions. We assume that m2 ≤ r ,s2 ≤ n, and k2 ≤ 2n.

Here, xi ∈ Rn is a state variable, ui ∈ Rr is a control parameter, and Nis a given number of steps. Vector � = (x0, x1, � � � , xN ) is called a trajectory,w = (u0,u1, � � � ,uN−1) is called a control, x0 is a starting point, and xN is anend point of corresponding trajectory. Let x0 be a starting point and letw be a control. Then the pair (x0,w) defines the corresponding trajectory� = (x0, x1, � � � , xN ). If the conditions (1.2)–(1.5) are satisfied, then we saythat the pair (x0,w) is feasible. The discrete optimization problem is tominimize the function

J (x0,w) =N−1∑i=0

fi(xi ,ui),

on the set of feasible pairs. A feasible pair (x0, w) is called a weak localminimum for the problem (1.1)–(1.5) if for some � > 0, the pair (x0, w)minimizes J (x0,w) over all feasible pairs (x0,w) satisfying

‖xi − xi‖n < �, i = 0,N , ‖ui − ui‖r < �, i = 0,N − 1,

where ‖ · ‖n and ‖ · ‖r are any norms in Rn and Rr .Discrete optimal control are considered in many books and papers.

We refer, for example, to [4, 5, 8]. Specifically, we refer to [9, 10] (seealso references therein), where discrete optimal control problems withequality type of constraints on control and end points are considered.They obtained second-order optimality conditions in terms of different setsof conditions. Also, we refer to [6] where sensitivity analysis for discreteoptimal control problems with equality type of constraints is developed.

The aim of this paper is to obtain first- and second-order necessaryoptimality conditions for the problem (1.1)–(1.5) that are meaningfulwithout normality assumptions. The obtained results are based on thegeneral theory developed in [1, 2] and they generalize our recent resultsobtained in [3] and [7] to the case of discrete optimal control problemswith state constraints.

2. OPTIMALITY CONDITIONS

For convenience, we discard all constraints corresponding with indicesj such that

K 1j (x0, xN ) < 0,

Optimality Conditions in Discrete Optimal Control 947

and we assume that

K 1(x0, xN ) = 0�

Let us define the Pontryagin’s function

Hi(x ,u, p, �0) : Rn × Rr × Rn × R → R , i = 0,N − 1

by

Hi(x ,u, p, �0) = 〈p,�i(x ,u)〉 − �0fi(x ,u)

and the small Lagrangian

l(x0, xN , �1, �2) : Rn × Rn × Rk1 × Rk2 → R

by

l(x0, xN , �1, �2) = 〈�1,K 1(x0, xN )〉 + 〈�2,K 2(x0, xN )〉�Definition 2.1. The pair (x0, w) satisfies Lagrange multipliers ruleif there exists � = (�0, �1, �2, �1, �2, �1, �2, p), �0 ∈ R , �1 ∈ Rk1 , �2 ∈ Rk2 ,�1 ∈ Rs1(N+1), �2 ∈ Rs2(N+1), �1 ∈ Rm1N , �2 ∈ Rm2N , p ∈ Rn(N+1), such that� �= 0, �0 ≥ 0, �1 ≥ 0, �1 ≥ 0, �1 ≥ 0 and such that the following conditionshold:

p0 = �H0

�x(x0, u0, p1, �0)

= �l�x0

(x0, xN , �1, �2) + �g 10

�x(x0)T�01 + �g 2

0

�x(x0)T�02, (2.1)

pi = �Hi

�x(xi , ui , pi+1, �0) − �g 1

i

�x(xi)T�i1 − �g 2

i

�x(xi)T�i2, i = 1,N − 1, (2.2)

pN = − �l�xN

(x0, xN , �1, �2) − �g 1N

�x(xN )T�N1 − �g 2

N

�x(xN )T�N2 , (2.3)

�Hi

�u(xi , ui , pi+1, �0) = �r 1i

�u(ui)

T�i1 + �r 2i

�u(ui)

T�i2, i = 0,N − 1, (2.4)

〈�i1, r

1i (ui)〉 = 0, i = 0,N − 1, 〈�i1, g 1

i (xi)〉 = 0, i = 0,N � (2.5)

Note that we consider the partial derivative as the row of thecorresponding size.

Denote by = (x0, w) the set of all Lagrange multipliers �corresponding with the pair (x0, w).

948 B. Marinkovic

Put

I 1 = i : r 1i (ui) = 0�, i = 0,N − 1, I 2 = i : g 1i (xi) = 0�, i = 0,N �

Let � be the set of all vectors (h, v), h = (h0, h1, � � � , hN )T , hi ∈ Rn ,v = (v0, v1, � � � , vN−1)

T , vi ∈ Rr , such that the following conditions hold:

N−1∑i=0

�fi�x

(xi , ui)hi +N−1∑i=0

�fi�u

(xi , ui)vi ≤ 0, (2.6)

hi+1 = ��i

�x(xi , ui)hi + ��i

�u(xi , ui)vi , i = 0,N − 1, (2.7)

�r 1i�u

(ui)vi ≤ 0, ∀i ∈ I 1,�r 2i�u

(ui)vi = 0, i = 0,N − 1, (2.8)

�g 1i

�x(xi)hi ≤ 0, ∀i ∈ I 2,

�g 2i

�x(xi)hi = 0, i = 0,N , (2.9)

�K 1

�(x0, xN )(x0, xN )(h0, hN )T ≤ 0,

�K 2

�(x0, xN )(x0, xN )(h0, hN )T = 0� (2.10)

By M we denote the subspace M consisting of all (h, v) such that theconditions (2.6)–(2.10) hold, only with equalities instead inequalities in(2.8)–(2.10). Note that M represents the maximum linear subspace in �.

Put

��i

�x(xi , ui) = Ci ,

��i

�u(xi , ui) = Di ,

�r 1i�u

(ui) = E 1i ,

�r 2i�u

(ui) = E 2i , Ei = [E 1

i ,E2i ]T ,

�g 1i

�x(xi) = F 1

i ,�g 2

i

�x(xi) = F 2

i , Fi = [F 1i , F

2i ]T ,

K (x0, xN ) = (K 1(x0, xN ),K 2(x0, xN ))T ,�K�x0

(x0, xN ) = K0,�K�xN

(x0, xN ) = KN �

Let Si , i = 0,N , be the matrices defined by

S0 = K0 + KN

N−1∏s=0

Cs ,

Si = KNCiCi+1 � � �CN−1Di−1, i = 1,N − 1,

SN = KNDN−1


and let W be the block matrix

W =

S0 S1 � � � SNF0 0 � � � 0

F1C0 F1D0 0 � � � 0

F2C1C0 F2C1D1 F2D1 0 � � � 0��

��

FN∏N−1

s=0 Cs FN∏N−1

s=1 CsD0 FN∏N−1

s=2 CsD1 � � � FNDN−1

0 E0 � � � 0��

��

0 0 � � � EN−1

,

where by 0 we denote the zero matrix of the corresponding size.For a given Lagrange multiplier �, we introduce the quadratic form ��

by the formula

��[(h, v)]2 = �2l�x2

0

(x0, xN , �1, �2)[h0]2 + �2l�x2

N

(x0, xN , �1, �2)[hN ]2

+ 2⟨

�2l�x0�xN

(x0, xN , �1, �2)h0, hN

⟩−

N−1∑i=0

�2Hi

�x2(xi , ui , pi+1, �0)[hi]2

−N−1∑i=0

�2Hi

�u2(xi , ui , pi+1, �0)[vi]2 − 2

N−1∑i=0

⟨�2Hi

�x�u(xi , ui , pi+1, �0)hi , vi

⟩

+N−1∑i=0

�2

�u2〈�i

1, r1i (ui)〉[vi]2 +

N−1∑i=0

�2

�u2〈�i

2, r2i (ui)〉[vi]2

+N∑i=0

�2

�x2〈�i1, g 1

i (xi)〉[hi]2 +N∑i=0

�2

�x2〈�i2, g 2

i (xi)〉[hi]2�

Here A[x]2 = 〈Ax , x〉 stands for the image of a bilinear mapping 〈Ax , x〉.Denote by a = a(x0, w) the set of all � ∈ (x0, w) such that

indM�� ≤ (m1 + m2 + s1 + s2)N + s1 + s2 + k1 + k2 − n − rN + dim(kerW ),

where kerW is the set of all (h0, v) such that W (h0, v)T = 0� Note thatindM�� is the index of the quadratic form �� to the space M , i.e.,the maximum dimension of subspaces of M on which it is negativedefinite.

950 B. Marinkovic

Theorem 2.2. Let (x0, w) be a weak local minimum for the problem (1.1)–(1.5). Then a �= ∅ and, moreover,

max�∈a ,|�|=1

��[(h, v)]2 ≥ 0, ∀(h, v) ∈ �� (2.11)

Proof. Define the functions

f (�,w) : Rn(N+1) × RrN → R , F i(�,w) : Rn(N+1) × RrN → Rn , i = 1,N ,

and

F (�,w) : Rn(N+1) × RrN → RnN

by

f (�,w) =N−1∑i=0

fi(xi ,ui),

F i+1(�,w) = xi+1 − �i(xi ,ui), i = 0,N − 1,

and

F (�,w) = (F 1(�,w), � � � , F N (�,w))T �

We shall formally rewrite the initial problem into the followingmathematical programming problem:

f (�,w) → inf; (2.12)

F (�,w) = 0, (2.13)

r 1i (ui) ≤ 0, r 2i (ui) = 0, i = 0,N − 1, (2.14)

g 1i (xi) ≤ 0, g 2

i (xi) = 0, i = 0,N , (2.15)

K 1(x0, xN ) ≤ 0, K 2(x0, xN ) = 0� (2.16)

The pair (�, w) is a weak local minimum for the preceding problem.Let us introduce the Lagrangian function L(�,w, �) by the formula

L(�,w, �) = �0f (�,w) + 〈p, F (�,w)〉 + 〈�1,K 1(x0, xN )〉

+ 〈�2,K 2(x0, xN )〉 +N−1∑i=0

〈�i1, r

1i (ui)〉 +

N−1∑i=0

〈�i2, r

2i (ui)〉

+N∑i=0

〈�i1, g 1i (xi)〉 +

N∑i=0

〈�i2, g 2i (xi)〉,


where � = (�0, �1, �2, �1, �2, �1, �2, p), �0 ∈ R , �1 ∈ Rk1 , �2 ∈ Rk2 , �1 ∈ Rm1N ,�2 ∈ Rm2N , �1 ∈ Rs1(N+1), �2 ∈ Rs2(N+1), p ∈ RnN .

Now, we shall apply assertions of Theorem 3.1 from [1]. First-orderoptimality conditions imply that there exist Lagrange multipliers � �= 0,�0 ≥ 0, �1 ≥ 0, �1 ≥ 0, �1 ≥ 0 such that

�L�(�,w)

(�, w, �) = 0, (2.17)

〈�i1, r

1i (ui)〉 = 0, i = 0,N − 1, 〈�i1, g 1

i (xi)〉 = 0, i = 0,N � (2.18)

It follows that (2.5) holds.Denote by the set of all Lagrange multipliers corresponding with the

pair (�, w).Let us consider the mapping

� (�,w) = (F (�,w),R 1(w),R 2(w),G 1(�),G 2(�),K (x0, xN )

)T,

where

R 1(w) = (r 11 (u1), � � � , r 1N−1(uN−1)

)T, R 2(w) = (

r 21 (u1), � � � , r 2N−1(uN−1))T

and

G 1(�) = (g 10 (x0), � � � , g

1N (xN )

)T, G 2(�) = (

g 20 (x0), � � � , g

2N (xN )

)T�

Put

A = ��(�,w)

(�, w)�

It is easy to see that the operator A is given by the following block matrix

A =

−C0 I 0 � � � −D0 � � � 0��

��

0 � � � −CN−1 I 0 � � � −DN−1

0 � � � E0 � � � 0��

��

0 � � � EN

G0 0 � � � 0��

��

0 � � � GN � � � 0K0 0 � � � KN 0 � � � 0

,

952 B. Marinkovic

where by I , resp. 0, we denote the identity matrix, resp. the zero matrix,of the corresponding size.

Denote by a the set of all � ∈ such that

indkerA�2L

�(�,w)2(�, w, �) ≤ codim(imA)�

In [1] it was proved that

max�∈a ,|�|=1

�2L�(�,w)2

(�, w, �)[(h, v)]2 ≥ 0, ∀(h, v) ∈ �, (2.19)

where � is the cone of critical directions corresponding with the pair(�, w), i.e., the set of all vectors (h, v), such that the following conditionsare satisfied:

�f�(�,w)

(�, w)(h, v)T ≤ 0, (2.20)

�F�(�,w)

(�, w)(h, v)T = 0, (2.21)

�r 1i�u

(ui)vi ≤ 0, ∀i ∈ I 1,�r 2i�u

(ui)vi = 0, i = 0,N − 1, (2.22)

�g 1i

�x(xi)hi ≤ 0, ∀i ∈ I 2,

�g 2i

�x(xi)hi = 0, i = 0,N , (2.23)

�K 1

�(x0, xN )(x0, xN )(h0, hN )T ≤ 0,

�K 2

�(x0, xN )(x0, xN )(h0, hN )T = 0� (2.24)

First, it is easy to see that � = � holds.Now, let us interpret the relation (2.17). From

L(�, w, �) = �0

N−1∑i=0

fi(xi , ui) +N−1∑i=0

〈pi+1, xi+1 − �i(xi , ui)〉

+ 〈�1,K1(x0, xN )〉 + 〈�2,K2(x0, xN )〉 +N−1∑i=0

〈�i1, r

1i (ui)〉

+N−1∑i=0

〈�i2, r

2i (ui)〉 +

N∑i=0

〈�i1, g 1i (xi)〉 +

N∑i=0

〈�i2, g 2i (xi)〉

we have that

�L�x0

= �0�f0�x

(x0, u0) − ��0

�x(x0, u0)

T p1 + �g 10

�x(x0)T�01 + �g 2

0

�x(x0)T�02

+ �K1

�x0(x0, xN )T�1 + �K2

�x0(x0, xN )T�2 = 0, (2.25)


�L�xi

= �0�fi�x

(xi , ui) + pi − ��i

�x(xi , ui)

T pi+1

+ �g 1i

�x(xi)T�i1 + �g 2

i

�x(xi)T�i2 = 0, i = 1,N − 1, (2.26)

�L�xN

= pN + �g 1N

�x(xN )T�N1 + �g 2

N

�x(xN )T�N2

+ �K1

�xN(x0, xN )T�1 + �K2

�xN(x0, xN )T�2 = 0, (2.27)

�L�ui

= �0�fi�u

(xi , ui) − ��i

�u(xi , ui)

T pi+1

+ �r 1i�u

(ui)T�i

1 + �r 2i�u

(ui)T�i

2 = 0, i = 0,N − 1, (2.28)

where

�L�xi

= �L�xi

(�, w, �),�L�ui

= �L�ui

(�, w, �)�

Put

p0 = �H0

�x(x0, u0, p1, �0)� (2.29)

From (2.25) and (2.29) we have that

p0 = �l�x0

(x0, xN , �1, �2) + �g 10

�x(x0)T�01 + �g 2

0

�x(x0)T�02�

From (2.26) we obtain that

pi = �Hi

�x(xi , ui , pi+1, �0) − �g 1

i

�x(xi)T�i1 − �g 2

i

�x(xi)T�i2, i = 1,N − 1�

From (2.27) we obtain that

pN = − �l�xN

(x0, xN , �1, �2) − �g 1N

�x(xN )T�N1 − �g 2

N

�x(xN )T�N2 �

Set p = (p0, p) and � = (�0, �1, �2, �1, �2, �1, �2, p). We proved that thereexists � for which (2.1)–(2.3) hold. From (2.28) we obtain that (2.4) holds.

954 B. Marinkovic

Let us calculate codim(im A). It is easy to see that ker A is the set ofall (h, v) such that the following equalities hold:

hi+1 − Cihi − Divi = 0, i = 0,N − 1, (2.30)

E 1i vi = 0, E 2

i vi = 0, i = 0,N − 1, (2.31)

F 1i hi = 0, F 2

i hi = 0, i = 0,N , (2.32)

K0h0 + KN hN = 0� (2.33)

Solving the equations (2.30) we obtain

hi =i−1∏s=0

Csh0 +i−2∑j=0

∏j<s≤i−1

CsDjvj + Di−1vi−1, i = 1,N � (2.34)

It is easy to see that from (2.31)–(2.34), we obtain that

dim(kerA) = dim(kerW )�

From the fact that dim(imA) = codim(kerA) we have that

codim(imA) = (n + m1 + m2)N + (s1 + s2)(N + 1) + k1 + k2 − codim(kerA)

= (n + m1 + m2)N + (s1 + s2)(N + 1) + k1 + k2

−n(N + 1) − rN + dim(kerW )

= (m1 + m2 + s1 + s2)N + s1 + s2 + k1 + k2 − n − rN

+ dim(kerW )�

Let us clarify the quadratic form �2L�(�,w)2 (�, w, �). From

�2L�xi�xj

≡ 0, ∀(i , j) : i �= j , (i , j) �= (0,N );

�2L�xi�uj

≡ 0,�2L

�ui�uj≡ 0, ∀i �= j ,

we obtain that

�2L�(�,w)2

(�, w, �) = ��

Theorem 2.2 was proved. �


ACKNOWLEDGMENT

The author would like to thank A.V. Arutyunov for the valuablesuggestions and the associate editor whose comments improved thepresentation considerably.

REFERENCES

1. A.V. Arutyunov (1996). Second-order conditions in extremal problems with finite-dimensionalrange. 2-normal maps. Izv. Ross. Akad. Nauk Ser. Mat. 60(1):37–62 (English transl. in Izv. Math.60).

2. A.V. Arutyunov (2000). Optimality Conditions: Abnormal and Degenerate Problems. Kluwer AcademicPublishers, Dordrecht.

3. A.V. Arutyunov and B. Marinkovic (2005). Necessary conditions for optimality in discreteoptimal control problems. Vestnik MGU. Ser. 15. 1:43–48 (in Russian).

4. V.G. Boltyanskii (1978). Optimal Control of Discrete Systems. Wiley, New York.5. A.D. Ioffe and V.M. Tikhomirov (1979). Theory of Extremal Problems. North-Holland, Amsterdam.6. B. Marinkovic (2006). Sensitivity analysis for discrete optimal control problems. Math. Met. Op.

Res. 63(3):513–524.7. B. Marinkovic (2007). Second-order optimality conditions in a discrete optimal control problem.

Optimization (to appear).8. A.I. Propoi (1973). Elements of the Theory of Optimal Discrete Processes. Nauka, Moscow (in Russian).9. R. Hilscher and V. Zeidan (2002). Second order sufficiency criteria for a discrete optimal

control problem. J. Differ. Equations Appl. 8(6):573–602.10. R. Hilscher and V. Zeidan (2002). Discrete optimal control: second order optimality conditions.

J. Differ. Equations Appl. 8(10):875–896.

optimality conditions in discrete optimal control problems with state constraints

Documents