Weak Conservation Laws for Minimizers

which are not Pontryagin Extremals

Delfim F. M. Torres

delfim@mat.ua.pt

Department of Mathematics

University of Aveiro

3810-193 Aveiro, Portugal

http://www.mat.ua.pt/delfim

Abstract

We prove a Noether-type symmetry theorem for in-variant optimal control problems with unrestrictedcontrols. The result establishes weak conservationlaws along all the minimizers of the problems, in-cluding those minimizers which do not satisfy thePontryagin Maximum Principle.

MSC 2000: 49K15, 70H33, 37J15.

Keywords. Noether’s symmetry theorem, invari-ant optimal control problems, gap between optimal-ity and existence, weak conservation laws.

1 Introduction

Emmy Noether’s first theorem [11] is one of the mostbeautiful and fundamental results of the calculus ofvariations. The result comprises a universal princi-ple, connecting the existence of a family of transfor-mations under which the functional to be minimizedis invariant (the existence of variational symmetries)with the existence of conservation laws (first inte-grals of the Euler-Lagrange differential equations).Conservation laws can then be used to simplify theproblem of finding the minimizers. They have playedan important role, both in mathematics and physics,since the birth of the calculus of variations in theeighteen century, having been extensively used bygiants like the Bernoulli brothers, Newton, Leibniz,Euler, Lagrange, and Legendre. Conservation laws,obtained from Noether’s theorem, have a profoundeffect on a vast number of disciplines, ranging fromclassical mechanics, where they find important inter-pretations such as conservation of energy, conserva-tion of momentum, or conservation of angular mo-mentum, to engineering, economics, control theoryand their applications [8].

The first extension of Noether’s theorem to the moregeneral context of optimal control was published in1973 [5]. Since then, many Noether-like theoremshave been obtained in the context of optimal control– see [17] and references therein. We recall that allsuch versions of Noether’s theorem assume the Pon-tryagin maximum principle [13] to be satisfied, anduse its conditions, including the adjoint system, intheir proofs.

Optimal control with unbounded controls is an areaof strong current activity, because of numerous appli-cations involving modern technology such as “smartmaterials” [9]. When there are no restrictions onthe values of the control variables, as in the calculusof variations, it is well known that optimal controlproblems may present solutions for which the Pon-tryagin Maximum Principle fails to be satisfied (seee.g. [19, §11.1]). This is due to the fact that thehypotheses of the existence theory need to be com-plemented with additional regularity conditions inorder to proceed with the arguments which lead tothe maximum principle [15]: unboundedness of thecontrols “propagates” through the dynamical controlsystem, often causing a lack of regularity for the so-lutions.

In spite of the gap between the hypotheses of nec-essary optimality conditions and existence theorems,J. Ball proved [1] that, for time-invariant problems,the conservation of the Hamiltonian (conservationof energy) is still valid for minimizers which mightnot satisfy the Euler-Lagrange necessary condition.More recently, in 2002, G. Francfort and J. Sivalo-ganathan proposed a generalization of Ball’s result,giving some applications to hyper-elasticity [7].

In this note we extend the previous results [7] fromthe calculus of variations framework to the optimalcontrol setting. We obtain weak conservation lawsfor minimizers which do not necessarily satisfy the

0-7803-9235-3/05/$20.00 © 2005 IEEE PhysCon 2005, St. Petersburg, Russia

134

Pontryagin maximum principle.

2 Bad Behavior in Optimal Control

The optimal control problem consists to minimize acost functional

I [x(·), u(·)] =∫ b

a

L (t, x(t), u(t)) dt

subject to a control system described by ordinarydifferential equations

x(t) = ϕ (t, x(t), u(t)) (1)

together with certain appropriate endpoint condi-tions. The Lagrangian L : [a, b]×R

n ×Rm → R and

the velocity vector ϕ : [a, b] × Rn × R

m → Rn are

given, and assumed to be smooth: L(·, ·, ·), ϕ(·, ·, ·)∈ C1. We are interested in the case where there areno restrictions on the control set: u(t) ∈ R

m. Wedenote the problem by (P ). In the particular caseϕ(t, x, u) = u, one obtains the fundamental problemof the calculus of variations, which covers all classi-cal mechanics. The choice of the classes X and U ,respectively of the state x : [a, b] → R

n and controlvariables u : [a, b] → R

m, play an important role inour discussion. In connection with the Pontryaginmaximum principle, the optimal controls are typi-cally assumed to be essentially bounded, U = L∞[13]; while to guarantee existence, compactness ar-guments require a bigger class of measurable con-trol functions, U = L1 [3]. Given an optimal con-trol problem with unrestricted controls, it may hap-pen that the Pontryagin maximum principle is valid,while existence of minimizers is not guaranteed; orit may happen that the minimizers predicted bythe existence theory fail to be Pontryagin extremals.Conservation laws are obtained from Noether’s the-orem, assuming that the minimizers are Pontryaginextremals. In this work our objective is to proveweak conservation laws valid for bad-behaved prob-lems with minimizers which are not Pontryagin ex-tremals. We begin to explain why bad-behavior canoccur.

The Pontryagin maximum principle is a necessaryoptimality condition which can be obtained from ageneral Lagrange multiplier theorem in spaces of in-finite dimension (cf. e.g. [10, 12]). Introducing theHamiltonian function

H(t, x, u, ψ) = −L(t, x, u) + ψ · ϕ(t, x, u) , (2)

where ψi, i = 1, . . . , n, are the “Lagrange multipli-ers” or the “generalized momenta”, the multipliertheorem asserts that the optimal control problem

is equivalent to the maximization of the augmentedfunctional

J [x(·), u(·), ψ(·)]

=∫ b

a

(H (t, x(t), u(t), ψ(t)) − ψ(t) · x(t)) dt . (3)

Let us assume, for simplicity, X = C1 ([a, b]; Rn),U = C ([a, b]; Rm). Let

(x(·), u(·), ψ(·)

)solve

the problem, and consider arbitrary C1-functionsh1 , h3 : [a, b] → R

n, h1(·) vanishing at a and b(h1(·) ∈ C1

0 ([a, b])), and arbitrary continuous h2 :[a, b] → R

m. Let ε be a scalar. By definition ofmaximizer, we have

J[(x + εh1)(·), (u + εh2)(·), (ψ + εh3)(·)

]≤ J

[x(·), u(·), ψ(·)

],

and one has the following necessary condition:

d

dεJ

[(x + εh1)(·), (u + εh2)(·), (ψ + εh3)(·)

]∣∣∣ε=0

= 0 .

(4)Differentiating (4) gives

0 =∫ b

a

[∂H

∂x

(t, x(t), u(t), ψ(t)

)· h1(t)

+∂H

∂u

(t, x(t), u(t), ψ(t)

)· h2(t)

+∂H

∂ψ

(t, x(t), u(t), ψ(t)

)· h3(t)

−h3(t) · ˙x(t) − ψ(t) · h1(t)]dt . (5)

Integrating the ψ(t) · h1(t) term by parts, and havingin mind that h1(a) = h1(b) = 0, one derives

∫ b

a

[(∂H

∂x

(t, x(t), u(t), ψ(t)

)+ ˙

ψ(t))· h1(t)

+∂H

∂u

(t, x(t), u(t), ψ(t)

)· h2(t)

+(

∂H

∂ψ

(t, x(t), u(t), ψ(t)

)− ˙x(t)

)· h3(t)

]dt = 0 .

(6)

Note that (6) was obtained for any variation h1(·),h2(·), and h3(·). Choosing h1(t) = h2(t) ≡ 0, andh3(·) arbitrary, one obtains the control system (1):

˙x(t) =∂H

∂ψ

(t, x(t), u(t), ψ(t)

), t ∈ [a, b] . (7)

With h1(·) arbitrary, and h2(t) = h3(t) ≡ 0, we ob-tain the adjoint system:

˙ψ(t) = −∂H

∂x

(t, x(t), u(t), ψ(t)

), t ∈ [a, b] . (8)

135

Finally, with h2(·) arbitrary, and h1(t) = h3(t) ≡ 0,the maximality condition is obtained:

∂H

∂u

(t, x(t), u(t), ψ(t)

)= 0 , t ∈ [a, b] . (9)

A necessary optimality condition for (x(·), u(·)) to bea minimizer of problem (P ) is given by the Pontrya-gin maximum principle: there exists ψ(·) such thatthe 3-tuple

(x(·), u(·), ψ(·)

)satisfy all the conditions

(7), (8), and (9). We recall that conditions (7), (8),and (9) imply the equality

d

dtH

(t, x(t), u(t), ψ(t)

)=

∂H

∂t

(t, x(t), u(t), ψ(t)

).

(10)For piecewise smooth state trajectories, X =PC1 ([a, b]; Rn), and piecewise continuous controls,U = PC ([a, b]; Rm), similar arguments than thoseused to derive conditions (7), (8), and (9) continueto be justifiable. In fact, as already mentioned, thearguments can be carried out for essentially boundedcontrols, U = L∞ ([a, b]; Rm), and Lipschitzian statevariables, X = W1,∞ ([a, b]; Rn). But if one tookU = L1 ([a, b]; Rm), and X = W1,1 ([a, b]; Rn), as re-quired by the existence theory, integration by partsof ψ(t) · h1(t) in (5) can no longer be justified, andone can not conclude with the adjoint system (8)and equality (10). This is more than a techni-cal difficulty, and explains the possibility of bad-behavior illustrated by the Ball-Mizel example [2].In the Ball-Mizel problem one has n = m = 1,L(t, x, u) = (x3 − t2)2u14 + εu2, and ϕ(t, x, u) = u:∫ 1

0

[(x3(t) − t2

)2u(t)14 + ε u(t)2

]dt −→ min ,

x (t) = u (t) ,

x (0) = 0 , x (1) = k .

For some values of the constants ε and k there ex-ists the unique optimal control u(t) = 2k

3 t−1/3 [4],which belongs to L1 but not to L∞. The Pontryaginmaximum principle is not satisfied since the adjointsystem (8)

˙ψ(t) = −∂H

∂x

(t, x(t), u(t), ψ(t)

) ∼= t−4/3

is not integrable. In this paper we obtain a newversion of Noether’s theorem, without using the ad-joint system (8) and the property (10). This makesNoether’s principle valid both for well- and bad-behaved optimal control problems.

3 Conservation Laws in Optimal Control

In 1918 Emmy Noether established the key result tofind conservation laws in the calculus of variations

[11,18]. We sketch here the standard argument usedto derive Noether’s theorem and conservation lawsin the optimal control setting (cf. e.g. [5, 14]).

Let us consider a one-parameter group of C1-transformations of the form

hs(t, x, u, ψ) = (ht(t, x, u, ψ, s), hx(t, x, u, ψ, s),hu(t, x, u, ψ, s), hψ(t, x, u, ψ, s)) , (11)

where s denote the independent parameter of thetransformations. We require that to the parametervalue s = 0 there corresponds the identity transfor-mation:

h0(t, x, u, ψ) = (t, x, u, ψ) . (12)

Associated to the group of transformations (11) weconsider the infinitesimal generators

T (t, x, u, ψ) =d

dsht(t, x, u, ψ, s)

∣∣∣∣s=0

,

X(t, x, u, ψ) =d

dshx(t, x, u, ψ, s)

∣∣∣∣s=0

,

U(t, x, u, ψ) =d

dshu(t, x, u, ψ, s)

∣∣∣∣s=0

,

Ψ(t, x, u, ψ) =d

dshψ(t, x, u, ψ, s)

∣∣∣∣s=0

.

(13)

Definition 1 The optimal control problem (P ) issaid to be invariant under a one-parameter group ofC1-transformations (11) if, and only if,

d

ds

{[H (hs (t, x(t), u(t), ψ(t)))

−hψ (t, x(t), u(t), ψ(t), s) ·dhx(t,x(t),u(t),ψ(t),s)

dtdht(t,x(t),u(t),ψ(t),s)

dt

]

·dht (t, x(t), u(t), ψ(t), s)dt

}∣∣∣∣∣s=0

= 0 , (14)

with H the Hamiltonian (2).

Having in mind (12), condition (14) is equivalent to

∂H

∂tT +

∂H

∂x· X +

∂H

∂u· U +

∂H

∂ψ· Ψ − Ψ · x(t)

− ψ(t) · d

dtX + H

d

dtT = 0 , (15)

where here, and to the end of the paper, allfunctions are evaluated at (t, x(t), u(t), ψ(t)) when-ever not indicated. Along a Pontryagin extremal(x(·), u(·), ψ(·)) equalities (7), (8), (9), and (10) are

136

in force, and (15) reduces to

dH

dtT − ψ(t) · X − ψ(t) · dX

dt+ H

dT

dt= 0

⇔ d

dt(ψ(t) · X − HT ) = 0 .

We have just proved Noether’s theorem for optimalcontrol problems.

Theorem 1 (Noether’s Theorem) If the opti-mal control problem is invariant under (11), in thesense of Definition 1, then

ψ(t) · X (t, x(t), u(t), ψ(t))− H (t, x(t), u(t), ψ(t))T (t, x(t), u(t), ψ(t)) = c

(16)

(c a constant; t ∈ [a, b]; T and X are given accordingto (13); H is the Hamiltonian (2)) is a conservationlaw, that is, (16) is valid along all the minimizers(x(·), u(·)) of (P ) which are Pontryagin extremals.

All available versions of Noether’s theorem found inthe literature are valid only for well-behaved opti-mal control problems (conservation laws are, by de-finition, valid for minimizers which are Pontryaginextremals). In the next section we provide the firstoptimal control version of Noether’s theorem valid inpresence of bad behavior (valid also for minimizerswhich are not Pontryagin extremals). For that weneed a new notion of conservation law.

4 Weak Conservation Laws in Optimal

Control

In 1879 Paul duBois-Reymond proved an importantbasic result. From duBois-Reymond lemma we knowthat∫ b

a

(ψ(t) · X − HT ) θ(t)dt = 0 , ∀ θ(·) ∈ C10 ([a, b]),

is a weak form of conservation law (16). Follows themain result of the paper.

Theorem 2 If the optimal control problem is invari-ant under (11), in the sense of Definition 1, then

∫ b

a

[ψ(t) · X (t, x(t), u(t), ψ(t))

−H (t, x(t), u(t), ψ(t))T (t, x(t), u(t), ψ(t))]θ(t)dt = 0

(θ(·) is an arbitrary W1,1([a, b]; R) function satisfy-ing θ(a) = θ(b) = 0) holds along all the minimizers(x(·), u(·), ψ(·)) ∈ W1,1 × L1 × W1,1 of (3).

Proof: Replacing the parameter s of the group(11) by function sθ(t), the infinitesimal generatorsare then given by

d

dsht (t, x, u, ψ, sθ(t))

∣∣∣∣s=0

= T (t, x, u, ψ)θ(t) ,

d

dshx (t, x, u, ψ, sθ(t))

∣∣∣∣s=0

= X(t, x, u, ψ)θ(t) ,

d

dshu (t, x, u, ψ, sθ(t))

∣∣∣∣s=0

= U(t, x, u, ψ)θ(t) ,

d

dshψ (t, x, u, ψ, sθ(t))

∣∣∣∣s=0

= Ψ(t, x, u, ψ)θ(t) ,

with T , X, U , and Ψ as in (13), and the necessaryand sufficient condition of invariance (15) takes theform(

∂H

∂tT +

∂H

∂x· X +

∂H

∂u· U +

∂H

∂ψ· Ψ

−Ψ · x(t) + HdT

dt− ψ(t) · dX

dt

)θ(t)

+ (HT − ψ(t) · X) θ(t) = 0 . (17)

Condition (5) with h1(t) = Xθ(t), h2(t) = Uθ(t),and h3(t) = Ψθ(t), gives

∫ b

a

[(∂H

∂x· X +

∂H

∂u· U +

∂H

∂ψ· Ψ − Ψ · x(t)

−ψ(t) · dX

dt

)θ(t) − (ψ(t) · X) θ(t)

]dt = 0 . (18)

Using (15) in (18) permits to write

∫ b

a

[(−∂H

∂tT − H

dT

dt

)θ(t)−(ψ(t) · X) θ(t)

]dt = 0 ;

(19)while, on the other hand, using (17) in (18), oneobtains∫ b

a

[(∂H

∂tT + H

dT

dt

)θ(t)+(HT ) θ(t)

]dt = 0 . (20)

The conclusion follows summing up (19) and (20):

∫ b

a

(HT − ψ(t) · X) θ(t)dt = 0 .

Invariance under an infinite continuous group oftransformations, which rather than dependence onparameters depend upon arbitrary functions, is con-sidered by Noether in the original paper [11]. This issometimes called “the second Noether theorem”. Werefer the reader to [16] for an extension of the secondNoether theorem to optimal control problems which

137

are semi-invariant under symmetries depending uponk arbitrary functions of the independent variable andtheir derivatives. Theorem 2 is easily formulated un-der more general notions of invariance.

Theorem 2 gives, from the invariance propertiesof the optimal control problems, weak conservationlaws along all the minimizers, including those min-imizers which does not satisfy the standard neces-sary optimality conditions like the Pontryagin Max-imum Principle or the Euler-Lagrange differentialequations. Such fact may be useful to identify moregeneral classes of well- and bad-behaved problems inthe calculus of variations and optimal control, e.g.,to synthesize a broad class of invariant problems ex-hibiting the Lavrentiev phenomenon [6]. This pos-sibility is under investigation and will be addressedelsewhere.

Acknowledgements

This work was partially supported by the Por-tuguese Foundation for Science and Technology(FCT) through the Control Theory Group (cotg) ofthe Centre for Research in Optimization and Control(CEOC).

References

[1] J. M. Ball, Minimizers and the Euler-Lagrange equa-tions, Trends and applications of pure mathematics tomechanics (Palaiseau, 1983), 1984, pp. 1–4. [MR 755716

(86e:49022)] [Zbl 0547.73013]

[2] J. M. Ball and V. J. Mizel, One-dimensional varia-tional problems whose minimizers do not satisfy theEuler-Lagrange equation, Arch. Rational Mech. Anal.90 (1985), no. 4, 325–388. [MR 801585 (86k:49002)]

[Zbl 0585.49002]

[3] Lamberto Cesari, Optimization—theory and applica-tions, Applications of Mathematics (New York), vol. 17,Springer-Verlag, New York, 1983. Problems with or-dinary differential equations. [MR 688142 (85c:49001)]

[Zbl 0506.49001]

[4] Frank H. Clarke and R. B. Vinter, On the conditionsunder which the Euler equation or the maximum prin-ciple hold, Appl. Math. Optim. 12 (1984), no. 1, 73–79.[MR 756513 (85m:49051)] [Zbl 0559.49012]

[5] Dorde S. Dukic, Noether’s theorem for optimum con-trol systems, Internat. J. Control (1) 18 (1973), 667–672.[MR 0341229 (49 #5979)] [Zbl 0281.49009]

[6] M. Foss, Examples of the Lavrentiev phenomenonwith continuous Sobolev exponent dependence, J. Con-vex Anal. 10 (2003), no. 2, 445–464. [MR 2043868

(2004m:49044)] [Zbl pre02066990]

[7] G. Francfort and J. Sivaloganathan, On conserva-tion laws and necessary conditions in the calculus ofvariations, Proc. Roy. Soc. Edinburgh Sect. A 132(2002), no. 6, 1361–1371. [MR 1950811 (2003j:49035)]

[Zbl 1027.49015]

[8] A. M. Gugushvili, O. G. Khutsishvili, V. K. Sesadze, G.N. Dalakishvili, N. A. Mchedlishvili, T. G. Khutsishvili,V. M. Kekenadze, and D. F. M. Torres, Symmetries andconservation laws in optimal control systems, GeorgianTechnical University, Tbilisi, 2003.

[9] Irena Lasiecka, Optimal control problems and Riccatiequations for systems with unbounded controls and par-tially analytic generators – applications to boundary andpoint control problems, Functional Analytic Methods forEvolution Equations, 2004, pp. 313–369.

[10] Antonio Leitao, Calculo variacional e controle otimo,Publicacoes Matematicas do IMPA. [IMPA MathematicalPublications], Instituto de Matematica Pura e Aplicada(IMPA), Rio de Janeiro, 2001 (Portuguese). 23o ColoquioBrasileiro de Matematica. [23rd Brazilian MathematicsColloquium]. [MR 1846384] [Zbl pre01659603]

[11] Emmy Noether, Invariant variation problems, TransportTheory Statist. Phys. 1 (1971), no. 3, 186–207. Translatedfrom the German (Nachr. Akad. Wiss. Gottingen Math.-Phys. Kl. II 1918, 235–257). [MR 0406752 (53 #10538)]

[Zbl 0292.49008]

[12] Pablo Pedregal, Introduction to optimization, Textsin Applied Mathematics, vol. 46, Springer-Verlag,New York, 2004. [MR 2014219 (2004j:90002)]

[Zbl 1036.90002]

[13] L. S. Pontryagin, V. G. Boltyanskiı, R. V. Gamkrelidze,and E. F. Mishchenko, Selected works. Vol. 4, Clas-sics of Soviet Mathematics, Gordon & Breach SciencePublishers, New York, 1986. The mathematical theoryof optimal processes; Edited and with a preface by R.V. Gamkrelidze Translated from the Russian by K. N.Trirogoff Translation edited by L. W. Neustadt With apreface by L. W. Neustadt and K. N. Trirogoff Reprint ofthe 1962 English translation. [MR 898009 (90a:01108)]

[Zbl 0882.01027]

[14] Delfim F. M. Torres, Conservation laws in optimal con-trol, Dynamics, bifurcations, and control (Kloster Irsee,2001), 2002, pp. 287–296. [MR 1901565 (2003c:49037)]

[Zbl pre01819752]

[15] Delfim F. M. Torres, Lipschitzian regularity of theminimizing trajectories for nonlinear optimal con-trol problems, Math. Control Signals Systems 16(2003), no. 2-3, 158–174. [MR 2006825 (2004i:49047)]

[Zbl 1041.93023]

[16] Delfim F. M. Torres, Gauge symmetries and Noether cur-rents in optimal control, Appl. Math. E-Notes 3 (2003),49–57 (electronic). [MR 1980565 (2004g:49038)]

[Zbl pre01925174]

[17] Delfim F. M. Torres, Quasi-invariant optimal controlproblems, Port. Math. (N.S.) 61 (2004), no. 1, 97–114.[MR 2040245] [Zbl 1042.49015]

[18] Bruce van Brunt, The calculus of variations, Univer-sitext, Springer-Verlag, New York, 2004. [MR 2004181

(2004m:49001)] [Zbl 1039.49001]

[19] Richard Vinter, Optimal control, Systems & Con-trol: Foundations & Applications, Birkhauser BostonInc., Boston, MA, 2000. [MR 1756410 (2001c:49001)]

[Zbl 0952.49001]

138