sufficient conditions for relative minima of broken extremals in optimal control theory

J. Math. Anal. Appl. 269 (2002) 98–128

www.academicpress.com

Sufficient conditions for relative minima ofbroken extremals in optimal control theory

John Noblea and Heinz Schättlerb,∗

a The Boeing Company, St. Louis, MO 63166, USAb Department of Systems Science and Mathematics, Washington University, St. Louis,

MO 63130-4899, USA

Received 23 April 2001; received in revised form 9 October 2001; accepted 1 November 2001

Submitted by B.S. Mordukhovich

Abstract

We formulate a version of the method of characteristics based on parametrizationsof extremals by their terminal values. Sufficient conditions are given for imbedding areference trajectory into a local field of broken extremals. For a problem with terminalmanifold of codimension 1 it is shown that a broken extremal is a relative minimum if(i) the restrictions of the flow to intervals where the control is continuous have nonsingularpartial derivatives with respect to the parameter and (ii) the switching surfaces are crossedtransversally. 2002 Elsevier Science (USA). All rights reserved.

1. Introduction

In this paper we develop sufficient conditions for a relative minimum for bro-ken extremals in an optimal control problem based on the method of characteris-

This material is based upon work supported by the National Science Foundation under grantNo. 9971747. Any opinions, findings, and conclusions or recommendations expressed in this materialare those of the authors and do not necessarily reflect the views of the National Science Foundation.

* Corresponding author.E-mail addresses:[email protected] (J. Noble), [email protected] (H. Schättler).

0022-247X/02/$ – see front matter 2002 Elsevier Science (USA). All rights reserved.PII: S0022-247X(02)00008-2

J. Noble, H. Schättler / J. Math. Anal. Appl. 269 (2002) 98–128 99

tics. Briefly, we consider the problem to minimize

J (u) =T∫

τ

L(t, x,u) dt + ϕ(T , x(T )) (1.1)

subject to dynamicsx = f (t, x,u), initial conditionx(τ) = ξ , and terminal con-straint(T , x(T )) ∈ N whereN is an imbedded submanifold ofR × R

n. The statespace isRn and admissible controls are Lebesgue-measurable functionsu whichtake values in a control setU ⊂ R

m.The Pontryagin Maximum Principle [1] gives first-order necessary conditions

for optimality of an input–trajectory pair(x∗, u∗). Pairs which satisfy thenecessary conditions of the Maximum Principle are called extremals. Intuitively(a precise definition will be given in Section 2.2), if the corresponding controlshave discontinuities we speak ofbroken extremals. We say a reference extremalΓ defines arelative minimumif there exists a setT of trajectories which containsΓ so thatΓ minimizes the objectiveJ (u) over all trajectories inT . If T consistsof all trajectories which steerξ into N , thenΓ is optimal.

In general the necessary conditions for optimality of the Maximum Principleare not sufficient and extremals need not be optimal. In fact, in optimal control,like in the classical calculus of variations, there exists a sizable gap betweenthe theories of necessary and sufficient conditions. Sufficiency theories usuallycenter around the related notions of fields of extremals or Hamilton–Jacobi theory.The latter is realized in optimal control by studying solutions to the so-calledHamilton–Jacobi–Bellman equation which for problem (1.1) reads

Vt(t, x) + minu∈U

Vx(t, x)f (t, x,u) + L(t, x,u) ≡ 0 (1.2)

subject to the boundary conditionV (t, x) = ϕ(t, x) for (t, x) ∈ N . For systemswith special structure, like linear–quadratic optimal control problems, thisequation may be solved explicitly. In general, however, even if the equation in(1.2) is strictly convex inu, the minimization leads to a highly nonlinear equationwhich can no longer be solved explicitly. Quite often, these solutions also exhibitsingularities which makes their analysis a difficult and challenging problem.

Various approaches have been pursued in the literature to tackle this problem.Recently PDE techniques seem to be at the forefront of research in the contextof viscosity solutions [2,3]. The classical method to solve first-order partial dif-ferential equations is the method of characteristics [4], and this method can beadjusted for application to the optimal control problem. Indeed, the characteris-tic equations for the Hamilton–Jacobi–Bellman equation are precisely the systemand adjoint equations given in the Maximum Principle [5] and the minimum con-dition in the Maximum Principle becomes the minimum condition in (1.2). Thisis the background of traditional optimal control techniques onregular synthesisgoing back to Boltyansky [6] in the early sixties. By now several refinementsof Boltyansky’s original construction have been given (e.g., Brunovsky [7] and

100 J. Noble, H. Schättler / J. Math. Anal. Appl. 269 (2002) 98–128

Piccoli and Sussmann [8]), yet all of these retain the basic principle of construct-ing the optimal value function by synthesizing a family of extremals and theircorresponding controls which solve (1.2). Newer results differ from the originalversions in the much weaker technical assumptions which one needs to make tohandle the singularities which arise in the construction of the value function. Upto now the most general version in making minimal assumptions is given in thepaper by Piccoli and Sussmann [8].

In this paper we develop the method of characteristics based on injectivityproperties of the flow of extremals for families of broken extremals. This flowis obtained by integrating the equations given in the necessary conditions foroptimality backward from the terminal manifold. While the method of character-istics is a classical and well-known technique in optimal control, our formulationemphasizing the mapping properties is to the best of our knowledge new.A similar approach was pursued by Young [9] and Nowakowski [10], but in thisapproach the role of the mapping properties is very much clouded by imprecisedefinitions like “descriptive” maps. Consequently Young’s results have foundlittle application, although many of the geometric problems in constructing asynthesis are explained and documented in his book. In our view his constructionsare obstructed by a strive for too much generality. In contrast, it was not our aimto be of utmost generality, but to keep the results easily verifiable and directlyapplicable.

The literature on higher-order necessary or sufficient conditions for optimalityof bang–bang controls is scarce. One of the earliest papers dealing with this issueexplicitly is [11] by Bressan who used the notion of directional convexity to de-velop high-order tests for optimality of bang–bang controls. Similar constructionswere later used by Schättler [12] in the local analysis of the optimality of bang–bang controls in dimension three. Sarychev [13] has introduced extended firstand second variations for bang–bang controls which add Dirac measures at theswitching times and he obtained corresponding first- and second-order conditionsfor optimality of bang–bang controls. Agrachev and Gamkrelidze [14,15] initi-ated a general research program which extends the use of Hamiltonian methods,especially symplectic geometry, in the study of optimal control problems. Earlyresults, giving necessary and sufficient conditions for optimality of extremal tra-jectories in terms of Morse and Maslov indices, were general and abstract, butrather difficult to apply. These results were developed further by Agrachev, Ste-fani and Zezza to obtain general sufficient conditions for strong minima of trajec-tories [16]. While this paper was under review we have become aware of a recentpaper by the same authors [17] which develops these conditions for bang–bangcontrols and also gives an algorithmic way to evaluate the positive definitenessof an associated quadratic form which allows to determine optimality. Essentiallythese conditions are obtained by varying the switching times of the reference con-trol. The assumptions made in [17] are in the spirit of our results below (exceptthat Agrachev, Stefani and Zezza consider the case with an arbitrary terminal man-


ifold, but for systems which are linear in the control), but the actual computationsare different and the precise relations between the two approaches still need tobe analyzed. The conditions we develop below are applicable to general systemsand reference trajectories which have “corners” (in the sense of Definition 2.3),not just to bang–bang controls. However, we need that the reference extremal canlocally be imbedded into a sufficiently smooth field of extremals. While this iscertainly not always the case, typically it will be satisfied for a generic trajectory.In fact, the assumptions made in [17] precisely relate to the ability of imbeddinga bang–bang extremal into a local field of bang–bang extremals by varying theswitching times.

The organization of the paper is as follows: In Section 2 we develop themethod of characteristics emphasizing mapping properties of the family ofextremals. By postponing injectivity requirements to the last step, the significanceof specific assumptions becomes clear. Differentiable solutions to the Hamilton–Jacobi–Bellman equation are constructed for both continuous controls and brokenextremals. We show that the value function remains continuously differentiableat switching surfaces which are codimension 1 imbedded submanifolds and arecrossed transversally by the optimal flow. These results, some of them classical,form the background for Section 3 where we localize the construction to the casewhen the parametrization set is a small neighborhood of a reference value whichrepresents a reference trajectory. Conditions are given which allow to imbed thereference trajectory into a local field of extremals and local characterizationsof regularity and transversality conditions at switchings are derived. In orderto simplify the presentation in this paper we only consider the considerablyless technical case when the terminal manifoldN has codimension 1. In [18]the proper concepts are developed under the terminologyterminal configurationwhich allows for a codimension 1 stratified set off the terminal manifold wherethe flow can overlap, but is memoryless.

2. The method of characteristics in optimal control

We show how a solution to the Hamilton–Jacobi–Bellman equation can beconstructed on a setR from a parametrized family of normal extremals whichsatisfy the Maximum Principle and cover the setR injectively. The constructionitself is classical and is the method of characteristics adapted to the optimalcontrol problem. The constructions below are based on ideas of Knobloch [19]and give a refinement of the proof in [20] by moving the entire constructionfrom the state space into the parameter space. Initially no injectivity assumptionsare made in the definitions and the precise roles of assumptions come out. Theconstruction is carried out both for continuous and piecewise continuous controlsto cover broken extremals.


2.1. Formulation of the problem

LetU be a subset ofRm, thecontrol set, and denote byU the class of all locallybounded Lebesgue measurable maps defined on some intervalI ⊂ R with valuesin U , u : I → U , thespace of(admissible) controls. Supposef :R×R

n ×Rm →

Rn, (t, x,u) → f (t, x,u), the dynamics of the control system, is a continuous map

which for fixedt ∈ R is r-times continuously differentiable in(x,u). Then, givena controlu ∈ U defined over the intervalI , the resulting time-varyingcontrolledvector fieldf u(t, x) = f (t, x,u(t)) satisfies theCr -Caratheodory conditions [21]and therefore the initial value problem

x = f u(t, x) = f (t, x,u(t)), x(τ ) = ξ, τ ∈ I, (2.1)

has a unique solutionxu(t; τ, ξ) defined on an maximal open interval of definitionJ = (tu−(τ, ξ), tu+(τ, ξ)) ⊂ I . That is, the functionxu is absolutely continuous onJ , satisfies the initial conditionxu(τ ; τ, ξ) = ξ , and satisfies Eq. (2.1) almosteverywhere onJ . Furthermore, as a function of the initial timeτ and the initialvalueξ , xu is r-times continuously differentiable [21]. Call this solutionxu thetrajectorycorresponding to the controlu ∈ U and call the pair(xu,u) acontrolledtrajectory. Henceforth, for convenience of notation we drop the superscriptu.

Let N be a k-dimensional imbedded submanifold of(t, x)-spaceR × Rn,

the terminal manifold. Thus near every pointq ∈ N there exist an open setΩ ⊂ R × R

n containingq and a mapΨ :Ω → R × Rn with componentsψi ,

i = 0, . . . , n − k, which have linearly independent gradients∇ψi and satisfyN ∩ Ω = (t, x) ∈ Ω : ψi(t, x) = 0, i = 0, . . . , n − k. The functionϕ :N → R

is a penalty term which is defined and sufficiently smooth on the terminalmanifoldN . Locally ϕ can always be extended to a smooth function inR × R

n.Let L :R×R

n ×Rm → R, (t, x,u) → L(t, x,u), be a continuous function which

for t fixed is r-times continuously differentiable in(x,u). This function willdefine theLagrangianof the optimal control problem. We consider the problemto minimize overU a cost functional given in Bolza form as

J (u; τ, ξ) =T∫

τ

L(t, x,u) ds + ϕ(T , x(T )) (2.2)

subject to the following dynamics, initial and terminal conditions:

x = f u(t, x) = f (t, x,u), x(τ ) = ξ, (T , x(T )) ∈ N. (2.3)

The terminal timeT is free. (A fixed terminal time would be modelled as ad-ditional terminal constraint in this set-up.)

The Pontryagin Maximum Principle [1] gives necessary conditions for acontrolled trajectory(x,u) to be optimal. In our notation we distinguish betweentangent vectors which we write as column vectors (such asx, f (t, x,u), etc.)and cotangent vectors which we write as row vectors (like the multipliersλ


andν in the statement of the Maximum Principle below). We denote the spaceof n-dimensional row vectors by(Rn)∗. Define the Hamiltonian functionH ,H :R × [0,∞) × (Rn)∗ × R

n × Rm → R, as

H(t,λ0, λ, x,u) = λ0L(t, x,u)+ λf (t, x,u). (2.4)

Theorem 2.1 [1,5,22]. Suppose the controlled trajectory(x,u) defined overthe interval [τ, T ] is optimal. Then there exist a constantλ0 0, a covectorν ∈ (Rn+1−k)∗ and an absolutely continuous functionλ : [τ, T ] → (Rn)∗ whichis a solution to the adjoint equation

λ(t) = −λ0Lx(t, x(t), u(t)) − λ(t)fx(t, x(t), u(t)), (2.5)

such that(λ0, λ(t)) = 0 for all t ∈ [τ, T ] and the following minimum condition issatisfied almost everywhere in[τ, T ]:

H(t,λ0, λ(t), x(t), u(t)) = minv∈U

H(t, λ0, λ(t), x(t), v). (2.6)

In addition the vector(H + λ0ϕt ,−λ + λ0ϕx) is orthogonal to the terminalconstraint at the endpoint; i.e., at the terminal timeT we have the followingtransversality condition:

0= λ0ϕt + νDtΨ + H, λ = λ0ϕx + νDxΨ. (2.7)

We call controlled trajectories(x,u) for which there exist multipliersλ0, λ

andν such that the conditions of the Maximum Principle are satisfiedextremalsand sometimes we refer to the triple(λ, x,u) as anextremal lift. Note that theconditions are linear in the multipliers(λ0, λ) and thus it is possible to normalizethis vector. In particular, ifλ0 > 0, then we can divide byλ0 and thus assumeλ0 = 1. These kind of extremals are callednormalwhile extremals withλ0 = 0are calledabnormal. The existence of optimal abnormal extremals can in generalnot be ruled out.

2.2. Parametrized families of extremals

We parametrize extremals through their endpoints in the terminal manifoldN

(k-dimensional) and the vectorν in the transversality condition which gives theterminal condition for the multiplierλ ((n + 1 − k)-dimensional). However, wealso need to enforce the transversality condition (2.7) onH which pins down theterminal time. Hence the parameter space isn-dimensional.

Definition 2.2. A Cr -parametrized familyE of extremal lifts is an 8-tuple(P ;T ; ξ, ν; x,u,λ0, λ) consisting of:

• An n-dimensional manifoldP and a pairT = (tin, tf ) of r-times contin-uously differentiable functionstin :P → R and tf :P → R which satisfy


tin(p) < tf (p) for all p ∈ P . They define the domain of the parametriza-tion asD = (t,p): p ∈ P , t ∈ Ip = [tin(p), tf (p)]. The functionstin andtfdefine the (compact) intervals of definition for the controlled trajectories withtf denoting the terminal time.

• An r-times continuously differentiable functionξ :P → N which parame-trizes the terminal conditions for the states.

• Extremal lifts consisting of controlled trajectories(x,u) :D → Rn × U ,

corresponding adjoint vectorsλ0 :P → [0,∞) and λ :D → (Rn)∗ and afunction ν :P → (Rn+1−k)∗ which parametrizes the terminal conditions ofthe costates. Specifically, we assume(1) the multipliers(λ0(p),λ(t,p)) are nontrivial for allt ∈ Ip ,(2) the controlsu = u(·,p), p ∈ P , parametrize admissible controls which

are continuous in(t,p) and for t fixed dependr-times continuouslydifferentiable onp with the derivatives continuous in(t,p),

(3) the trajectoriesx = x(t,p) solve the following terminal value problemsfor the dynamics

x(t,p) = f (t, x(t,p),u(t,p)), x(tf (p),p) = ξ(p) ∈ N,

(4) the costateλ = λ(t,p) solves the corresponding adjoint equation

λ(t,p) = −λ0(p)Lx(t, x(t,p),u(t,p))

− λ(t,p)fx(t, x(t,p),u(t,p)),

with terminal conditions

λ(tf (p),p) = λ0(p)ϕx(tf (p), ξ(p)) + ν(p)DxΨ (tf (p), ξ(p)),

(5) the controls solve the minimization problem

H(t,λ0(p),λ(t,p), x(t,p),u(t,p))

= minv∈U

H(t, λ0(p),λ(t,p), x(t,p), v),

(6) the transversality condition on the terminal time,

H(tf (p),λ0(p),λ(tf (p),p), ξ(p),u(tf (p),p)

)+ λ0(p)ϕt (tf (p), ξ(p)) + ν(p)DtΨ (tf (p), ξ(p)) = 0,

holds.

Note that it is not assumed that the parametrizationE of extremals coversthe state-space injectively. It follows from standard results about differentiabledependence on parameters of solutions to ordinary differential equations that thetrajectoriesx = x(t,p) and their time-derivativesx(t,p) arer-times continuouslydifferentiable inp for fixed t and that the derivatives are continuous jointly in(t,p). These partial derivatives can be calculated as solutions to the corresponding


variational equations. Also, at the moment we do not impose any regularityproperties on the parametersλ0 andν. Consequently the multiplierλ = λ(t,p)

may not be differentiable inp. For the general theory this will not be needed.However, ifλ0 andν are alsor-times continuously differentiable, then the costateλ(t,p) has smoothness properties identical tox(t,p). We call this then anicelyCr -parametrized family of extremals.

The definition below gives the necessary modifications if parametrized familiesof broken extremals are considered.

Definition 2.3. A Cr -parametrized family of broken extremal liftsis an 8-tuple (P ;T ; ξ, ν;x,u, λ0, λ) whereT = tin = tm+1, . . . , t1, t0 = tf is a finitefamily of r-times continuously differentiable functionstj :P → R which satisfytj+1(p) < tj (p) for all p ∈ P , j = 0, . . . ,m, and the conditions of Definition 2.2are satisfied piecewise in the sense of the following modifications:

• Let Dj = (t,p): p ∈ P , tj+1(p) t tj (p), andD∗j = (t,p): p ∈ P ,

tj+1(p) < t < tj (p). The controlu(·) is given by an admissible controluj = uj (t,p) on D∗

j which is r-times continuously differentiable inp fort fixed, the partial derivatives inp are continuous onD∗

j and the functionuj

itself has a continuous extension toDj .• The conditions definingx andλ stay in effect (i.e.,x(·,p) andλ(·,p) satisfy

the system and adjoint equations of the Maximum Principle withu(·,p)).

Although these definitions are formulated in general, we very much think ofP

as a sufficiently small neighborhood of some parameterp0 corresponding to a ref-erence extremal lift(x(·,p0), u(·,p0), λ0(p0), λ(·,p0)). Clearly, and this is seenby looking at simple text-book examples, like, for instance, Bushaw’s problem(the problem of steering the double-integrator to the origin inR

2 time-optimally),the smoothness properties required inp in these definitions will seldom be sat-isfied globally. Locally, however, like in Bushaw’s problem, such a structure istypically valid for all but few exceptional trajectories (two abnormal extremals inBushaw’s problem) and simply is a consequence of smooth dependence of solu-tions of ODE’s on parameters and initial conditions.

In order to make this point more explicitly, we briefly describe two types ofoptimal control problems and outline howCr -parametrized families of brokenextremal lifts naturally arise in this context. Consider the problemΣr , r = 1,2,to minimize an objective of the form

T∫0

(L(t, x) + 1

rur

)dt + φ(x(T )) (2.8)

over all Lebesgue-measurable functionsu : [0, T ] → [0,1] subject to

x = f (t, x) + ug(t, x), x(0) = x0, (2.9)


with fixed terminal timeT . In the dynamicsf andg are time-varying vector fieldswhich satisfy so-calledCr+1-Caratheodory conditions [21] which guarantee thatfor any controlu the initial value problem (2.9) has a unique solution which isCr+1 in the initial data. Problems of this type are quite common as mathematicalmodels in biomedical problems like the chemotherapy of cancer [23,24] and HIV[25] when the number of cancer cells or infectedCD4+T cells needs to beminimized over a prescribed fixed therapy horizon.

It is easy to see that these problems are normal and thus we can normalizeλ0 ≡ 1 for all extremals. For the quadratic objective the HamiltonianH is strictlyconvex inu and overR has a unique minimum atu = −λg(t, x). Therefore,depending on whether this value lies in the interval[0,1] or not, the minimizingcontrol over[0,1] is given by

u =0 if λg(t, x) 0,

−λg(t, x) if 0 λg(t, x) −1,1 if λg(t, x) −1.

(2.10)

A nicely Cr -parametrized family of broken extremal lifts is constructed byintegrating the system and adjoint equations backward from the terminal timeT with terminal conditions

x(T ,p) = p, λ(T ,p) = ∂φ

∂x(p); (2.11)

i.e., the terminal value of the trajectory is taken as parameter. It follows from stan-dard results on smooth dependence of solutions to ODE on initial data [21] thatthe solutionsx(t,p) andλ(t,p) of this system with either one ofu(t,p) ≡ 1,u(t,p) = −λg(t, x(t,p)) or u(t,p) ≡ 0 depend smoothly on the terminal condi-tion and thus initially generate a nicelyCr -parametrized family of extremal liftsas defined above. Once switchings between these candidates arise, these smooth-ness properties will be inherited by the new flow except in degenerate cases whenit is not possible to solve the equations

Φ0(t,p) = λ(t,p)g(t, x(t,p)) ≡ 0 (2.12)

or

Φ1(t,p) = λ(t,p)g(t, x(t,p)) ≡ −1 (2.13)

smoothly fort = τ (p). By the implicit function theorem this can be done if thetime-derivativeΦi (t,p), i = 0,1, does not vanish at the switchings. These timederivatives are easily calculated as

Φi (t,p) = −Lx(t, x(t,p))g(t, x(t,p))

+ λ(t,p)[f,g](t, x(t,p)) + gt (t, x(t,p))

, (2.14)

where [f,g] denotes the Lie bracket of the time-varying vector fieldsf (t, ·)andg(t, ·). In particular, no information other than the states and costates at the


switching are required. Thus, “typically” it is possible to construct a nicelyCr -parametrized family of broken extremal lifts around a reference extremal if thecontrol only switches finitely many times between these three types of candidatesfor optimality.

Note that in this case the controls actually remain continuous as they “switch”from the interior valueu = −λg(t, x) to one of the boundary values. However,the parametrizations inp will no longer beCr across the “switching surface”. Inthis sense the natural candidates for optimality in the problemΣ2 are preciselyfamilies of broken extremals in the sense of Definition 2.3. The “switches” inthis case are seen in higher derivatives of the trajectories at the “switching times”and trajectories of this type naturally fall into the category ofbroken extremalsconsidered in our paper.

Bang–bang controls typically arise as candidates for optimality for the problemΣ1 when anL1-type objective is used. Here the controlsu = u(t,p) are de-termined by the zeroes of the switching function

Φ(t,p) = 1+ λ(t,p)g(t, x(t,p)). (2.15)

Clearly the flows for the constant controls are smooth and again a nicelyCr -parametrized family of broken extremal lifts can be constructed if the equationΦ(t,p) ≡ 0 can be solved smoothly forτ = t (p) near the switching times. Thisagain will be the case if the derivativesΦ of the switching function do not vanishat the switching times for the reference trajectory. Details of applications of thesestandard arguments for general systems are given in [18] and also in [23] wheresome of the results developed here are applied to study locally optimal controlsfor a mathematical model for cancer chemotherapy.

2.3. The Shadow-Price Lemma

Define the parametrized costC :D → R along aCr -parametrized familyE ofextremal lifts as

C(t,p) =tf (p)∫t

L(s, x(s,p),u(s,p)) ds + ϕ(tf (p), ξ(p)). (2.16)

It follows from our assumptions and the above smoothness properties thatC iscontinuously differentiable int on D∗ = (t,p): tin(p) < t < tf (p), p ∈ P and that bothC and its time-derivative(∂C/∂t)(t,p) are r-times continuouslydifferentiable inp. The following relation is crucial to the whole construction:

Lemma 2.4 (Shadow-Price Lemma).Let E be a C1-parametrized family ofextremal lifts. Then we have that

λ0(p)∂C

∂p(t,p) = λ(t,p)

∂x

∂p(t,p). (2.17)


Proof. It suffices to show that forp fixed both sides have the samet-derivativeand identical values at the terminal timetf (p). By extending the controlsu =u(t,p) continuously beyond the terminal timetf (p), without loss of generalitywe may assume that the extremals and covectors are defined on open time-intervals which containtf (p) and both sides of Eq. (2.17) are continuouslydifferentiable functions int for fixed p. We start by calculating the values atthe terminal time. For simplicity of notation we drop the arguments. All functionsand all their derivatives are evaluated at their proper arguments. We have

∂C

∂p(tf (p),p) = ϕt

∂tf

∂p+ ϕx

(f

∂tf

∂p+ ∂x

∂p

)+ L

∂tf

∂p

= (ϕt + ϕxf + L)∂tf

∂p+ ϕx

∂x

∂p.

Without loss of generality we have assumed thatϕ is defined and differentiable inthe ambient state space. By construction we haveΨ (tf (p), x(tf (p),p)) ≡ 0 andthus

Ψt

∂tf

∂p+ Ψx

(f

∂tf

∂p+ ∂x

∂p

)≡ 0.

Hence, adjoining this equation with multiplierν = ν(p) to λ0(∂C/∂p) we get atthe endpoint

λ0∂C

∂p(tf (p),p) = (λ0ϕt + νΨt + (λ0ϕx + νΨx)f + λ0L)

∂tf

∂p

+ (λ0ϕx + νΨx)∂x

∂p.

But it follows from the transversality conditions (2.7) thatλ = λ0ϕx + νΨx and0 = λ0ϕt + νΨt + H . Thus we obtain at timetf (p) that

λ0∂C

∂p= (λ0ϕt + νΨt + H)

∂tf

∂p+ λ

∂x

∂p= λ

∂x

∂p. (2.18)

It remains to show that both sides have the same time-derivatives. Using theadjoint equation and the variational equation for∂x/∂p we get

d

dt

λ(t,p)

∂x

∂p(t,p)

= λ(t,p)

∂x

∂p(t,p) + λ(t,p)

∂2x

∂t∂p(t,p)

= (−λ0Lx − λfx)∂x

∂p+ λ

(fx

∂x

∂p+ fu

∂u

∂p

)

= −λ0Lx∂x

∂p− λ0Lu

∂u

∂p+ Hu

∂u

∂p

= λ0∂2C

∂t∂p(t,p) + Hu

∂u

∂p.


Hence the proof will be completed by verifying that

Hu(t, λ0(p),λ(t,p), x(t,p),u(t,p))∂u

∂p(t,p) ≡ 0 onD. (2.19)

To see this, fix a point(t,p) in D∗. Forq sufficiently close top, the control valuev = u(t, q) is admissible and therefore it follows from the minimization propertyof the extremal controlu(t,p) that

h(ξ) = λ0(p)L(t, x(t,p),u(t, ξ)) + λ(t,p)f (t, x(t,p),u(t, ξ))

has a local minimum atξ = p. Since this function is differentiable inξ , we havethat

0= gradh(p) = Hu(t, λ0(p),λ(t,p), x(t,p),u(t,p))∂u

∂p(t,p).

Hence

d

dt

λ(t,p)

∂x

∂p(t,p)

= λ0(p)

∂2C

∂t∂p(t,p).

This proves Eq. (2.17). No smoothness is required on the multipliersλ0(p) or ν(p) for the Shadow-

Price Lemma to be valid. We now generalize the Shadow-Price Lemma to familiesof broken extremals.

Lemma 2.5. For a Cr -parametrized family of broken extremal lifts the Shadow-Price identity

λ0(p)∂C

∂p(t,p) = λ(t,p)

∂x

∂p(t,p) (2.20)

holds on each open domainD∗j , j = 0,1, . . . ,m.

Proof. Without loss of generality we only consider the casem = 1. Let

D∗1 = (t,p): tin(p) < t < t1(p), p ∈ P ,

T1 = (t,p): t = t1(p), p ∈ P ,D∗

0 = (t,p): t1(p) < t < tf (p), p ∈ P .Recall that the parametrized costC0 :D0 → R is defined by

C0(t,p) =tf (p)∫t

L(s, x(s,p),u(s,p)) ds + ϕ(tf (p), x(tf (p),p)

)


and let

C1(t,p) =t1(p)∫t

L(s, x(s,p),u(s,p)) ds + C0(t1(p), x(t1(p),p)

)denote the parametrized cost onD1. Also denote byxi the restriction ofx toDi , i = 0,1. By Lemma 2.4 the result holds onD∗

0. We therefore need to extendthe Shadow-Price identity beyond the switching surface toD∗

1. We know thatC1 agrees withC0 on T1 and it follows as in the proof of Lemma 2.4 that for(t,p) ∈ D∗

1,

λ0(p)∂2C1

∂t∂p(t,p) = d

dt

λ(t,p)

∂x1

∂p(t,p)

. (2.21)

However, we need to analyze the behavior of the partial derivatives on the switch-ing surfaceT1. Sinceu0 andu1 extend continuously toT1, we can extendx0, x1and the restriction ofλ toD0 andD1 as solutions of the corresponding differentialequation (not necessarily as extremals) onto a neighborhood ofT1. The followingelementary lemma relates the partial derivatives on the switching surface.

Lemma 2.6. Let z0 ∈ Rn and let Z be an open neighborhood ofz0. Suppose

g :Z → R andh :Z → R are continuously differentiable functions which satisfyh(z) = 0 on z ∈ Z: g(z) = 0. If g(z0) = 0 and∇g(z0) = 0, then there exist aneighborhoodW of z0 contained inZ and a continuous functionk :W → R sothath(z) = k(z)g(z) for z ∈ W and∇h(z0) = k(z0)∇g(z0).

Since we have for(t,p) ∈ T1 = (t,p): t − t1(p) = 0, p ∈ P thatC0(t,p)

= C1(t,p), andx0(t,p) = x1(t,p) = x(t,p), it follows that there exists a contin-uous vector-valued functionk = (k0, k1, . . . , kn) defined near a reference parame-terp0 such that fort = t1(p) we have

gradC1(t,p) = gradC0(t,p) + k0(p)

(1,−∂t1

∂p(p)

), (2.22)

and for each componentx(i) of x0 andx1

gradx(i)1 (t,p) = gradx(i)

0 (t,p) + ki(p)

(1,−∂t1

∂p(p)

). (2.23)

But from the definition of the parametrized costs we have

∂Ci

∂t(t,p) = −L(t, xi(t,p),ui(t,p)), i = 0,1, (2.24)

and thus

k0(p) = L(t1(p), x0(t1(p),p),u0(t1(p),p)

)− L

(t1(p), x1(t1(p),p),u1(t1(p),p)

).


Similarly, if we setk(p) = (k1(p), . . . , kn(p)), then

k(p) = x1(t1(p),p) − x0(t1(p),p)

= f(t1(p), x1(t1(p),p),u1(t1(p),p)

)− f

(t1(p), x0(t1(p),p),u0(t1(p),p)

).

It follows from the minimum condition (2.6) that the function

t → H(t,λ0(p),λ(t,p), x(t,p),u(t,p))

is continuous att = t1(p), i.e., has the same value for bothu0(t1(p),p) andu1(t1(p),p). Therefore

λ0(p)k0(p) − λ(t1(p),p)k(p)

= λ0L(. . . , u0(t1(p),p)

) + λf(. . . , u0(t1(p),p)

)−λ0L

(. . . , u1(t1(p),p)

) − λf(. . . , u1(t1(p),p)

)= H

(t1(p), . . . , u0(t1(p),p)

) − H(t1(p), . . . , u1(t1(p),p)

) ≡ 0,

where some arguments have been suppressed to simplify notation. Hence, from(2.22) and using the Shadow-Price Lemma forC0, we have that

λ0(p)∂C1

∂p(t1(p),p) = λ0(p)

∂C0

∂p(t1(p),p) − λ0(p)k0(p)

∂t1

∂p(p)

= λ(t1(p),p)

(∂x0

∂p(t1(p),p) − k(p)

∂t1

∂p(p)

)

= λ(t1(p),p)∂x1

∂p(t1(p),p).

Thus

λ0(p)∂C1

∂p(t1(p),p) = λ(t,p)

∂x1

∂p(t1(p),p) for all (t,p) ∈ D∗

1. (2.25)

Although the partial derivatives ofC andx with respect to the parameters are

not continuous at the switching surface, their jumps cancel in the Shadow-PriceLemma. It is this identity which generates solutions to the Hamilton–Jacobi–Bellman equation.

2.4. C1-solutions to the Hamilton–Jacobi–Bellman equation

We formulate results about the differentiability of the value function for aparametrized family of extremal lifts. We first define feedback controls. Since weneed to deal with discontinuous feedbacks, rather than getting into the subtletiesof when solutions to ODE’s with discontinuous right-hand sides exist, we simply


postulate this as definition. We call a functionu∗ :R → U anadmissible feedbackcontrol if for each (τ, ξ) ∈ R the differential equationx = f (t, x,u∗(t, x)),x(τ) = ξ , has a unique solutionx∗(t) = x∗(t; τ, ξ) forward in time(t τ ) and theassociated open-loop controlu(t) = u∗(t, x∗(t)) lies inU . In this case the open-loop and the closed-loop control give rise to the same controlled trajectory from(τ, ξ). The feedback is called optimal onR if each of these open-loop controls isoptimal for the problem with initial conditions(τ, ξ) ∈ R. In this case the value-function is given byV (τ, ξ) = J (u∗; τ, ξ).

Theorem 2.7. LetE be aCr -parametrized family of normal extremal lifts,r 1,and denote byσ the flow of the trajectories

σ :D∗ → R × Rn, (t,p) → σ(t,p) = (t, x(t,p)). (2.26)

Suppose the restriction ofσ to some open setO is aC1,r -diffeomorphism onto anopen subsetR ⊂ R × R

n of the state-space, i.e., is injective with a nonvanishingJacobian determinant onO , is continuously differentiable int and r-timescontinuously differentiable inp. Then the function

V :R → R, V = C σ−1, (2.27)

is continuously differentiable int and r-times continuously differentiable inxonR. The function

u∗ :R → R, u∗ = u σ−1, (2.28)

is an admissible feedback control which is continuous int and r-times continu-ously differentiable inx. Together the pair(V ,u∗) solves the Hamilton–Jacobi–Bellman equation

Vt(t, x) + minu∈U

Vx(t, x)f (t, x,u) + L(t, x,u) ≡ 0 (2.29)

onR. Furthermore, the following identities hold in the parameter space onO :

Vt(t, x(t,p)) = −H(t,λ(t,p), x(t,p),u(t,p)), (2.30)

Vx(t, x(t,p)) = λ(t,p). (2.31)

If E is a nicelyCr -parametrized family of extremal lifts, thenV is (r + 1)-timescontinuously differentiable inx onR and we also have

Vxx(t, x(t,p)) = ∂λT

∂p(t,p)

(∂x

∂p(t,p)

)−1

. (2.32)

Proof. For t fixed, the mapσ(t, ·) :p → σ(t,p) is a Cr -diffeomorphism andits inverseσ−1(t, ·) :x → σ−1(t, x) is r-times continuously differentiable inx.ThereforeV andu∗ are well-defined andr-times continuously differentiable inx.


The smoothness properties int carry over from the parametrization. Also, thesolution to

x = f (t, x,u∗(t, x)), x(tin) = x0 = x(tin,p0) (2.33)

is given by x(·,p0) and thusu∗(·, x(·,p0)) = u(·,p0) ∈ U . Henceu∗ is anadmissible feedback control. SinceC = V σ , we have that

∂C

∂p(t,p) = Vx(t, x(t,p))

∂x

∂p(t,p)

and thus, in view of Lemma 2.4 and the fact that∂x/∂p is nonsingular, Eq. (2.31)follows. Furthermore,

−L(t, x(t,p),u(t,p)) = ∂C

∂t(t,p)

= Vt(t, x(t,p)) + Vx(t, x(t,p))x(t,p)

= Vt(t, x(t,p)) + λ(t,p)f (t, x(t,p),u(t,p))

which gives (2.30). But then the minimum condition in the definition of extremalsimplies that the pair(V ,u∗) solves the Hamilton–Jacobi–Bellman equation: wehave for(t, x) = (t, x(t,p)) ∈ R and arbitraryv ∈ U that

Vt(t, x) + Vx(t, x)f (t, x, v) + L(t, x, v)

= Vt(t, x(t,p)) + Vx(t, x(t,p))f (t, x(t,p), v) + L(t, x(t,p), v)

Vt (t, x(t,p)) + Vx(t, x(t,p))f (t, x(t,p),u(t,p))

+L(t, x(t,p),u(t,p))

≡ 0.

If E is nicelyCr -parametrized, thenλ is Cr in p and thus, sinceVx = λ σ−1 onR, it follows thatVx is still r-times continuously differentiable inx. In particular,and observing that we need to take a transpose inλ to keep the notation consistent,we get

Vxx(t, x(t,p))∂x

∂p(t,p) = ∂λT

∂p(t,p)

which implies Eq. (2.32). Note that, although it is not required that the parametrization of the trajectories

is injective on the terminal manifoldN , the functionV always has a continuousextension toN (in the sense that the extended function is continuous on the do-mainR = R ∪ N ). The reason is thatC(tf (p),p) = ϕ(tf (p), ξ(p)), i.e., the costC(t,p) defined in (2.16) for pointst = tf (p) only depends on the terminal pointσ(tf (p),p) = (tf (p), ξ(p)) but not on the parameterp. Hence for(t, x) ∈ N wecan extend the definition ofV = C σ−1 by taking any of the pre-images ofσ


which are mapped toTf = (t,p): t = tf (p) by σ−1. If the mapσ extends toa diffeomorphism into a neighborhood of the sectionTf (the terminal manifoldN is then necessarily of codimension 1), the functionV extends with the samesmoothness properties as in Theorem 2.7 to a neighborhood ofN .

We now show that the value functionV corresponding to a parametrized familyE of broken extremal lifts remains continuously differentiable at a switchingsurface provided certain regularity conditions are met. We first describe the localsituation.

Definition 2.8. Let W denote a sufficiently small neighborhood of a point(t0,p0)

which lies on the parametrizationT = (t,p): t = τ (p),p ∈ P of a switchingsurface. LetD∗

0 = (t,p) ∈ W : t > τ(p), D∗1 = (t,p) ∈ W : t < τ(p), and

denote the restrictions of the corresponding trajectories by subscripts. Also letR0 = σ(D∗

0), S = σ(T ), R1 = σ(D∗1), and setR = R0 ∪ S ∪ R1. We say aCr -

parametrized family of broken extremal lifts has aregular switchingat (t0, x0) =(t0, x(t0,p0)) if the controlsu1 andu2 extend asC0,r functions ontoW and if thecorresponding extensionsσ0 andσ1 areC1,r -diffeomorphisms onW .

Note that this definition only requires thatσ0 andσ1 are injective separately.This does not yet imply that the associated flowσ of trajectories,

σ : (t,p) → (t, x(t,p)) =(t, x0(t,p)) if (t,p) ∈ D∗

0,

(t, x1(t,p)) if (t,p) ∈ D∗1 ∪ T ,

(2.34)

is injective.

Definition 2.9. We say aCr -parametrized familyE of broken extremal lifts has aregular crossingat (t0, x0) = (t0, x(t0,p0)) if it has a regular switching at(t0, x0)

and if the associated flowσ of trajectories is injective on a sufficiently smallneighborhoodW of (t0,p0).

If (t0, x0) = (t0, x(t0,p0)) is a regular switching, then it follows that theswitching surfaceS = σ(T ) is an imbedded submanifold of codimension 1. Theassociated flowσ will be injective, for instance, if the corresponding flowsσ0 andσ1 are transversal toS and the corresponding dynamics point to the same sidesof the tangent space toS at (t0, x0). These relations will be developed further inSection 3.

Definition 2.10. We sayE defines a classicalCr -field of extremals overD∗ =(t,p) ∈ D: tin(p) < t < tf (p), p ∈ P if the mapσ :D∗ → R × R

n, (t,p) →(t, x(t,p)), is a C1,r -diffeomorphism, i.e., is injective with a nonvanishingJacobian determinant and is r-times continuously differentiable inp.

Definition 2.11. We sayE defines aCr -parametrizedfield of broken extremalsover D∗

s if it is a Cr -parametrized family of broken extremal lifts for which


the mapσ is injective and the restrictionσj of σ to D∗j = (t,p): p ∈ P ,

tj+1(p) < t < tj (p) is aC1,r -diffeomorphism for eachj = 0, . . . ,m.

Then we have the following result:

Theorem 2.12. Let E define aC1-parametrized field of normal broken extremalswhich has a regular crossing at(t0, x0) = (t0, x(t0,p0)). LetW be a sufficientlysmall neighborhood of(t0,p0) such that the combined flowσ is injective and theindividual flowsσ0 andσ1 are regular onW . Then the associated value functionV :R = σ(W) → R, V = C σ−1, is a continuously differentiable solution to theHamilton–Jacobi–Bellman equation at(t0, x0).

Proof. SinceE has a regular switching at(t0, x0) = (t0, x(t0,p0)) there existC0,1-extensionsu0 andu1 of the controls toW and without loss of generality wemay assume that the flow mapsσi , i = 1,2, are diffeomorphisms onW . Let C0andC1 be the corresponding extensions of the cost functions, i.e., ifC(τ(p),p)

denotes the cost of the parametrized family of extremals on the switching surface,then fori = 0,1 and all(t,p) ∈ W we have

Ci(t,p) =τ (p)∫t

L(s, xi(s,p),ui(s,p)) ds + C(τ(p),p). (2.35)

Then the functionsVi :R → R, Vi = Ci σ−1i , are well defined and continuously

differentiable onR. However, sinceVi only satisfies the maximum condition onD∗

i , the functions are solutions to the Hamilton–Jacobi–Bellman equation only onRi = σ(D∗

i ). For V1 this follows since the Shadow-Price Lemma remains validfor broken extremals. The identities

∂Vi

∂t(t, xi(t,p)) = −H(t,λ(t,p), xi(t,p),ui(t,p)), (2.36)

∂Vi

∂p(t, xi(t,p)) = λ(t,p) (2.37)

remain valid onD∗0 andD∗

1, respectively. Hence the gradients ofV0 andV1 areequal onN . (The Hamiltonian remains continuous at the switchings.) Thus,V0andV1 are continuously differentiable functions onR and both functions and theirgradients have identical values onS. Hence the composite functionV :R → R

defined by

V (t, x) =V1(t, x) for (t, x) ∈ R1 ∪ S,

V0(t, x) for (t, x) ∈ R0(2.38)

is continuously differentiable onS with ∇V = ∇V0 = ∇V1 and it solves theHamilton–Jacobi–Bellman equation sinceV0 andV1 solve it onR0 andR1, re-spectively.


Corollary 2.13. Let E define aC1-parametrized classical field or a field ofnormal broken extremals with regular crossings overD∗. SetR = σ(D∗), R =σ(D∗)∪N and letV :R → R, V = C σ−1, be the corresponding value function.ThenV is a continuously differentiable solution to the Hamilton–Jacobi–Bellmanequation onR which has a continuous extension to the terminal manifoldN .

Proof. For the case of a classical field of extremals (the flow is a diffeomorphismoff the terminal manifoldN ) nothing needs to be shown. The result followsfrom Theorem 2.7. These arguments carry over to fields of broken extremals:Theorem 2.12 implies thatV is aC1-solution to the Hamilton–Jacobi–Bellmanequation away fromN .

Corollary 2.13 gives the following result on optimality. Its proof is a classicaland elementary argument which is omitted.

Corollary 2.14 [5,22]. Let E be a Cr -parametrized field of normal brokenextremals overD∗ with regular crossings. SetR = σ(D∗), R = σ(D∗) ∪ N , andlet V :R → R, V = C σ−1, with continuous extension toR which satisfiesV (t, x) = ϕ(t, x) for all (t, x) ∈ N . Then the feedback controlu∗ :R → U ,u∗ = u σ−1, is optimal onR (i.e., with respect to any other control for whichthe corresponding trajectory lies inR up to its terminal point in N) and thecorresponding value function is given by V. In particular,V is continuouslydifferentiable onR.

3. Sufficient conditions for relative minima

In this section we derive sufficient conditions for an extremalΓ0 = (x(·,p0),

u(·,p0)) of a parametrized family of extremals to be locally optimal relative toother trajectories which stay close tox(·,p0).

Definition 3.1. We say that an extremalΓ0 defined over a compact interval[tin(p0), tf (p0)] provides a relative minimum over a setR if the restriction ofthe trajectory to(tin(p0), tf (p0)) is contained in the interior ofR and if any othertrajectory which steersx(tin(p0),p0) into the terminal manifold and lies inR doesnot give a better value for the cost.

Thus, the extremalΓ0 is optimal over all trajectories which lie inR. It is notrequired that the corresponding controls remain close as well.

We assume as given aCr -parametrized family of normal (broken) extremallifts and give sufficient conditions under which this family locally defines aCr -parametrizedfield of normal broken extremalswith regular crossingsnear a ref-erence parameterp0. These conditions relate to the regularity of the mapσ along


the segments on which the broken extremals are smooth as well as to transversal-ity conditions on the switching surfaces. We also need a local injectivity conditionat the terminal manifold. In Section 3.2 we then give sufficient conditions for therequired regularity of∂x/∂p including transversality conditions at switchings.

3.1. Local imbeddings

Let E be a Cr -parametrized family of normal broken extremals with theparametrizations of the switching times given by continuously differentiablefunctionsti , i = 0, . . . ,m + 1, tin = tm+1 < tm < · · · < t1 < t0 = tf . In order tokeep technical details at a minimum, we only consider the case when the terminalmanifold is of codimension 1. We refer the reader to [18] for the considerablymore technical general case. Henceforth we assume also that

(A) the control u = u(t,p) extends as aC0,1-function into a neighborhoodDf of the setTf = (t,p) ∈ D: t = tf (p) parametrizing the endpoints oftrajectories in the terminal manifoldN .

Under this assumption the system and adjoint equations can be defined alsofor (t,p) ∈ Df and without loss of generality we may assume the solutionsexist onDf . Note, however, that these extensions will not satisfy the minimumcondition of the Maximum Principle for timest > tf (p).

Proposition 3.2. LetE be aCr -parametrized family of normal extremal lifts withcodimension1 terminal manifoldN and suppose there exists a parameterp0 ∈ P

such that(∂x/∂p)(t,p0) is nonsingular fortin(p0) t tf (p0). Then thereexists a neighborhoodW of p0 so that the restriction ofE to (t,p): tin(p) t tf (p), p ∈ W defines aCr -parametrized field of extremals.

Proof. In this proposition we do not yet consider additional switchings offN and thereforeσ(t,p) is C1,r . By the regularity ofσ it follows from theimplicit function theorem that for each pointα ∈ [tin(p0), tf (p0)] × p0 thereexists a neighborhood,Gα , of α on whichσ is aC1,r -diffeomorphism. Withoutloss of generality we may takeGα of the form Gα = Iα × Wα , whereIα isan open interval andWα an open neighborhood ofp0. The setsGα: α ∈[tin(p0), tf (p0)]×p0 form an open cover of[tin(p0), tf (p0)]×p0 and by theHeine–Borel theorem there exists a finite subcoverGαi : αi ∈ [tin(p0), tf (p0)] ×p0, i = 1, . . . , r. LetW = ⋂r

i=1 Wαi . Then the mapσ :D → R × Rn restricted

to D∗ = (t,p): tin(p) < t < tf (p), p ∈ W is a C1,r -diffeomorphism. For, ifσ(t1,p1) = σ(t2,p2), then trivially t1 = t2 and this time lies in some intervalIαi .Without loss of generality sayt1 ∈ Iαi . But σ is a C1,r -diffeomorphism onIαi × Uαi and since bothp1 and p2 lie in W ⊆ Wαi , we must havep1 = p2as well. Thusσ is injective onD∗. This proves the theorem.


Fig. 1. Intersections of a flow of broken extremals.

We extend this construction to families of normal broken extremals. It isclear from Theorem 3.2 that the restrictions of the flow to the setsDi = (t,p):ti+1(p) t ti (p), p ∈ W will be a field of smooth extremals provided the ma-trix (∂x/∂p)(t,p0) is nonsingular on[ti+1(p0), ti(p0)] andW is chosen as a suf-ficiently small neighborhood ofp0. However, this condition for each of the subin-tervals is not sufficient to guarantee that we can imbed the reference trajectoryx(t,p0), tin(p0) t tf (p0), into a field of broken extremals, as the example inFig. 1 shows. Clearly, each of the restricted flows defines a field, but the compos-ite flow does not because of the overlap near the switching surface. This is thetypical conjugate point behavior [26] as can also be seen near fold-singularitiesfor smooth families of parametrized extremals (see [27]). This behavior needs tobe excluded by appropriate transversality conditions.

Let E be a Cr -parametrized family of normal broken extremal lifts andconsider the parametrizationT = (t,p): t = τ (p), p ∈ W of a switchingsurfaceS nearp0, S = (t, x): t = τ (p), x = x(τ(p),p), p ∈ W . Assume thereexists a continuously differentiable functionψ = ψ(t, x) so thatS = (t, x):ψ(t, x) = 0. Then the flow of the parametrized family of extremals crossesS

transversally if for all(t, x) ∈ S

ψt (t, x) + ψx(t, x)f (t, x,ui) > 0, i = 1,2, (3.1)

whereu1 andu2 denote the controls prior to and after the switching. The positivesign is chosen without loss of generality.

Definition 3.3. We say aCr -parametrized familyE of normal broken extremallifts has regular and transversal crossings atp0 if all switching surfacesSi =(t, x): t = ti(p), x = x(ti(p),p), p ∈ W for i = 1, . . . ,m are imbedded co-dimension 1 submanifolds and if the flow of extremals has regular crossings andis transversal to the switching surfacesSi at (ti, xi) = (ti(p0), x(ti(p0),p0)).

Theorem 3.4. Let E be a Cr -parametrized family of normal broken extremallifts with codimension1 terminal manifoldN and suppose there exists ap0 ∈ P


such that(i) the matrix(∂xi/∂p)(t,p0) is nonsingular onti+1(p0) t ti(p0)

for i = 0, . . . ,m, and (ii) the trajectoryx(t,p0) has regular and transversalcrossings. Then there exists a neighborhoodW of p0 such that the restrictionof E to (t,p): tin(p) t tf (p), p ∈ W defines aCr -parametrized field ofnormal broken extremals which has regular and transversal crossings.

Proof. Without loss of generality we consider extremals with only 1 switchingsurface. We can use the arguments in the proof of Theorem 3.2 to show thatthere exist neighborhoodsW0 andW1 of p0, such that the restrictions ofE to(t,p): tin(p) t t1(p), p ∈ W1 and (t,p): t1(p) t tf (p), p ∈ W0define Cr -fields of extremals. We now use the transversal crossings of theextremals to show that there exists aW ⊂ W0 ∩ W1 so that the mapσ is injectiveonD = (t,p): tin(p) t tf (p),p ∈ W and thus that the restriction ofE to D

defines aCr -field of broken extremals.Since(∂x0/∂p)(t1(p0),p0) is nonsingular by assumption, the mapσ0 : (t,p)

→ (t, x0(t,p)) (defined via theC0,r -extension ofu(t,p)) is a C1,r -diffeomor-phism near(t0,p0) with inverseσ−1

0 : (t, x) → (t,π(t, x)). Hence near(t0, x0) =(t0, x(t0,p0)) the switching surfaceS can be described asS = (t, x): ψ(t, x) =t − τ (π(t, x)) = (Id,−τ ) σ−1

0 (t, x) = 0 and∇ψ(t0, x0) = 0 since

∇ψ(t, x) = (1, −∇τ (p)

)(1 0

∂x0∂t

(τ (p),p)∂x0∂p

(τ (p),p)

)−1

= (1, −∇τ (p)

)(1 0

−(∂x0∂p

)−1 ∂x0∂t

(∂x0∂p

)−1

)

=(1+ ∇τ (p)

(∂x0∂p

(τ (p),p))−1

f (τ(p), x(p),u0(p)),

−∇τ (p)(∂x0∂p

(τ (p),p))−1

).

Note that∂ψ

∂t(t0, x0) + ∂ψ

∂x(t0, x0)f (t0, x0, u0(t0,p0)) ≡ 1. (3.2)

Thus,S is an imbeddedn-dimensional submanifold. The transversality condition(3.1) onψ implies that

dψ

dt

(t1(p0), xi(t1(p0),p0)

)> 0, i = 0,1.

It follows that there exists anε > 0 such that the above inequality will holdfor (t,p) satisfying ‖t − t1(p0)‖ < ε and ‖p − p0‖ < ε. By the continuityof t1(·) there exists an open neighborhoodW ⊂ p: ‖p − p0‖ < ε so thatt1(W ) ⊂ (t1(p0) − ε, t1(p0) + ε). We choose aδ > 0 small enough so thatW = p: ‖p −p0‖ < δ ⊂ W0 ∩W1 ∩ W . Injectivity of the flow will follow if wecan show thatS = Y0∩Y1, whereYi = σ(Di), i = 0,1, andD0 = (t,p): t1(p) t tf (p), p ∈ W , D1 = (t,p): tin(p) t t1(p), p ∈ W . It is apparent that


S ⊂ Y0 ∩ Y1. We now prove thatY0 ∩ Y1 ⊂ S, or equivalently,Sc ⊂ (Y0 ∩ Y1)c,

where the superscript “c” denotes the set complement. Let(t, x) ∈ Sc and suppose(t, x) ∈ Y0 ∩ Y1. It follows that there exist(t,p1) ∈ D0 and(t,p2) ∈ D1 such thatσ0(t,p1) = (t, x) = σ1(t,p2). Since(t, x) ∈ Sc we havet1(p1) < t < t1(p2) andthusp1 = p2. Evaluatingψ at the switching surface we get

ψ(t1(p1), x0(t1(p1),p1)

) = ψ(t1(p2), x1(t1(p2),p2)

) = 0.

By construction the functionsψ(t, xi(t,pi+1)), i = 0,1, are monotonically in-creasing on[t1(p1), t1(p2)] and thus

ψ(t, x) = ψ(t, x0(t,p1)) > ψ(t1(p1), x0(t1(p1),p1)

) = 0

= ψ(t1(p2), x1(t1(p2),p2)

)> ψ(t, x1(t,p2)) = ψ(t, x).

Contradiction. This proves that the combined flow is injective near the switchingsurface; i.e., we have a local field.Corollary 3.5. Let E be theCr -parametrized family of normal broken extremallifts with codimension1 terminal manifoldN satisfying conditions(i) and (ii)of Theorem3.4 and letW be a neighborhood ofp0 for which E is a field withregular and transversal crossings. Then settingR = σ(D∗) andR = σ(D∗)∪N ,the mappingV :R → R, V = C σ−1, is a continuously differentiable solution tothe Hamilton–Jacobi–Bellman equation onR which has a continuous extensionontoR and the extremalΓ0 = (x(·,p0), u(·,p0)) is a relative minimum.

3.2. Sufficient conditions for regular and transversal crossings

We now give criteria which allow to verify conditions (i) and (ii) of Theo-rem 3.4 (i.e., conditions for the flow to be regular and to have transversal cross-ings). We start with characterizing when the mapσ is regular. For this we simplymodify classical results on Riccati equations to our set-up (see also [27]). This re-quires that the adjoint variables are differentiable inp and thus we consider onlynicely parametrized families of extremal lifts.

Corollary 3.6. LetE be a nicelyC1-parametrized family of normal extremal liftsand suppose the mapσ :D → R

n, (t,p) → x(t,p), is a C1-diffeomorphism ofsome open subsetO ⊂ intD onto an open subsetR ⊂ R

n. Then fort such that(t,p) ∈ O , the function

S(t,p) = Vxx(t, x(t,p)) = ∂λT

∂p(t,p)

(∂x

∂p(t,p)

)−1

(3.3)

satisfies the differential equation

S + Sfx + f Tx S + Hxx + (Sfu + Hxu)

∂u

∂p

(∂x

∂p

)−1

≡ 0, (3.4)


where the partial derivatives off and H are evaluated along the extremalcorresponding to the parameterp.

Proof. The matrices∂x/∂p and ∂λT /∂p satisfy the variational equations of,respectively, the dynamics and the covariational equations. Equation (3.4) followsby a direct calculation from these differential equations:

S =[

d

dt

(∂λT

∂p

)](∂x

∂p

)−1

+(

∂λT

∂p

)[d

dt

(∂x

∂p

)−1]

=[

d

dt

(∂λT

∂p

)](∂x

∂p

)−1

−(

∂λT

∂p

)(∂x

∂p

)−1[d

dt

(∂x

∂p

)](∂x

∂p

)−1

=(

−(Lxx

∂x

∂p+ Lxu

∂u

∂p

)− f T

x

∂λT

∂p− λ

(fxx

∂x

∂p+ fxu

∂u

∂p

))(∂x

∂p

)−1

− S

(fx

∂x

∂p+ fu

∂u

∂p

)(∂x

∂p

)−1

= −Sfx − f Tx S − Hxx − (Sfu + Hxu)

(∂u

∂p

)(∂x

∂p

)−1

. We will now use Eq. (3.4) to give equivalent characterizations of the regularity

of the mapσ . For the case when the control takes values in the interior of thecontrol set, these are classical results about linear systems and Riccati equationswhich we briefly recall (see, e.g., [28]). Then we will give conditions for theregularity of broken extremals.

Suppose the controlu takes values in the interior of the control set on an inter-val I or, more generally, assumeHu vanishes identically along(λ, x,u) alongI .In this case it follows from the minimum condition that the Legendre conditionholds in the sense thatHuu is positive semidefinite along(λ, x,u). If Huu is pos-itive definite we say that thestrengthened Legendre conditionholds. Then

∂u

∂p= −H−1

uu

(Hux

∂x

∂p+ f T

u

∂λT

∂p

)(3.5)

and under the assumptions of Corollary 3.6 we can eliminate the control term fromEq. (3.4) to get the customary Riccati equation for the second derivativesVxx :

S = −Sfx − f Tx S − Hxx + (Sfu + Hxu)H

−1uu

(Hux + f T

u S).

It is well known thatthe existence of a bounded solution to this Riccati equationover an intervalI is equivalent to the regularity of the Jacobian∂x/∂p [28]. Thisfollows from the lemma below which goes back to Legendre and the calculus ofvariations.


Lemma 3.7 [29]. LetA, B, M, N , X0 andY0 be(n × n)-matrices defined on anopen intervalI containingtin and assumeX0 is nonsingular. Let(X(t), Y (t)) bea solution to the initial value problem(

X

Y

)=

(A(t) −M(t)

−N(t) −B(t)

)(X

Y

),(

X(tin)

Y (tin)

)=

(X0Y0

). (3.6)

The matrixX is nonsingular onI if and only if there exists a solutionS on I forthe Riccati equation

S + SA(t) + B(t)S − SM(t)S + N(t) ≡ 0,

S(tin) = S0 = Y0X−10 . (3.7)

From Lemma 3.7 we obtain the following characterization for the regularityof σ :

Corollary 3.8. LetE be a nicelyC1-parametrized family of normal extremal liftsand let Γp : [tin(p), tf (p)] → R

n × U be an extremal which satisfiesHu = 0and the strengthened Legendre conditionHuu > 0. Suppose(∂x/∂p)(tf (p),p)

is nonsingular. Then(∂x/∂p)(t,p) is nonsingular on[τ, tf (p)], τ tin(p), ifand only if the solutionS(t,p) to the Riccati equation(3.8),

S + Sfx + f Tx S + Hxx − (Sfu + Hxu)H

−1uu

(Hux + f T

u S) ≡ 0, (3.8)

with terminal condition

S(tf (p),p) = ∂λT

∂p(tf (p),p)

(∂x

∂p(tf (p),p)

)−1

exists on[τ, tf (p)].

If the controlu does not depend on the parameterp, then (3.4) simplifies to alinear Lyapunov differential equation,

S + Sfx + f Tx S + Hxx ≡ 0. (3.9)

But this equation always has a solution under our assumptions. Therefore weobtain the following result from Lemma 3.7:

Corollary 3.9. LetE be a nicelyC1-parametrized family of normal extremal liftsand letΓp : [tin(p), tf (p)] → R

n ×U be an extremal for which the correspondingcontrol does not depend onp; i.e., (∂u/∂p)(t,p) ≡ 0. Then(∂x/∂p)(t,p) isnonsingular on[tin(p), tf (p)] if (∂x/∂p)(tf (p),p) is nonsingular.


This situation arises in the analysis of bang–bang trajectories when conse-quently the regularity of the mapσ only depends on the behavior at the switch-ing surfaces. We now analyze the behavior of the flow around a switching sur-face. Suppose the switching surface is described in the parameter space by aCr -function τ :P → R, p → τ (p), and letT = (t,p): p ∈ P , t = τ (p). Theswitching surfaceS = (t, x): t = τ (p), x = x(τ(p),p), p ∈ P will be an n-dimensional imbedded submanifold near(t0, x0) = (τ (p0), x(τ (p0),p0)) if themapΞ :P → S, p → (τ (p), x(τ (p),p)), has a nonsingular Jacobian atp0. Alsolet D0 = (t,p): τ (p) t tf (p), p ∈ P , D1 = (t,p): tin(p) t τ (p),

p ∈ P , and denote the controls and trajectories onDi by a subscripti = 0,1. Weassumethat the controlsui = ui(t,p) extend asC0,r -functions into open neigh-borhoodsDi of Di , i = 0,1 (i.e., extend beyond the switching surfaceT ). Thusthe flowsxi(t,p) and the adjoint variablesλi(t,p) also extend asC1,r -functionsinto a neighborhood ofDi and without loss of generality we can take it asDi ,i = 0,1. Like in the proof of Lemma 2.5 the following relation follows fromLemma 2.6:

∂x1

∂p(τ(p),p) − ∂x0

∂p(τ(p),p)

= [f

(τ (p), x0(τ (p),p),u0(τ (p),p)

)− f

(τ (p), x1(τ (p),p),u1(τ (p),p)

)]∇τ (p), (3.10)

where the gradient is a row vector. Note that(∂x0/∂p)(τ (p),p) and(∂x1/∂p) ×(τ (p),p) differ by a rank 1 matrix. The following relation from linear algebra iswell known [30]:

Lemma 3.10. SupposeA ∈ Rn×n is nonsingular andv ∈ R

n is a nonzero vector.ThenB = A + uvT is nonsingular if and only if1+ vT A−1u = 0. In this case

(A + uvT

)−1 = A−1 − A−1uvT A−1

1+ vT A−1u.

To simplify the notation we setx(p) = x0(τ (p),p) = x1(τ (p),p), u0(p) =u0(τ (p),p), andu1(p) = u1(τ (p),p). Then we have

Corollary 3.11. Suppose the controlsu0 and u1 extend asC0,r -functions intoopen neighborhoods of the set(t,p): t = τ (p). If (∂x0/∂p)(t,p) is nonsingularfor t = τ (p), then(∂x1/∂p)(t,p) is nonsingular fort = τ (p) if and only if

∇τ (p)

(∂x0

∂p(τ(p),p)

)−1

× (f (τ(p), x(p),u0(p)) − f (τ(p), x(p),u1(p))

) = −1. (3.11)


Proof. The statement is trivially correct if∇τ (p) = 0 and follows by applyingLemma 3.10 to (3.10) otherwise.

Condition (3.11) is equivalent to a transversality condition on the flow ofx1 at the switching surface. For, if(∂x0/∂p)(t,p) is nonsingular att = τ (p),then the mapσ0 : (t,p) → (t, x0(t,p)) is locally a C1,r -diffeomorphism andhence invertible. As before we define the switching surfaceS = (t, x): t =τ (p), x = x(p) = x0(τ (p),p), in the (t, x)-space in a neighborhoodW of thereference point(tp, xp) = (τ (p), x(p)) asS = (t, x) ∈ W : ψ(t, x) = 0, whereψ :W → R is given byψ(t, x) = (I,−τ ) σ−1

0 (t, x) and isC1,r . Differentiatingψ gives

∂ψ

∂t(τ (p), x(p)) + ∂ψ

∂x(τ(p), x(p))f (τ (p), x(p),u1(p))

= 1+ ∇τ (p)

(∂x0

∂p(τ(p),p)

)−1

× (f (τ(p), x(p),u0(p)) − f (τ(p), x(p),u1(p))

). (3.12)

Furthermore, by construction (see also Theorem 3.4)

∂ψ

∂t(τ (p), x(p)) + ∂ψ

∂x(τ(p), x(p))f (τ (p), x(p),u0(p)) ≡ 1. (3.13)

Thus, we have

Theorem 3.12. Suppose the controlsu0 andu1 extend asC0,r -functions onto aneighborhood of(t,p): t = τ (p) and suppose(∂x0/∂p)(t,p) is nonsingular att = τ (p0). Then the switching surfaceS = (t, x): t = τ (p), x = x(τ(p),p),

p ∈ P is an n-dimensional imbedded submanifold near(t0, x0) = (τ (p0),

x(τ (p0),p0)). The combined flowσ : (t,p) → (t, x(t,p)) has regular andtransversal crossings at the switching surfaceS at (t0, x0) if and only if

1+ ∇τ (p0)

(∂x0

∂p(τ(p0),p0)

)−1

× (f (t0, x0, u0(p0)) − f (t0, x0, u1(p0))

)> 0. (3.14)

This theorem characterizes transversal crossings in the parameter space. Typi-cally this is how the data in a regular synthesis will be constructed. Furthermore,under this condition the regularity ofσ is guaranteed for the flow ofσ1 at thenew terminal manifold. This then allows us to characterize the regularity ofσ on[tin(p), τ (p)] using the earlier results with(∂x1/∂p)(τ (p),p) given by (3.10) asterminal condition.

In order to apply Theorem 3.12 we still need an effective way to calculate(3.14). Such a procedure indeed exists whenever the “switchings” occur on


surfaces which are zero sets of smooth functions in(t,p)-space. For example, inthe case of bang–bang trajectories typically the parametrizationst = τ (p) of theswitching surfaces are obtained by solving an equation of the typeΦ(t,p) = 0given by some switching function fort . This indeed allows to calculate thequantity

∇τ (p)

(∂x0

∂p(τ(p),p)

)−1

(3.15)

without calculating any partial derivatives with respect top or matrix inversions.However, the specifics depend on the special form of the dynamics. Here weonly develop the equations for the optimal control problemΣ1 considered inSection 2.2. Recall that the switching function is given by

Φ(t,p) = 1+ λ(t,p)g(t, x(t,p)) (3.16)

and the control in the parametrized family of extremals satisfies

u(t,p) =

1 if Φ(t,p) < 0,0 if Φ(t,p) > 0.

(3.17)

Thust = τ (p) is the local solution of the equationΦ(t,p) = 0 near a switching(t0,p0) of the reference trajectory. Such a solution exists by the implicit functiontheorem if the time-derivativeΦ(t0,p0) does not vanish (see (2.14)). It followsby implicit differentiation that

∇τ (p0) = − (∂Φ/∂p)(t0,p0)

Φ(t0,p0). (3.18)

Using S(t,p) = λ(t,p)((∂x/∂p)(t,p))−1, which in the case of bang–bangtrajectories is easily computed as solution to the linear equation (3.9), we have that

∂Φ

∂p(τ(p),p) = λ(t,p)Dxg(t, x(t,p))

∂x

∂p(t,p) + gT (t, x(t,p))

∂λT

∂p(t,p)

= λ(t,p)Dxg(t, x(t,p)) + gT (t, x(t,p))S(t,p)

∂x

∂p(t,p).

(3.19)

Settingx0 = x(t0,p0) we therefore obtain

∇τ (p0)

(∂x0

∂p(t0,p0)

)−1

= − 1

Φ(t0,p0)

(λ(t0,p0)Dxg(t0, x0) + gT (t0, x0)S(t0,p0)

).

Further simplifications in (3.14) can be made because of the special form of thedynamics. Let∆u = u0(t0,p0) − u1(t0,p0) denote the jump in the control, at theswitching surface. It follows from the minimization property of the controls that

∆u = −sgnΦ(t0,p0) (3.20)


and thus we have

1+ ∇τ (p0)

(∂x0

∂p(t0,p0)

)−1(f (t0, x0, u0(p0)) − f (t0, x0, u1(p0))

)= 1+ 1

|Φ(t0,p0)|(λ(t0,p0)Dxg(t0, x0) + gT (t0, x0)S(t0,p0)

)g(t0, x0).

(3.21)

Hence we have the following result:

Theorem 3.13. Consider the optimal control problemΣ1 and suppose thereference controlu(·,p0) has a bang–bang switch att0 with Φ(t0,p0) = 0.Then the switching surfaceS is an n-dimensional imbedded submanifold near(t0, x0), x0 = x(t0,p0), and there exists a continuously differentiable functionτ defined in some neighborhoodW of p0 such thatS = (t, x): t = τ (p),

x = x(τ(p),p), p ∈ W . Assuming that(∂x0/∂p)(t0,p0) is nonsingular, thecombined flowσ : (t,p) → (t, x(t,p)) has a regular and transversal crossing atS at (t0, x0) if and only if

|Φ(t0,p0)| +λ(t0,p0)Dxg(t0, x0) + gT (t0, xp0)S(t0,p0)

g(t0, x0) > 0.

(3.22)

Thus all necessary calculations are subsumed in the computation ofS. Basedon the formulas given in this paper, it is not difficult to develop an algorithmicscheme which verifies whether a given trajectory has transversal crossings bypropagating the solutionS to (3.4) between the switching surfaces. The preciseformulas, however, depend on the model and we only refer the reader to [23,24]where this algorithm has been developed for bang–bang controls in a problemof cancer chemotherapy. If some of the controls are not constant, it is possiblethat the solutionS need not exist over the full interval and this correspondsto conjugate points and singularities in the value-function. In the family ofparametrized extremal lifts this shows in singularities in the parametrizations andwe refer the reader to [27] where fold and cusp-singularities have been analyzedfrom this aspect.

4. Conclusion

Our results provide an effective method to determine the local optimality ofa reference trajectory where the control is allowed to have discontinuities. It isnot claimed that our result can be used to analyze all possible scenarios. Clearlythis is not the case. In some sense we developed sufficient conditions for a localminimum in the so-called nondegenerate case. In our results we were always as-suming themost regular structure possible. Trajectories were assumed to cross


switching surfaces transversally. Tangential crossings were not allowed and in-deed may lead to much more complicated behaviors. Similarly, switching surfaceswere submanifolds parametrized locally as functionsτ = τ (p), not just arbitraryn-dimensional imbedded submanifolds of the(t, x)-space. There is no reason whyone should be able to parametrize the switching times inp, but it certainly is themost regular behavior. For instance, in [23] and [24] mathematical models foroptimal control of cancer chemotherapy are considered described by bilinear sys-tems. In these models singular controls are not optimal and the local optimality ofbang–bang trajectories can be established using our framework. More degeneratephenomena require extra and increasingly more complex and difficult analysis. Itwould be interesting to pursue some of these more degenerate cases to understandthe limitations of the method of characteristics as it is developed here.

References

[1] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze, E.F. Mishchenko, The MathematicalTheory of Optimal Processes, MacMillan, 1964.

[2] M.G. Crandall, P.L. Lions, Viscosity solutions of Hamilton–Jacobi equations, Trans. Amer. Math.Soc. 277 (1983) 1–42.

[3] W.H. Fleming, H.M. Soner, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993.

[4] F. John, Partial Differential Equations, Springer-Verlag, 1982.[5] L. Berkovitz, Optimal Control Theory, Springer-Verlag, New York, 1974.[6] V.G. Boltyansky, Sufficient conditions for optimality and the justification of the dynamic

programming method, SIAM J. Control 4 (1966) 326–361.[7] P. Brunovsky, Existence of regular synthesis for general control problems, J. Differential

Equations 38 (1980).[8] B. Piccoli, H.J. Sussmann, Regular synthesis and sufficiency conditions for optimality, SIAM J.

Control Optim. 39 (2001) 359–410.[9] L.C. Young, Lectures on the Calculus of Variations and Optimal Control Theory, W.B. Saunders,

Philadelphia, 1969.[10] A. Nowakowski, Field theories in the modern calculus of variations, Trans. Amer. Math. Soc. 309

(1988) 725–752.[11] A. Bressan, A high-order test for optimality of bang–bang controls, SIAM J. Control Optim. 23

(1985) 38–48.[12] H. Schättler, On the local structure of time-optimal bang–bang trajectories, SIAM J. Control

Optim. 26 (1988) 186–204.[13] A.V. Sarychev, First- and second-order sufficient optimality conditions for bang–bang controls,

SIAM J. Control Optim. 35 (1997) 315–440.[14] A. Agrachev, R. Gamkrelidze, Symplectic geometry for optimal control, in: H. Sussmann (Ed.),

Nonlinear Controllability and Optimal Control, Marcel Dekker, 1990, pp. 263–277.[15] A. Agrachev, R. Gamkrelidze, Symplectic methods for optimization and control, in: B. Jakub-

czyk, W. Respondek (Eds.), Geometry of Feedback and Optimal Control, Marcel Dekker, 1998,pp. 19–78.

[16] A. Agrachev, G. Stefani, P.L. Zezza, A Hamiltonian approach to strong minima in optimalcontrol, in: Proceedings of Symposia in Pure Mathematics, Vol. 64, American MathematicalSociety, 1998, pp. 11–22.


[17] A. Agrachev, G. Stefani, P.L. Zezza, Strong optimality for a bang–bang trajectory, SIAM J.Control Optim., accepted.

[18] J. Noble, Parametrized Families of Broken Extremals and Sufficient Conditions for RelativeMinima, D.Sc. Dissertation, Washington University, St. Louis, MO (1999).

[19] H.W. Knobloch, A. Isidori, D. Flockerzi, Topics in Control Theory, Birkhäuser, 1993.[20] H. Schättler, Hinreichende Bedingungen für ein starkes relatives Minimum bei Kontroll Proble-

men, Diplomarbeit, Bayerische Julius-Maximilians-Universität Würzburg, Germany (1982).[21] E.J. McShane, Integration, Princeton University Press, 1944.[22] W.H. Fleming, R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New

York, 1975.[23] U. Ledzewicz, H. Schättler, Optimal bang–bang controls for a 2-compartment model in cancer

chemotherapy, J. Optim. Theory Appl., accepted.[24] U. Ledzewicz, H. Schättler, Analysis of a cell-cycle specific model for cancer chemotherapy,

J. Biol. Syst., accepted.[25] D. Kirschner, S. Lenhart, S. Serbin, Optimal control of chemotherapy of HIV, J. Math. Biol. 35

(1997) 775–792.[26] J. Noble, H. Schättler, On the optimality of parametrized families of bang–bang trajectories,

in: Proceedings of the Third World Congress of Nonlinear Analysts, Catania, Italy, NonlinearAnal. 47 (2000) 351–362.

[27] M.D. Kiefer, H. Schättler, Parametrized families of extremals and singularities in solutions to theHamilton–Jacobi–Bellman equation, SIAM J. Control Optim. 37 (1999) 1346–1371.

[28] A.E. Bryson Jr., Y.C. Ho, Applied Optimal Control, Hemisphere Publishing, Washington, DC,1975.

[29] B.D.O. Anderson, J. Moore, Linear Optimal Control, Prentice–Hall, 1971.[30] T. Kailath, Linear Systems, Prentice–Hall, 1980.

sufficient conditions for relative minima of broken extremals in optimal control theory

Documents