THE SOLUTION TO THE CONJECTURE ON PROPERNESS OFWEAKLY RELAXED DELAYED CONTROLS∗

JAVIER F. ROSENBLUETH†

SIAM J. CONTROL OPTIM. c© 2000 Society for Industrial and Applied MathematicsVol. 39, No. 2, pp. 352–358

Abstract. In 1986 Warga proposed a “weak” relaxation procedure applicable to fully nonlinearproblems with delays in the control variables and showed that the resulting relaxed problem hasa solution. However, in the event of commensurate delays, several examples were found for whichweakly relaxed controls cannot be approximated with original controls, so that this extension fails tobe “proper.” Although the case of commensurate delays was solved by the introduction of a “strong”model, the question of how to properly relax noncommensurately delayed controls has remainedopen, and a natural candidate has been precisely that of weakly relaxed controls. In this paper, ageneral counterexample is constructed which rules out the weak relaxation as a proper relaxationwhen there are two or more delays. It is hoped that this result will provide some insight into theproblem of finding a general representation of properly relaxed controls.

Key words. optimal control problems, systems with time delays, proper relaxation procedures

AMS subject classifications. 49A10, 49A50, 49D20

PII. S0363012998333001

1. Introduction. This paper concerns the problem of finding a proper relax-ation procedure for optimal control problems with nonadditively coupled delays (or,more generally, shifts) in the control variables. Usually in relaxation theory the aimis to find a relaxation procedure for a given original problem which leads us to itsproper extension. This means that the set of controls for the original problem is densein the space of controls for the relaxed problem. Such procedures are well studiedfor delay free problems. For problems with delays the situation is more difficult andless understood. For example, there are natural relaxation procedures which give aproper extension while there are other natural relaxation procedures which do not(see [2, 8]).

Research in this area starts in [11]. There are two basic problems:

1. the existence of a proper relaxation, and

2. the representation of the set of relaxed controls when it exists.

Problem 1 has been affirmatively answered in [11] when there is only one constantdelay, in [8] when the constant delays are commensurate, in [13] for arbitrary constantdelays, and in [12] for certain variable delays.

This paper deals with problem 2. Specifically, a general counterexample is con-structed which rules out the weak relaxation proposed in [11] as a proper relaxationwhen there are two or more delays. This is a significant step forward in this directionwhich finally clarifies questions raised in [1, 2, 3, 4, 5, 6, 7, 8, 11]. In order to under-stand this contribution, let us briefly state the problem we shall be concerned withtogether with some basic notation and previous results.

For T ⊂ R compact and R a compact metric space, denote by M(T,R) thespace of measurable functions mapping T to rpm(R) with the weak star topology ofL1(T,C(R))∗, where rpm(R) is the space of Radon probability measures on R with

∗Received by the editors January 26, 1998; accepted for publication (in revised form) March 14,2000; published electronically August 31, 2000.

http://www.siam.org/journals/sicon/39-2/33300.html†IIMAS–UNAM, Apartado Postal 20-726, Mexico DF 01000, Mexico (jfrl@leibniz.iimas.unam.

mx).

352

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

WEAKLY RELAXED DELAYED CONTROLS 353

the weak star topology of C(R)∗. Let U(T,R) be the space of measurable functionsmapping T to R, embedded in M(T,R) by identifying each u ∈ U(T,R) with thefunction t → δu(t), where δa (also written as δa) is the Dirac measure at a.

It is well known that M(T,R) coincides with cl U(T,R), the weak star closure ofU(T,R). For optimal control problems where U(T,R) is the space of ordinary controls,under the usual assumptions on the data of the problem, existence of minimizers in thespaceM(T,R) of relaxed controls can thus be assured, and they can be approximatedwith ordinary controls. This fact is summarized by saying that M(T,R) provides a“proper” relaxation procedure for U(T,R). For a full account of these ideas, togetherwith a thorough study of relaxation and its importance in optimal control theory, werefer to Warga’s book [10].

For optimal control problems involving delays in the controls, several attemptshave been made to find proper relaxation procedures. The space of ordinary delayedcontrols we shall consider (also studied in [1, 2, 3, 4, 5, 6, 7, 8]), which illustrates themain difficulties encountered when addressing relaxation questions, is given by

U(θ1, . . . , θk) = (u0, u1, . . . , uk) ∈ U(T,Ωk+1) | ui(t) = ui−1(t−∆i)

almost everywhere (a.e.) in Ti (i = 1, . . . , k),where T = [0, 1], 0 < θ1 < · · · < θk < 1 are given real numbers, Ω is a given compactmetric space, and θ0 = 0, ∆i = θi − θi−1, Ti = [∆i, 1] (i = 1, . . . , k).

Warga [11] proposed (in a more general setting) a natural extension of U(θ1, . . . , θk),which we call the weak relaxation procedure, given by

Mw(θ1, . . . , θk) = µ ∈ M(T,Ωk+1) | Piµ(t) = Pi−1µ(t−∆i) a.e. in Ti (i = 1, . . . , k),where if, say, µ ∈ M(T,Ωn) for some n ∈ N and S ⊂ 0, 1, . . . , n − 1, then PSµ(t)denotes the projection onto the S coordinates of µ(t). Equivalently, µ ∈ M(T,Ωk+1)is a weakly relaxed control if and only if∫

Ti

dt

∫ϕ(t, ri)µ(t)(dr) =

∫Ti

dt

∫ϕ(t, ri−1)µ(t−∆i)(dr)

for all ϕ in L1(T,C(Ω)) and i = 1, . . . , k, where r = (r0, . . . , rk).One readily verifies that the set of weakly relaxed controls contains the set of

ordinary controls and, regarding it as a subspace of M(T,Ωk+1) with the weak startopology, it is compact. In [11] the question of properness of this model (that is, ifthe equality Mw(θ1, . . . , θk) = cl U(θ1, . . . , θk) holds) was posed but could not beproved. For the one delay case, it is shown in [3, 8] that this model is indeed a properrelaxation procedure.

In [8] Rosenblueth and Vinter introduced an abstract relaxation procedure, whichwe call the D-model, and properness of this procedure was established by Warga andZhu [13]. However, as we point out in [9], determining the set of D-relaxed controlsfor specific problems is a very difficult and perhaps even a hopeless task, so there isa need to find more concrete characterizations of the closure of the space of ordinarydelayed controls.

Now, the conjecture mentioned in our title, considered in [1, 2, 3, 4, 5, 6, 7, 8, 11],is that

for any Ω ⊂ Rm compact and 0 < θ1 < θ2 < 1, Mw(θ1, θ2) = cl U(θ1, θ2).This statement is false. In [8], Rosenblueth and Vinter exhibit an element of

Mw(θ1, θ2) lying outside cl U(θ1, θ2) for the case when θ2 = 2θ1 and Ω = [0, 1].

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

354 JAVIER F. ROSENBLUETH

Rosenblueth [4] extends the inequality Mw(θ1, θ2) = cl U(θ1, θ2) to pairs differentthan (θ1, 2θ1) and whose quotient is a rational number (this settles a question raisedby Andrews [1]). In the event of commensurate delays, a “strong” relaxation proce-dure was introduced in [8] and shown to be proper (see also [4, 9]). For the noncom-mensurate case, the problem of how to characterize D-relaxed controls has remainedunsolved, but a natural candidate has been precisely the space of weakly relaxed con-trols (see [1, 5, 6]). In particular, it is shown in [6] that, if Ω = 0, 1 and θ1/θ2is irrational, then any constant element of Mw(θ1, θ2) can be approximated with el-ements of U(θ1, θ2). The present paper finally solves this question. We show thatinequality also holds for noncommensurate θ1, θ2:

For any 0 < θ1 < θ2 < 1, there exists µ ∈ Mw(θ1, θ2) such that µ ∈ cl U(θ1, θ2).It is hoped that the counterexample used to solve this question will provide some in-sight into the problem of finding a general representation of properly relaxed controls.

2. The solution to the conjecture. In the following theorem we assume thatΩ = [0, 1]. Essentially the same arguments apply if Ω ⊂ R is any compact setcontaining at least two points.

Theorem 2.1. For any 0 < θ1 < θ2 < 1 there exists µ ∈ Mw(θ1, θ2) such thatµ ∈ cl U(θ1, θ2).

Proof. Let 0 < θ1 < θ2 < 1 and set α := θ2 − θ1. For all (u, v, w) ∈ Ω3 let

h(u, v, w) := min|(u, v − 1, w − 1)|, |(u− 1, v, w)|,g(u, v, w) := min|(u, v, w − 1)|, |(u− 1, v − 1, w)|,

and, for any t ∈ T , x0, x1 ∈ R, and (u, v, w) ∈ Ω3, let

f(t, x0, x1, u, v, w) :=

(x0 − t/2)2 + h(u, v, w) if t ∈ [0, θ2),

(x0 − t/2)2 + g(u, v, w) if t ∈ [θ2, 1].

Consider the problem (P) of minimizing x1(1) subject to(x0(t), x1(t)) = (u(t), f(t, x0(t), x1(t), u(t), v(t), w(t))) a.e. in T,

(x0(0), x1(0)) = (0, 0),

(u, v, w) ∈ U(θ1, θ2).Let µ ∈ M(T,Ω3) be given by

µ(t) =

12δ(0, 1, 1) +

12δ(1, 0, 0) if t ∈ [0, θ2),

12δ(0, 0, 1) +

12δ(1, 1, 0) if t ∈ [θ2, 1].

Since

Piµ(t) =1

2δ0 +

1

2δ1 for all t ∈ T and i = 0, 1, 2,

we have µ ∈ Mw(θ1, θ2). Note that its corresponding cost is zero and, since the costcannot be negative, the minimum of the problem posed on Mw(θ1, θ2) is zero.

Let 0 < a < minα, θ1, 1− θ2 so that the intervals I := [0, a), I + α, and I + θ2are disjoint and belong to [0, 1). Let (x0, x1, u, v, w) be any admissible original processfor (P), so that (u, v, w) ∈ U(T,Ω3) and

v(t) = u(t− θ1) a.e. in [θ1, 1],(2.1)

w(t) = v(t− α) a.e. in [α, 1],(2.2)

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

WEAKLY RELAXED DELAYED CONTROLS 355

and observe that the following relations hold a.e. in I:

w(t+ α) = v(t), v(t+ θ2) = u(t+ α), w(t+ θ2) = u(t).

Indeed, by (2), w(t+α) = v(t) a.e. in [0, 1−α] and, since a < θ1 < 1−θ2+θ1 = 1−α,we have I ⊂ [0, 1− α]. By (1), v(t+ θ2) = u(t+ α) a.e. in [−α, 1− θ2] ⊃ I. Finally,by (2), w(t+ θ2) = v(t+ θ1) a.e. in [−θ1, 1− θ2] and, by (1), v(t+ θ1) = u(t) a.e. in[0, 1− θ1]. Hence w(t+ θ2) = u(t) a.e. in [0, 1− θ2] ⊃ I and the three relations holda.e. in I.

Therefore,

x1(1) =

∫ θ2

0

(x0(s)− s/2)2 + h(u(t), v(t), w(t))dt

+

∫ 1

θ2

(x0(s)− s/2)2 + g(u(t), v(t), w(t))dt

≥∫ a

0

h(u(t), v(t), w(t))dt+

∫ α+a

α

h(u(t), v(t), w(t))dt

+

∫ θ2+a

θ2

g(u(t), v(t), w(t))dt

=

∫ a

0

h(u(t), v(t), w(t)) + h(u(t+ α), v(t+ α), v(t))+g(u(t+ θ2), u(t+ α), u(t))dt.

Fix t ∈ I and let r0 = u(t), r1 = v(t), r2 = u(t + α), s0 = w(t), s1 = v(t + α),and s2 = u(t+ θ2). Define

ϕ(t) := h(r0, r1, s0) + h(r2, s1, r1) + g(s2, r2, r0),

and let

m0 :=

1 if h(r0, r1, s0) = |(r0, r1 − 1, s0 − 1)|,0 if h(r0, r1, s0) = |(r0 − 1, r1, s0)|,

m1 :=

1 if h(r2, s1, r1) = |(r2, s1 − 1, r1 − 1)|,0 if h(r2, s1, r1) = |(r2 − 1, s1, r1)|,

m2 :=

1 if g(s2, r2, r0) = |(s2, r2, r0 − 1)|,0 if g(s2, r2, r0) = |(s2 − 1, r2 − 1, r0)|.

Observe now that

m0 = m1 ⇒ ϕ(t) ≥ |1− r1|+ |r1|,m1 = m2 ⇒ ϕ(t) ≥ |1− r2|+ |r2|,m0 = m2 ⇒ ϕ(t) ≥ |1− r0|+ |r0|.

On the other hand, if m0 = m1 and m1 = m2, then m0 = m2 and so, in all cases,ϕ(t) ≥ 1. It follows that

x1(1) ≥∫ a

0

ϕ(t)dt ≥ a > 0

and so the infimum of (P) posed over the original admissible processes is positive.This implies that µ cannot be approximated with elements of U(θ1, θ2).

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

356 JAVIER F. ROSENBLUETH

3. Extensions to other delay-relaxation problems. The term “weak relax-ation procedure” was first introduced in [8] referring to the model proposed by Wargain [11]. To be exact, the latter is slightly different from the one studied in [8] andmentioned in section 1. The subtle difference, which we shall explain below, leads tothe study of another delay-relaxation problem for which a proof similar to the one ofTheorem 2.1 can be applied.

Consider the following optimal control problem involving constant delays in thecontrol variables. Let T := [0, 1] and suppose we are given real numbers 0 < θ1 <· · · < θk < 1, a point ξ ∈ Rn, a compact set Ω ⊂ Rm, and functions g mapping Rn

to R and f mapping T ×Rn ×Rm(k+1) to Rn. Let T := [−θk, 1] and consider theproblem (P) of minimizing g(x(1)) subject to

x(t) = f(t, x(t), u(t), u(t− θ1), . . . , u(t− θk)) a.e. in T,x(0) = ξ,

u(t) ∈ Ω a.e. in T ,

where u is any measurable function mapping T to Rm.In [11] Warga reformulated this “original control problem” (P) by treating the

control functions as independent variables which satisfy certain compatibility condi-tions in terms of the delays. The model of relaxation proposed by Warga was obtainedby generalizing these conditions in the corresponding space of relaxed controls. To bespecific, let

W(θ1, . . . , θk)

:= (u0, u1, . . . , uk) ∈ U(T ,Ωk+1) | ui(t) = u0(t− θi) a.e. in T (i = 1, . . . , k)and consider the problem (W) of minimizing g(x(1)) subject to

x(t) = f(t, x(t), u(t)) a.e. in T,

x(0) = ξ,

u ∈ W(θ1, . . . , θk).

As Warga mentions in [11], it is a simple fact to show that (P) and (W) are equivalent.The “weak” extension of W(θ1, . . . , θk) proposed by Warga is given by

Sw(θ1, . . . , θk) := µ ∈ M(T ,Ωk+1) | Piµ(t) = P0µ(t− θi) a.e. in T (i = 1, . . . , k).In [8] Rosenblueth and Vinter considered the problem (see the notation of section

1), which we label (RV), of minimizing g(x(1)) subject tox(t) = f(t, x(t), u(t)) a.e. in T,

x(0) = ξ,

u ∈ U(θ1, . . . , θk).It should be noted that, in this reformulation of the problem, ordinary controls aremeasurable functions defined on the interval T = [0, 1] and not on T = [−θk, 1] as inthe reformulation (W) of (P) given in [11]. As before, (P) and (RV) are equivalent(see [7] for details). The notion of “weakly relaxed controls” applied to (RV) yieldsthe set

Mw(θ1, . . . , θk) = µ ∈ M(T,Ωk+1) | Piµ(t) = Pi−1µ(t−∆i) a.e. in Ti (i = 1, . . . , k)

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

WEAKLY RELAXED DELAYED CONTROLS 357

and, since the three problems are equivalent, so is the question of properness of thetwo models of relaxed controls.

Now, in [5, 6] we studied a similar but larger space of ordinary controls to whichthe notion of weakly relaxed controls can also be applied. A different class of optimalcontrol problems is derived from these spaces, and the question of properness of theweak model with respect to the larger class of ordinary controls can be posed. Considerthe space of original controls

U ′(θ1, . . . , θk)= (u0, u1, . . . , uk) ∈ U(T,Ωk+1) | ui(t) = u0(t− θi) a.e. in [θi, 1] (i = 1, . . . , k).

The notion of weakly relaxed controls applied to problem (R) of minimizing g(x(1))subject to

x(t) = f(t, x(t), u(t)) a.e. in T,

x(0) = ξ,

u ∈ U ′(θ1, . . . , θk)

corresponds to

M′w(θ1, . . . , θk) := µ ∈ M(T,Ωk+1) | Piµ(t) = P0µ(t−θi) a.e. in [θi, 1] (i = 1, . . . , k),

which we shall call the space of R-weakly relaxed controls, and the open question hasbeen, again, if the relation M′

w(θ1, . . . , θk) = cl U ′(θ1, . . . , θk) holds.Note that (R) is similar to (P) but not equivalent. The definition of U ′(θ1, . . . , θk)

as the space of ordinary delayed controls does not correspond to a reformulation of(P) and, as one can easily show,

U ′(θ1, . . . , θk)= (u0, u1, . . . , uk) ∈ U(T,Ωk+1) | ui(t) = ui−1(t−∆i) a.e. in [θi, 1] (i = 1, . . . , k).

Comparing with the definition of the set U(θ1, . . . , θk), it is clear that U(θ1, . . . , θk) ⊂U ′(θ1, . . . , θk), but the two sets may not coincide.

In [7] we proved an important consequence of this fact by exhibiting an element ofboth Mw(θ1, θ2) and M′

w(θ1, θ2) which belongs to the weak star closure of U ′(θ1, θ2)but not to that of U(θ1, θ2). Also in [7] we showed that, for certain delays, the spaceof R-weakly relaxed controls does provide a proper relaxation procedure. The resultproved in [7] states that, given 0 < θ1 < θ2 < 1 and Ω ⊂ Rm compact, if θ1 ≥ 1/2,then M′

w(θ1, θ2) = cl U ′(θ1, θ2) and, if θ1 < 1/2 and θ1 and θ2 are commensurate,then cl U ′(θ1, θ2) may be strictly contained in M′

w(θ1, θ2).A new result for this model can now be obtained with a proof similar to the one

of Theorem 2.1. The foregoing arguments can be applied to the conjecture stated interms of M′

w(θ1, θ2) and U ′(θ1, θ2). If θ1 + θ2 < 1, with a problem similar to theprevious one, it is not difficult to see that

µ(t) =

12δ(0, 1, 1) +

12δ(1, 0, 0) if t ∈ [0, θ1 + θ2),

12δ(0, 0, 1) +

12δ(1, 1, 0) if t ∈ [θ1 + θ2, 1]

belongs to M′w(θ1, θ2) but not to cl U ′(θ1, θ2). We state this result.

Theorem 3.1. For any 0 < θ1 < θ2 < 1 with θ1 + θ2 < 1 there exists µ ∈M′

w(θ1, θ2) such that µ ∈ cl U ′(θ1, θ2).

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

358 JAVIER F. ROSENBLUETH

REFERENCES

[1] T. Andrews, An Existence Theory for Optimal Control Problems with Time Delays, Ph.D.thesis, Imperial College, University of London, London, UK, 1989.

[2] J. F. Rosenblueth, Strongly and weakly relaxed controls for time delay systems, SIAM J.Control Optim., 30 (1992), pp. 856–866.

[3] J. F. Rosenblueth, Proper relaxation of optimal control problems, J. Optim. Theory Appl.,74 (1992), pp. 509–526.

[4] J. F. Rosenblueth, Approximation of strongly relaxed minimizers with ordinary delayed con-trols, Appl. Math. Optim., 32 (1995), pp. 33–46.

[5] J. F. Rosenblueth, Relaxation of delayed controls: A review, IMA J. Math. Control Inform.,12 (1995), pp. 181–206.

[6] J. F. Rosenblueth, Constant relaxed controls with noncommensurate delays, IMA J. Math.Control Inform., 13 (1996), pp. 195–209.

[7] J. F. Rosenblueth, Certain classes of properly relaxed delayed controls, IMA J. Math. ControlInform., to appear.

[8] J. F. Rosenblueth and R. B. Vinter, Relaxation procedures for time delay systems, J. Math.Anal. Appl., 162 (1991), pp. 542–563.

[9] J. F. Rosenblueth, J. Warga, and Q. J. Zhu, On the characterization of properly relaxeddelayed controls, J. Math. Anal. Appl., 209 (1997), pp. 274–290.

[10] J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, NewYork, 1972.

[11] J. Warga, Nonadditively Coupled Delayed Controls, private communication, 1986.[12] J. Warga, A proper relaxation of controls with variable shifts, J. Math. Anal. Appl., 196 (1995),

pp. 783–793.[13] J. Warga and Q. J. Zhu, A proper relaxation of shifted and delayed controls, J. Math. Anal.

Appl., 169 (1992), pp. 546–561.

Dow

nloa

ded

11/2

6/14

to 1

8.10

1.24

.154

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php