theory and applications of optimal control problems with

Theory and Applications of Optimal ControlProblems with Time Delays

Helmut Maurer

University of MunsterApplied Mathematics: Institute of Analysis and Numerics

Universite Pierre et Marie Curie, Paris, March 10, 2017

Helmut Maurer [2mm] University of Munster Applied Mathematics: Institute of Analysis and Numerics [-4mm]Theory and Applications of Optimal Control Problems with Time Delays

What can you expect from this talk ?


Challenges for Optimal Control Problems with Delays

Theory and Numerics for non-delayed optimal controlproblems with control and state constraints are well developed:

1 Necessary and sufficient conditions,

2 Stability and sensitivity analysis,

3 Numerical methods: Boundary value methods,Discretization and NLP, Semismooth Newton methods,

4 Real-time control techniques for perturbed extremals.

CHALLENGE: Establish similar theoretical and numericalmethods for delayed (retarded) optimal control problems.


Overview

1 Case Study: Combination Therapies for Cancer(with Ledzewicz, Schattler, Klamka, Swierniak)

2 Optimal Control Problems with Time Delays in State andControl Variables

3 Minimum Principle for State-Constrained ControlProblems

4 Numerical Treatment: Discretize and Optimize(with L. Gollmann)

5 A Non-Convex Academic Example with a State Constraint

6 Case Study:Two-stage Continuous Stirred Tank Reactor(CSTR)

7 Case Study: Optimal Control of a Tuberculosis Modelwith Time Delays (with C. Silva, D.F. Torres)


Combination Therapies of Cancer

Tumour Anti-Angiogenesis: J. Folkman (1972) et al.

State and control variables:

p : primary tumour volume [mm3]q : carrying capacity, or endothelial support [mm3]u : anti-angiogenic agentv : chemotoxic agent


Combination Therapies of Cancer: Literature

U. Ledzewicz and H. Schaettler: Antiangiogenic therapy incancer treatment as an optimal control problem, SIAM Journal onControl and Optimization, 46, (2007), 1052–1079. (Monotherapy)

Hahnfeldt et al model with Gompertzian Growth:

U. Ledzewicz, H. Maurer, and H. Schattler, Optimal andsuboptimal protocols for a mathematical model for tumorantiangiogenesis in combination with chemotherapy, MathematicalBiosciences 22, pp. 13–26 (2009).

Ergun et al model with Gompertzian Growth:

U. Ledzewicz, H. Maurer, and H. Schaettler, On optimaldelivery of combination therapy for tumors, Mathematical Biosciencesand Engineering, 8, (2011), 307–323.

Hahnfeldt et al model with Logistic Growth:

J. Klamka, H. Maurer and A. Swierniak: Local Controllabilityand Optimal Control for a Model of Combined Anticancer therapy withControl Delays, Math. Biosc. Eng. 14(1), 195–216 (2017).


Optimal Control Problem

p : tumor volume, q : carrying capacity,u : anti-angiogenic control, v : chemotoxic control,y : total amount of u, z : total amount of v .

Dynamics of the Hahnfeldt et al model

p(t) = G (p(t), q(t))− ϕ p(t) v(t),

q(t) = b p(t)− q(t) (d p(t)2/3 + µ+ γ u(t) + η v(t))

y(t) = u(t),

z(t) = v(t).

Initial conditions: p(0) = p0, q(0) = q0, y(0) = 0, z(0) = 0.

Growth functions

Gompertzian Growth : G (p, q) = −ξ p ln(p/q)Logistic Growth : G (p, q) = ξ p (1− p/q)


Control problem and parameters

Control problem

Minimize final tumor volume p(T )

subject to the dynamic constraints, the control constraints

0 ≤ u(t) ≤ umax, 0 ≤ v(t) ≤ vmax,

and the constraints on the total amount of drugs

y(T ) ≤ ymax, z(T ) ≤ zmax.

PARAMETERS (obtained from mice):

ξ = 0.084, b = 5.85, d = 0.00873, γ = 0.15,ϕ = 0.2, η = 0.05, µ = 0.02.

BOUNDS: umax = 75, ymax = 300, vmax = 2, zmax = 10.Helmut Maurer [2mm] University of Munster Applied Mathematics: Institute of Analysis and Numerics [-4mm]Theory and Applications of Optimal Control Problems with Time Delays

Monotherapy : only anti-angiogenic control u

Ledzewicz, Schattler (2007):Gompertzian Growth G (p, q) = −ξpln(p/q) , free terminal time T .

Compute singular control in feedback form:

u = using(p, q) = 1γ

(ξ ln(pq

)+ b p

q + 23ξ

db

qp1/3− (µ+ d p2/3)

).

0 10 20 30 40 50 60 70 80

0 1 2 3 4 5 6

time t (days)

control u

2000

5000

8000

11000

14000

0 1 2 3 4 5 6

time t (days)

tumor p and vasculature q

pq

Optimal control is bang-singular-bang. Sufficient conditions by

synthesis analysis or switching time optimization.


Approximation of bang-singular-bang control u

Gompertzian Growth and free terminal time.

u(t) =

umax for 0 ≤ t < t1uc for t1 ≤ t ≤ t20 for t2 < t ≤ T

,

t1 = 0.07386,uc = 46.08t2 = 6.463T = 6.615

0 10 20 30 40 50 60 70 80

0 1 2 3 4 5 6

time t (days)

control u

0 10 20 30 40 50 60 70 80

0 1 2 3 4 5 6

time t (days)

control u

p(T ) = 8533 p(T ) = 8541

SSC hold for the approximative control w.r.t. z = (t1, t2, uc).


Combination therapy: anti-angiogenic u, chemotherapy v

Ledzewicz, M., Schattler (2009): Gompertzian Growth, T is free.

Compute singular control in feedback form:

u = using(p, q, v) = 1γ

(ξ ln(pq

)+ b p

q + 23ξ

db

qp1/3− (µ+ d p2/3)

)+ϕ−η

γ v .

Solution for ymax = 300 and zmax = 10 (total amount of drugs):

0 10 20 30 40 50 60 70 80

0 1 2 3 4 5

time t (days)

control u

0

0.5

1

1.5

2

0 1 2 3 4 5

time t (days)

control v

0

3000

6000

9000

12000

15000

0 1 2 3 4 5

time t (days)


pq

Surprising fact: chemotherapy v always starts later (pruning).Helmut Maurer [2mm] University of Munster Applied Mathematics: Institute of Analysis and Numerics [-4mm]Theory and Applications of Optimal Control Problems with Time Delays

Combination Therapy with Logistic Growth

Klamka, Maurer, Swierniak (2015):Logistic Growth G (p, q) = ξp(1− p/q), T is free.

Controls u and v are bang-bang: there are no singular arcs !

Chemotherapy control : v(t) ≡ 2 for terminal time T = 5.

-20 0

20 40 60 80

100

0 1 2 3 4 5

time t (days)

control u and switching function φu

φu

0

4000

8000

12000

16000

0 1 2 3 4 5

time t (days)


pq

SSC hold for bang-bang controls !


Dynamics with control delays

p : tumor volume, q : carrying capacity,u : anti-angiogenic control, delay du = 10.6,v : chemotoxic control, delay dv = 1.84,y : total amount of u, z : total amount of v .

Dynamics of the Hahnfeldt et al model

p(t) = G (p(t), q(t))− ϕ p(t) v(t − dv ),

q(t) = b p(t)− q(t) (d p(t)2/3 + µ+ γ1 u(t) + γ2 u(t − du)+η v(t)) ),

y(t) = u(t), ( 0 ≤ u(t) ≤ umax , y(T ) ≤ ymax )

z(t) = v(t), ( 0 ≤ v(t) ≤ vmax , z(T ) ≤ zmax )

Initial conditions: p(0) = p0, q(0) = q0, y(0) = 0, z(0) = 0.

Objective

Minimize final tumor volume p(T )


Combination therapy with control delays

Klamka, Maurer, Swierniak (2015) : Logistic Growth, T = 16 fixed.

Solution for umax = 40, ymax = 300 and vmax = 2, zmax = 10 ,

Delays du = 10.6, dv = 1.84

-20-10

0 10 20 30 40 50

0 2 4 6 8 10 12 14 16

time t (days)

control u and switching function φu

φu

-0.5 0

0.5 1

1.5 2

2.5 3

0 2 4 6 8 10 12 14 16

time t (days)

control v and switching function φv

φv

0

5000

10000

15000

20000

0 2 4 6 8 10 12 14 16

time t (days)


pq

Necessary conditions are satisfied (extremal solution),but we cannot check sufficient conditions.


Delayed Optimal Control Problem with State Constraints

State x(t) ∈ Rn, Control u(t) ∈ Rm, Delays dx , du ≥ 0.

Dynamics and Boundary Conditions

x(t) = f (x(t), x(t − dx), u(t), u(t − du)), a.e. t ∈ [0, tf ] ,

x(t) = x0(t), t ∈ [−dx , 0] ,

u(t) = u0(t), t ∈ [−du, 0),

ψ(x(tf )) = 0q (0 ≤ q ≤ n).

Control and State Constraints

umin ≤ u(t) ≤ umax , S(x(t)) ≤ 0, ∀ t ∈ [0, tf ] (S : Rn → Rk) .

Minimize

J(u, x) = g(x(tf )) +

∫ tf

0f0(x(t), x(t − dx), u(t), u(t − du)) dt

All functions are assumed to be sufficiently smooth.Helmut Maurer [2mm] University of Munster Applied Mathematics: Institute of Analysis and Numerics [-4mm]Theory and Applications of Optimal Control Problems with Time Delays

Literature on optimal control with time-delays

Time delays in state variables and pure control constraints:

Kharatishvili (1961), Oguztoreli (1966), Banks (1968),Halanay (1968), Soliman, Ray (1970, chemical engineering),Warga (1968,1972): optimization in Banach spaces),Guinn (1976) : transform delayed problems to standard problems),Colonius, Hinrichsen (1978), Clarke, Wolenski (1991),Dadebo, Luus (1992), Mordukhovich, Wang (2003–).

Time delays in state variables and pure state constraints:

Angell, Kirsch (1990).

State and control delays and mixed control–state constraints:

Gollmann, Maurer (OCAM 2009, JIMO 2014),

Time delays in state and control variables and state constraints:

Vinter (2016) : Maximum Principle for a general problem


Method of steps : Warga (1968) and Guinn (1976)

Transform an optimal control problem with delays to a standardnon–delayed optimal control problem: requires commensurability.Then apply the necessary conditions for non-delayed problems:

Jacobson, Lele, Speyer (1975): KKT conditions in Banachspaces.

Maurer (1979) : Regularity of multipliers for state constraints.

Hartl, Sethi, Thomsen (SIAM Review 1995): Survey onMaximum Principles.

Vinter (2000): (Nonsmooth) Optimal Control.

Applications to mixed control-state constraints:

single delays: Gollmann, Kern, Maurer (OCAM 2009),multiple delays: Gollmann, Maurer (JIMO 2014)


Hamiltonian

Hamiltonian (Pontryagin) Function

H(x , y , λ, u, v) :=λ0f0(t, x , y , u, v) + λf (t, x , y , u, v)

y variable with y(t) = x(t − dx)

v variable with v(t) = u(t − du)

λ ∈ Rn, λ0 ∈ R adjoint (costate) variable

Let (u, x) ∈ L∞([0, tf ],Rm)×W1,∞([0, tf ],Rn) be a

locally optimal pair of functions. Then there exist

an adjoint function λ ∈ BV([0, tf ],Rn) and λ0 ≥ 0,

a multiplier ρ ∈ Rq (associated with terminal conditions),

a multiplier function (measure) µ ∈ BV([0, tf ],Rk),

such that the following conditions are satisfied for a.e. t ∈ [0, tf ] :Helmut Maurer [2mm] University of Munster Applied Mathematics: Institute of Analysis and Numerics [-4mm]Theory and Applications of Optimal Control Problems with Time Delays

Minimum Principle

(i) Advanced adjoint equation and transversality condition:

λ(t) =tf∫t

(Hx(s) + χ[0,tf−dx ](s)Hy (s + dx) ) ds +tf∫tSx(x(s)) dµ(s)

+ (λ0g + ρψ)x(x(tf )) ( if S(x(tf )) < 0 ),

where Hx(t) and Hy (t + dx) denote evaluations along the optimaltrajectory and χ[0,tf−dx ] is the characteristic function. (ii)

Minimum Condition for U = [ umin , umax ] :

H(t) + χ[0,tf−du ](t)H(t + du)

= min w∈U [ H(x(t), y(t), λ(t),w , v(t))

+χ[0,tf−du ](t)H(x(t + du), y(t), λ(t + du), u(t + du),w) ].

(iii) Multiplier condition and complementarity condition:

dµ(t) ≥ 0,

tf∫0

S(x(t)) dµ(t) = 0.


Regularity conditions for dµ(t) = η(t)dt if du = 0

Boundary arc : S(x(t)) = 0 for t1 ≤ t ≤ t2.

Assumption : u(t) ∈ int(U) for t1 < t < t2 .

Under certain regularity conditions we have dµ(t) = η(t) dtwith a continuous multiplier η(t) for all t1 < t < t2 .

Adjoint equation and jump conditions

λ(t) = −Hx(t)− χ[0,tf−dx ](t)Hy (t + dx)− η(t)Sx(x(t))

λ(tk+) = λ(tk−)− νkSx(x(tk)) , νk ≥ 0,

at each contact or junction time tk , νk = µ(tk+)− µ(tk−).

Minimum condition: no control constraints and delays

Hu(t) = 0 .

This condition allows to compute the multiplier η = η(x , λ).


Discretization and NLP

Consider MAYER problem with cost functional J(u, x) = g(x(tf )) .

Commensurability Assumption: There exists a stepsize h > 0 andintegers k , l ,N ∈ N with

dx = k · h, du = l · h, tf = N · h .

Grid points ti := i · h (i = 0, 1, . . . ,N), tN = tf .

Approximation at grid points:

u(ti ) ≈ ui ∈ Rm , x(ti ) ≈ xi ∈ Rn (i = 0, . . . ,N).

For simplicity EULER method:

xi+1 = xi + h · f (ti , xi , xi−k , ui , ui−l), i = 0, 1, ...,N − 1,

x−i := x0(−ih) (i = 0, .., k), u−i := u0(−ih) (i = 1, .., l).


Large-scale NLP Problem

Include mixed control-state constraint C (x(t), u(t)) ≤ 0.

Minimize

J(u, x) = g(xN)

subject to

xi+1 − xi − h · f (ti , xi , xi−k , ui , ui−l) = 0, i = 0, ..,N − 1,

ψ(xN) = 0,

C (xi , ui ) ≤ 0, i = 0, ..,N,

S(xi ) ≤ 0, i = 0, ..,N,

x−i := x0(−ih) (i = 0, .., k), u−i := u0(−ih) (i = 1, .., l).

Optimization Variable:

z := (u0, x1, u1, x2, ..., uN−1, xN) ∈ RN(m+n)


Multipliers and NLP-Solvers

Approximations:

Adjoint variable : λ(ti ) ≈ λi , multiplier for ODE,

Multiplier : η(ti ) ≈ ηi/h , multiplier for S(xi ) ≤ 0.

AMPL : Programming language (Fourer, Gay, Kernighan)

IPOPT: Interior point method (A. Wachter et al.)

LOQO: Interior point method (B. Vanderbei et al.

WORHP : SQP–method (C. Buskens, M. Gerdts)

Other NLP solvers embedded in AMPL : cf. NEOS server.

Special feature: solvers provide LAGRANGE-multipliers.

BOCOP : F. Bonnans, P. Martinon.


Academic Example: state delays

state x(t) ∈ R, control u(t) ∈ R, delay d ≥ 0

Dynamics and Boundary Conditions

x(t) = x(t − d)2 − u(t), t ∈ [0, 2],

x(t) = x0(t) = 1, t ∈ [−d , 0],

x(2) = 1

Control and State Constraints

x(t) ≥ α, i.e., S(x(t)) = −x(t) + α ≤ 0, t ∈ [0, 2]

Minimize

J(u, x) =

∫ 2

0(x(t)2 + u(t)2) dt


Optimal solutions without state constraints

min

∫ 2

0(x(t)2+u(t)2) dt s.t. x(t) = x(t − d)2−u(t), x0(t) ≡ 1, x(2) = 1

optimal state and control for

delays d = 0.0, d = 0.1, d = 0.2, d = 0.5,

0.5 0.55

0.6 0.65

0.7 0.75

0.8 0.85

0.9 0.95

1

0 0.5 1 1.5 2

state x

d=0d=0.1d=0.2d=0.5

-0.5

0

0.5

1

1.5

2

2.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u

d=0d=0.1d=0.2d=0.5


Optimal solutions with state constraint x(t) ≥ α = 0.7

Optimal state and control for

delays d = 0.0, d = 0.1, d = 0.2, d = 0.5

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 0.5 1 1.5 2

state x

d=0d=0.1d=0.2d=0.5

-0.5

0

0.5

1

1.5

2

2.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u

d=0d=0.1d=0.2d=0.5

Optimal controls u(t) are continuous !


Minimum Principle

Augmented Hamiltonian: y(t) = x(t − d)

H(x , y , λ, η, u) = u2 + x2 + λ(y2 − u) + η(−x + α)

Adjoint equation

λ(t) = −Hx(t)− χ[0,2−d ]Hy (t + d)

=

{−2x(t)− 2λ(t + d)x(t) + η(t) , 0 ≤ t ≤ 2− d

−2x(t) + η(t) , 2− d ≤ t ≤ 2

}

Minimum condition

Hu(t) = 0 ⇒ u(t) = λ(t)/2


Boundary arc x(t) = α = 0.7 for t1 ≤ t ≤ t2

x(t) ≡ α ⇒ x(t) = 0 ⇒ x(t − d)2 = u(t) = λ(t)/2

Computation of multiplier η(t) by differentiation

η(t) = 2(2x(t−d)(x(t−2d)2−λ(t−d)/2)+x(t)+λ(t+d)x(t))

delays d = 0.0, d = 0.1, d = 0.2, d = 0.5, d = 1.0

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 0.5 1 1.5 2

state x

d=0d=0.1d=0.2d=0.5

0

0.5

1

1.5

2

2.5

3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

multiplier η


Two-Stage Continuous Stirred Tank Reactor (CSTR)

Time delays are caused by transport between the two tanks.


Two Stage CSTR

Dadebo S., Luus R. Optimal control of time-delay systems bydynamic programming, Optimal Control Applications and Methods13, pp. 29–41 (1992).

A chemical reaction A⇒ B is processed in two tanks.

State and control variables:

Tank 1 : x1(t) : (scaled) concentration

x2(t) : (scaled) temperature

u1(t) : temperature control

Tank 2 : x3(t) : (scaled) concentration

x4(t) : (scaled) temperature

u2(t) : temperature control

State variables denote deviations from equilibrium.


Dynamics of the Two-Stage CSTR

Reaction term in Tank 1 : R1(x1, x2) = (x1 + 0.5) exp(

25x21+x2

)Reaction term in Tank 2 : R2(x3, x4) = (x3 + 0.25) exp

(25x41+x4

)Dynamics:

x1(t) =−0.5− x1(t)− R1(t),

x2(t) =−(x2(t) + 0.25)− u1(t)(x2(t) + 0.25) + R1(t),

x3(t) = x1(t − d)− x3(t)− R2(t) + 0.25,

x4(t) = x2(t − d)− 2x4(t)− u2(t)(x4(t) + 0.25) + R2(t)− 0.25.

Initial conditions:

x1(t) = 0.15, x2(t) = −0.03, −d ≤ t ≤ 0,

x3(0) = 0.1, x4(0) = 0.

Delays d = 0.1, d = 0.2, d = 0.4 in the state variables x1, x2 .


Optimal control problem for the Two-Stage CSTR

Minimizetf∫0

( x21 + x22 + x23 + x24 + 0.1u21 + 0.1u22 ) dt (tf = 2) .

Hamiltonian with yk(t) = xk(t − d), k = 1, 2 :

H(x , y , λ, u) = f0(x , u) + λ1x1+λ2(−(x2 + 0.25)− u1(x2 + 0.25) + R1(x1, x2) )

+λ3(y1 − x3 − R2(x3, x4) + 0.25)

+λ4(y2 − 2x4 − u2(x4 + 0.25) + R2(x3, x4) + 0.25)

Advanced adjoint equations:

λ1(t) = −Hx1(t)− χ [ 0,tf−d ](t)λ3(t + d),

λ2(t) = −Hx2(t)− χ [ 0,tf−d ](t)λ4(t + d),

λk(t) = −Hxk (t) (k = 3, 4).

The minimum condition yields Hu = 0 and thus

u1 = 5λ2(x2 + 0.25), u2 = 5λ4(x4 + 0.25).


Two-Stage CSTR with free x(tf ) : x1, x2, x3, x4

-0.02 0

0.02 0.04 0.06 0.08 0.1

0.12 0.14 0.16

0 0.5 1 1.5 2

concentration x1

d=0.1d=0.2d=0.4

-0.03

-0.02

-0.01

0

0.01

0.02

0 0.5 1 1.5 2

temperature x2

d=0.1d=0.2d=0.4

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

0 0.5 1 1.5 2

concentration x3

d=0.1d=0.2d=0.4

0 0.005 0.01

0.015 0.02

0.025 0.03

0.035 0.04

0.045

0 0.5 1 1.5 2

temperature x4

d=0.1d=0.2d=0.4

Delays d = 0.1, d = 0.2, d = 0.4.


Two-Stage CSTR with free x(tf ) : u1, u2, λ2, λ4

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u1

d=0.1d=0.2d=0.4

0

0.05

0.1

0.15

0.2

0.25

0.3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u2

d=0.1d=0.2d=0.4

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

adjoint variable λ2

d=0.1d=0.2d=0.4

0 0.02 0.04 0.06 0.08 0.1

0.12 0.14 0.16 0.18 0.2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2


d=0.1d=0.2d=0.4

Delays d = 0.1, d = 0.2, d = 0.4.


Two-Stage CSTR with x(tf ) = 0 : x1, x2, x3, x4

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0 0.5 1 1.5 2

concentration x1

d=0.1d=0.2d=0.4

-0.03

-0.02

-0.01

0

0.01

0.02

0 0.5 1 1.5 2

temperature x2

d=0.1d=0.2d=0.4

-0.02

0

0.02

0.04

0.06

0.08

0.1

0 0.5 1 1.5 2

concentration x3

d=0.1d=0.2d=0.4

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0 0.5 1 1.5 2

temperature x4

d=0.1d=0.2d=0.4

Delays d = 0.1, d = 0.2, d = 0.4.


Two-Stage CSTR with x(tf ) = 0 : u1, u2, λ3, λ4

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u1

d=0.1d=0.2d=0.4

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u2

d=0.1d=0.2d=0.4

-0.1

-0.05

0

0.05

0.1

0.15

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2


d=0.1d=0.2d=0.4

0

0.05

0.1

0.15

0.2

0.25

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2


d=0.1d=0.2d=0.4

Delays d = 0.1, d = 0.2, d = 0.4.


Two-Stage CSTR with x(tf ) = 0 and x4(t) ≤ 0.01

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 0.5 1 1.5 2

temperature x4

d=0.1d=0.2d=0.4

-0.5-0.4-0.3-0.2-0.1

0 0.1 0.2 0.3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u1

d=0.1d=0.2d=0.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0 0.5 1 1.5 2

multiplier η for x4 <= 0.01

d=0.1d=0.2d=0.4

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

control u2

d=0.1d=0.2d=0.4

Delays d = 0.1, d = 0.2, d = 0.4.


Two-Stage CSTR: time-optimal with x(tf ) = 0

d = 0 : tf = 1.8725496

-0.2-0.15-0.1

-0.05 0

0.05 0.1

0.15 0.2

0 0.5 1 1.5 2

control u1

-0.6-0.5-0.4-0.3-0.2-0.1

0 0.1 0.2 0.3 0.4

0 0.5 1 1.5 2

control u1 and (scaled) switching function φ1

u1φ1

-0.2-0.15-0.1

-0.05 0

0.05 0.1

0.15 0.2

0 0.5 1 1.5 2

control u1

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0 0.5 1 1.5 2

control u1 and (scaled) switching function φ1

u1φ1

d = 0.4 : tf = 2.0549

Boccia, Falugi, Maurer, Vinter : CDC 2013 .Vinter (2016) : Maximum Principle in general form.


Compartment Model for Tuberculosis

C.J. Silva and D.F.M. Torres, Optimal control strategies forTuberculosis treatment: a case study in Angola, Numerical Algebra,Control and Optimization, 2 (2012), 601–617.

C.J. Silva, H. Maurer, and D.F.M. Torres, Optimal control of aTuberculosis model with state and control delays, MathematicalBiosciences and Engineering 14(1), pp. 321–337 (2017).

5 state variables and delay in I (t):

S : Susceptible individualsL1 : early latent individuals, recently infected (less than two years)I : infectious individuals, who have active TBL2 : persistent latent individualsR : recovered individualsN : total population N = S + L1 + I + L2 + R, assumed constant

dI : delay in I , represents delay in diagnosis


Dynamical TB model with state and control delays

Control variables and delays:u1 : effort on early detection and treatment

of recently infected individuals L1,du1 : delay on the diagnosis of latent TB

and commencement of latent TB treatment,u2 : chemotherapy or post-exposure vaccine

to persistent latent individuals (L2),du2 : delay in the prophylactic treatment of persistent latent L2.

Dynamical model : R = N − S − L1 − I − L2

S(t) = µN − βN I (t)S(t)− µS(t),

L1(t) = βN I (t)(S(t) + σL2(t) + σRR(t))−(δ + τ1 + ε1u1(t − du1) + µ)L1(t),

I (t) = φδL1(t) + ωL2(t) + ωRR(t)− τ0I (t − dI )− µI (t),

L2(t) = (1− φ)δL1(t)− σ βN I (t)L2(t)−(ω + ε2u2(t − du2) + τ2 + µ)L2(t).


Optimal control problem with state and control delays

Initial functions in view of delays (N = 30000) :

S(0) = 76N/120, L1(0) = 36N/120, L2(0) = 2N/120,I (t) = 5N/120 ∀ − dI ≤ t ≤ 0, R(0) = N/120,

uk(t) = 0 ∀ − duk ≤ t < 0, k = 1, 2.

Control constraints

0 ≤ u1(t) ≤ umax1 = 1, 0 ≤ u2(t) ≤ umax

2 = 1 for 0 ≤ t ≤ T .

Control problem: x = (S , L1, I , L2) ∈ R4, u = (u1, u2) ∈ R2

Minimize L1 objective or L2 objective

J1(x , u) =∫ T0 ( I (t) + L2(t) + W1 u1(t) + W2 u2(t) ) dt ,

J2(x , u) =∫ T0 ( I (t) + L2(t) + W1 u1(t)2 + W2 u2(t)2 ) dt

(weights W1,W2 > 0)

subject to the dynamic and control constraints.


Parameters in the TB Model

Symbol Description Valueβ Transmission coefficient 100µ Death and birth rate 1/70 yr−1

δ Rate at which individuals leave L1 12 yr−1

φ Proportion of individuals going to I 0.05ω Reactivation rate for persistent latent infections 0.0002 yr−1

ωR Reactivation rate for treated individuals 0.00002 yr−1

σ Factor reducing the risk of infection as a result ofaquired immunity to a previous infection for L2 0.25

σR Rate of exogenous reinfection of treated patients 0.25τ0 Rate of recovery under treatment of active TB 2 yr−1

τ1 Rate of recovery under treatment of L1 2 yr−1

τ2 Rate of recovery under treatment of L2 1 yr−1

N Total population 30, 000T Total simulation duration 5 yrε1 Efficacy of treatment of early latent L1 0.5ε2 Efficacy of treatment of persistent latent L2 0.5

Tabelle: Parameter values.


Optimal controls of the non-delayed control problems

Adjoint variable λ = (λS , λL1 , λI , λL2)

Switching functions φk(t) = Huk [t] = −Wk − εkλLk (t) Lk(t) (k = 1, 2).

Control law for Maximum Principle:

uk(t) =

{1 , if φk(t) > 00 , if φk(t) < 0

}, k = 1, 2.

0

0.5

1

1.5

2

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u1 and switching function φ1

u1φ1

-0.5

0

0.5

1

1.5

2

2.5

3

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u2 and switching function φ2

u2φ2

Weights W1 = W2 = 50 : Optimal controls are bang-bang.SSC are satisfied, since SSC hold for the switching timeoptimization problem and the strict bang-bang property holds.


State Variables in the Non-Delayed Control Problem

Optimal state variables for weights W1 = W2 = 50 :

0 5000

10000 15000 20000 25000 30000 35000

0 1 2 3 4 5

susceptible S and recovered R

SR

0 200 400 600 800

1000 1200 1400 1600

0 1 2 3 4 5

infectious I

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

0 1 2 3 4 5

early latent L1

0

2000

4000

6000

8000

10000

12000

14000

0 1 2 3 4 5

persistent latent L2


Optimal controls : comparison of L1 and L2 objectives

Objectives:

J1(x , u) =∫ T0 ( I (t) + L2(t) + W1 u1(t) + W2 u2(t) ) dt,

J2(x , u) =∫ T0 ( I (t) + L2(t) + W1 u1(t)2 + W2 u2(t)2 ) dt

L2 functional : optimal controls are continuous.

0 0.2 0.4 0.6 0.8

1 1.2 1.4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u1 for J1 and J2 objective

J1J2

0 0.2 0.4 0.6 0.8

1 1.2 1.4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u2 for J1 and J2 objective

J1J2

Weights W1 = W2 = 50


Optimal controls for delays dI = 0.1, du1 = du2 = 0.2

Switching functions for k = 1, 2

φk(t) =

{−Wk − εkλL1(t + duk )L1(t + duk ) for 0 ≤ t ≤ T − duk−Wk for T − duk ≤ t ≤ T

}Optimal controls u1 and u2 are bang-bang with one switch.The strict bang-bang property holds.

-0.2 0

0.2 0.4 0.6 0.8

1 1.2

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u1 and (scaled) switching function φ

u1φ1

0 5000

10000 15000 20000 25000 30000 35000

0 1 2 3 4 5

susceptibles S and recovered R

SR

0 200 400 600 800

1000 1200 1400 1600

0 1 2 3 4 5

infectious individuals I

-0.5 0

0.5 1

1.5 2

2.5 3

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u2 and (scaled) switching function φ

u2φ2

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

0 1 2 3 4 5

early latent L1

0 2000 4000 6000 8000

10000 12000 14000

0 1 2 3 4 5

early latent L2


Sensitivity of optimal control w.r.t. parameter β

0 0.2 0.4 0.6 0.8

1 1.2 1.4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u1

β=50β=150

0 0.2 0.4 0.6 0.8

1 1.2 1.4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

control u2

β=150β=150

Comparison of extremal controls for β = 50 and β = 150 :

L1 objective, W1 = W2 = 150, delays dI = 0.1, du1 = du2 = 0.2.


Trahison des Images ?

Ceci n’etait pas une conference.

Merci pour votre attention !


theory and applications of optimal control problems with

Documents