Jounml of Mathematical Sciences, Vol. 97, No. 2, 1999

O P T I M A L C O N T R O L OF T H E M O N O D - I Y E R U S A L I M S K I I KINETIC S Y S T E M

I. N. L y a s h e n k o and E. I. L y a s h e n k o UDC 517.977

A special problem of optimal control of a closed three-dimensional kinetic system of ordinary differential equations is studied. This problem is a mathematical model of optimal harvesting, which has the fol-Iowing biological interpretation. From a medium with a continuously growing population, stimulated by a flow of nutritious substances and inhibited by excretory products, a part of the biomass is extracted, continuously or discretely, and thus excluded from the reproductive cycle. The problem is posed to find the control of biomass production such that the total harvest gathered over a fixed time interval [0, 7"] is maximal. At the final moment T the process @ terminated by extracting the remainder of the biomass. Bibliography: 11 titles.

BACKGROUND

In the continuous case, the one-dimensional optimal harvesting problem for an autonomous system was formulated by R. Bellman [1]. He also investigated the problem using methods of the qualitative theory of differential equations. In the discrete case, a particular one-dimensional problem for an autonomous system (the Ferhiilst model) was solved by Yu. M. Svirezhev and E. Ya. Elizarov [2, 3] with the help of the recurrence-relation method of dynamic programming. This method was also used for investigating a general one-dimensional problem for a nonautonomous system [4] (first in the discrete case and then, passing to limit, in the continuous case).

N. D. Iyerusalimskii and N. M. Neronova [5] established a functional relation between the growth rate of microorganisms and the concentration of substrate and metabolic products. A relation between the growth rate and lighting was established in [6, 7]. In [8], a system of kinetic equations, based on the substance balance (the Monod-Iyerusalimskii model), was proposed for the concentrations of biomass, nutritious substances, and metabolic products. In [9, 10], the problem of optimization of transient processes arising in bringing the cultivator to the steady state was numerically investigated. However, this problem for the Monod-Iyerusalimskii model thus far remains unsolved.

Experiments on continuous cultivation of bacteria showed that nothing more than the minimum sub- strate concentration, the maximum inhibitor concentration, and the biomass concentration is needed to describe the main distinctive features of the biomass growth [5]. The relationship between the specific rate of the enzymatic reaction and the substratum concentration is described by the Michaelis-Menten equation

#ms (1) ~1 = "

ks + s

where #m is the limit to which the specific rate #1 tends with increasing substrate concentration s, and ks is a constant equal to the substrate concentration for/~1 = [~rn/2. The retardation of the enzymatic reaction by inhibitors, including metabolic products, obeys the same law by which the substrate stimulates its progress. The following equality holds [5]:

p s k , (2) I z = 1~1 -- Pl kp + p = #m ks + s kp + p '

where p is the concentration of metabolic products and kp is a constant equal to the product concentration at which # =/~1/2.

Let a nutrient medium of concentration s o arrive continuously at the cultivator at a dilution rate of u ( l /h) , and let, the mixture of unreacted nutrient medium, biomass, and metabolic products leave the

Translated from Obchyslyuval~na ta PrykIadna Matematyka, No. 80, 1996, pp. 47-58, Original article submitted December 10, 1995.

3 9 3 2 1072-3374/99/9702-3932522.00 �9 Kluwer Academic/Plenum Publishers

cultivator at the same rate u ( l /h) . Since nutrition is the only source of carbon for the formation of cellular mass and metabolic products, it is natural to make use of the law of conservation of mass. To this end, it is necessary to express all the quantities involved in the balance in common units, namely in moles of carbon�9 The kinetic equations in the new notation are known as the Monod-Iyerusalimskii model [8-10]:

# o x y

= (% + y)(k~ + z) - ~ '

9 = - a #oZy + u(y o _ y), (k~ + v)(k~ + z)

= ( a - I) ~ o z y (k~ + v)(k~ + z) - ~z,

(3)

where a = const > 1 is the efficiency coefficient, equal to the amount of the limiting component needed for an increase of the biomass by one unit, and y0 is the concentration of the arriving nutrient substrate (in moles of carbon)�9 The system of equations (3) is closed and relates the concentration of biomass x with the concentrations of nutrient substrate y and metabolic products z.

GENERAL KINETIC SYSTEM

Consider a general closed system of kinetic equations of the form

= f ( t , x , y, z) - ux , x(O) = xo,

9 = - o f ( t , x , y, z) + u ( y ~ - y) , y(O) = Yo,

= (~ - 1 ) f ( t , ~, y , ~) - ~z, z(O) = zo,

(4)

where f ( t , x , y , z) is a known function satisfying the conditions for the existence of a solution of the initial- value problem (4) on the interval [0, T].

The differential equations (4) have no biological specificity and therefore they can describe any contin- uous technological process accompanied by the ingress of raw materials and egress of products�9

For the system of kinetic equations (4) we formulate the optimal harvesting problem in the form of the following optimal control problem:

fo T F ( u ) = ux dt + x ( T ) --~ sup , (x,y,z,u)6W

= f ( t , X, Y, Z) -- ~X, x(O) = Xo,

9 = - o f ( t , x, v, z) + u(v ~ - y), v(o) = yo < v ~

= (~ - 1 ) / ( t , x, v, z) - ~ , z(O) = zo,

~( t ) > o, v( t ) > o, z ( t ) > o, ~( t ) > o.

(5)

(6) (7) (8) (9)

Here V t is the admissible region described by relations (6)-(9) and u( t ) is the control of the following technological meaning: u ---- 0 (inaccurate process), 0 < u < oc (running process), u = c~ (a partial discharge of the mass, immediate dilution)�9

In accordance with the generalized theorem on sufficient conditions for optimality of continuous pro- cesses [11, p. 122], we construct an auxiliary continuously differentiable function ~(t, x, y, z) such that a sequence of admissible processes {xs(t), ys(t), zs(t) , Us(t)} E V t, s = 1, 2 , �9149 is a minimizing sequence for the functionals

O~ O~ O~ R ( t , x , y , z , u ) = - ~ + ~ - ~ x ( f - u x ) + - ~ y ( - a f + u(y ~ - y)) + ~ ( ( a - 1 ) f - u z ) + ux -~

d2(x, y, z) = ~ (T , x, y, z) - x ( T ) -+ inf (~,u,z)ev$

sup , (x,y,z,u)eV ~

3933

Choosing an auxiliary function p(~, x, y, z) so that the functional R(t, x, y, z, u) is independent of u, we get the relation

0p 0p 0p x ~ + (v - v ~ + z ~ = x,

from which we conclude tha t it is possible to set p(t, x, y, z) = x. In this case ~(z, y, z) - 0 and a sufficient condition for optimality of the problem (5)-(9) is the condition

n ( t , ~ , y , z , ~ ) = i ( t , ~ , y , z ) ~ max (10) (~,u,z)ev2

uniformly in all t E [0, T]. Note that the functional F ( u ) of the problem (5)-(9) can be rewrit ten in the form

// F(~) = ~(o) + I [t, ~(t), y(t) , z ( t ) ] dr,

and the requirement for its optimality agrees with the requirement (10). The system of kinetic equations (6)-(8) is a closed system of differential equations. Adding the equa-

tions (6), (7), and (8) and then integrating the equation obtained, we get

( f ) ~( t ) + v ( t ) + z( i ) - v ~ = (=o + v0 + z0 - v ~ exp - ~ d t . (11)

Now we multiply Eq. (6) by ~ - 1, subtract Eq. (8) from it, and then integrate the equation obtained to get

( f ) ( o t - - 1 ) x ( t ) - z ( t ) = ( ( c~- - l )x0- -zo)exp -- u d t . (12)

Adding (11) and (12), we get the equality

( f ) y O _ a x ( t ) _ y ( t ) = ( y O _ a x o _ Y o ) e x p -- u d t . (13)

Hence the system of three differential equations (6)-(8) is equivalent to one differential equation (6) and two balance relations (12), (13).

The initial conditions of the system (6)-(8) are called consistent if they are chosen so that

aXo + Yo = yO, z0 = (c~ - 1)xo. (14)

In the case of consistent initial conditions (14), the system of equations (6), (12), (13) becomes simpler and equivalent either to the system

= I ( t , ~, v , z ) - ~=, =(0) = ~o, (15)

v = v ~ - ~ = , z = (~ - 1 )= (16)

or to the one-dimensional initial-value problem

= f( t , x, yO _ c~x, (~ - 1)=) - u x , x(O) = Xo. (17)

Solutions x( t ) > O, y ( t ) = yO _ a x ( t ) > O, and z ( t ) = (~ - 1)x(t) >_ 0 of the system (15), (16) are also called consistent solutions.

The sufficient optimality condition (10) on a consistent solution has the form

f (t , x ( t ) , yO _ ax($), (o~ - 1)x(~)) = f0 (t, x($)) ---+ max (18) o<z(t)<v~ lo,

uniformly in all t 6 [0, T]. Under the condition of uniqueness of an optimal solution ~(t) of the problem (18), the following cases

are possible:

1) ~(t) = 0, 9(t) =- yO, ~(t) = 0 (nothing is cultivated); 2) 0 < ~(t) < yO/~, 9 ( t ) = y0 _ c~(t) > 0, 5(t) = (a - 1)~(t) > 0 (the optimal process proceeds); 3) ~(t) = yO/e, 9(t) ---- 0, 5(t) = (~ - 1)y~ (the growth is stopped).

3934

Remark. If the initial conditions of system (6)-(8) are inconsistent, that is, the conditions (14) are not fulfilled, then, by virtue of relations (12), (13), for u > 0 and t >> 1/u the solutions x(t), y(t), and z(t) reach the s teady state and turn asymptotically into consistent solutions.

Under the uniqueness condition the optimal solution ~(t), obtained from (18), is a turnpike trajectory for the general opt imal harvesting problem (5)-(9):

~(t) = arg max f ( t , x, yO _ ax, (a - 1)x) (19) O<_x<y~

uniformly in all t E [0, T]. If a turnpike trajectory ~(t) has already been constructed, then the optimal control can be found from

Eq. (6), that is,

On a consistent solution the three-dimensional opt imal harvesting problem (5)-(9) reduces to the one-dimensional problem of the form

// F(u) = ux dt + x (T) --* sup, (21)

gc = fo(t, x) - ux, x(O) = xo, (22)

0 < x(t) < y0/ , _> 0, (23)

where f0(t, x) = f ( t , x, yO _ ax , (a - 1)x). (24)

The opt imal control problem (21)-(23) is a generalization of the optimal harvesting problem considered in [4] and differs from the latter only in the additional restriction x(t) < y~ The results obtained in [4] can be easily extended to the problem (21)-(23).

L e m m a 1. Let, in the domain [0 < t < T] • [0 < Xo <__ x < y~ a function fo( t ,x) be continuously differentiable with respect to t, x, and xx and convex in x for every t. Then the solution of the initial-value problem

gc = fo( t ,x) , x(O) = xo (25)

is a monotone-increasing and convex-upward function of the initial value Xo.

L e m m a 2. Under the conditions of Lemma 1, a turnpike trajectory Jz(t) is "stable" (unique) if

0 < a rgmax fo(t, x) = ~(t) < y ~ (26) x

uniformly in all t E [0, T].

T h e o r e m . Under the conditions of Lemmas 1 and 2, the optimal harvesting problem (21)-(23) has a unique solution. The maximal harvest is given by the relation

{ foT fo(t,~c(t))dt

F(~z) = Xo + fo fo(t, x(t)) dt + f / f o ( t , ~.(t)) dt

f [ fo ( t , x ( t ) )d t

for e(O) <_ zo,

for x~ nin <: x 0 < x(O),

for x 0 < x~ nin, (27)

where v is the moment at which a trajectory of the solution z( t ) of the initial-value problem (25) crosses the turnpike trajectory (26); x~ in is the initial value of a trajectory of the solution x(t) of the initial-value problem (25) that crosses the turnpike (26) for the first t ime only at the final moment T.

On the basis of this theorem, in the case of semiconsistent initial conditions x0 + Yo + zo = yO it is quite easy to solve the problem of optimization of a t ransient process, that is, the problem of the fastest way of going from the initial state (xo, Yo, z0) into the turnpike trajectory.

3935

Indeed, if x0 > ~(0), then it is necessary to eliminate the excess biomass concentration Xo - ~(0) by immediate dilution of the culture by the nutrient medium of concentration yO. This is accomplished by discharging the part 7 --- 1 - ~c(O)/xo of the cultivator volume and subsequently filling the released volume by the nutrient medium of concentration yO. It is not difficult to calculate that in this case the initial conditions become consistent, while the initial biomass concentration falls into the turnpike. Further control lies in maintaining the biomass concentration x(t) in the optimal mode ~(t) by choosing the program control (20). In doing so, the nutrition concentration y(~) and the product concentration z($) tend to their optimal values ~(t) and 5(t).

If Xo < ~(0), then we use the control u = 0 up to the moment ~ = u when the growth trajectory x(~) goes into the turnpike, x(u) ---- ~(v). After this, we keep the concentration x(t) on the turnpike trajectory by program control (20).

As was observed in [10], the main functioning mode of the continuous cultivator is the turnpike mode when there exists an equilibrium in the cultivator, that is, over a long period of t ime the biomass, substrate, and product concentrations are in the optimal ratio. The cultivator can be started either from some optimal initial conditions ~(0), ~(0), and 5(0), or from some "seeding biomass" satisfying the optimal initial conditions ~(0), ~(0), and 5(0) but considerably smaller by volume than the cultivator. In the first case, the growth trajectory is on the turnpike from the very beginning, while in the second case the filling of the cultivator with the mixture of the "seeding mass" and nutrient medium of concentration y0 leads to the cases of semiconsistent initial conditions x0 < ~(0).

In the general case of inconsistent initial conditions, the optimization of the transient process presents a real challenge. We can only recommend that one preliminarily bring the initial conditions to consistency in one way or another.

THE MONOD-IYERUSALIMSKII KINETIC SYSTEM

Consider the optimal harvesting problem in the case of autonomous system of differential equations (3), tha t is, when in (5)-(9) we have

I(t, z , y , z ) = ~ o z y (ky + y)(k= + z)" (28)

Then

. o z ( y ~ - a z ) f o ( x ) = (k v -F yO _ a x ) ( k z q- (a - 1)x)'

- ~ k ~ x yo _ ~ z k~ ] I;(=) = . o (k~ + y0 - ~=)2 kz + (~ - 1)= + k~ + 7 - ~ = (kz + ( ~ : 1)=)2j"

From the necessary extremum condition f~ (x) = 0 we get the quadratic equation

Y x + = 0 , I

which always has one root lying in the interval (0, y0/~), namely,

~(yo) = yo /~

1 _t_ v / ~ '

D = I k y + y o I c~ > 0 .

where

Since

2a2ky (c~ - 1)x :g (=) = , o - (k~ ~:~ -- ~x)3 kz + (~ - 1)=

2ak u ( a - 1)k= (k. + yO _ c~x)2 (k= + (a - 1)x) 2 _- .2L - .l

(29)

(3o)

< O,

0 < x < yO/o 6

3936

the function fo(x) is convex upward on [0, y~ and the point 2, defined by relation (29), is a local maximum point.

Moreover, yO4-5

~(yO)_ l + v ~ '

and the program control (20) acquires the form

~(yo)=

~(yo) _ ((i - 1 )y~ 1 + v ~ ' (31)

,oyO(1 +{-~){-~

Note that

d ~(yO) = 2(ky + y ~ + 3kyy ~

dY ~ 2(iVY(1 + V"-'D )2(kv + yO)2 + y ~ 1 7 6 1 7 6 1)ky/(~k2) > 0,

2 ~ 4 - ~ ( 1 + v ~ ) ~ ( k ~ + vo) ~

(32)

and, as a consequence, 2(y ~ is a monotone-increasing function of yO. Let us analyze the value of the functional of the corresponding optimal control problem (5)-(9) on the

turnpike trajectory. We have

F(~) = ~(v ~ + Fo(y ~ = 2(y ~ + T~o (1 + k~/yo + % / v o ~ ) ( ( i _ 1 + (ikz/V o + ~ k z v ~ / y o ) "

The first te rm in this relation coincides with the initial value 2 of the turnpike trajectory. The second term is the harvest increase over the period [0, 7"]. We have

dY o ~-6 + y - ~ ( i - 1+ -~-- + yO ,]

x 1+ 5 x/-D-2v/_~(kv+yO) 2 1 ( i - l + + ( i ~ 7 7 ] ( o

+ I+V+ ~ 1+V5+ i >o. yO 2 2(k v + yO)2v'-D a k~

Hence the harvest increase on the turnpike is a monotone-increasing function of the incoming nutrition concentration yO, and therefore it attains its upper limiting value as yO __~ co, namely,

lim Fo(y ~ = T / t o , (33) yO--~OO ( i L

which is a bounded value for (i > 1. The parameter yO must be chosen, and its value depends on the preassigned level of the turnpike 2.

We have /

yO _ (i2 = (i2 { 1

From here we get the quadratic equation

y02 _ (2(i2 - k~)y ~ + (i2~2 _ (i((i - 1 ) k # 2 kz

ky + yO 1 a -- "

- 2kv(i2 = O,

with only one root, namely

yO = a2. - + + akv2 + a ( a - 1)~kv k~

(34)

3937

satisfying the condition yO _ ~ ~ 0. As was noted in [10], the domain of existence of the variables x, y, and z is determined by biological

restrictions: the biomass should not be squashed because of its excessive thickness (0 < x ~ Xma• and the substrate should not suppress the growth (0 < y < Ycrit), since cells can degenerate if y > Ycrit- Relations (29), (31), and (34) simply offer a means of choosing the value of the main parameter of the Monod-Iyerusalimskii model yO.

Let (xo, Y0, z0) be consistent initial conditions, and let u = 0 be a control. Then the solution of the corresponding initial-value problem

~0x(y ~ - ~z ) x(o) = ~o, (35) = (ky + yO _ o~)(kz + (o - 1)~)'

can be obtained in the following explicit form:

ky + y ~ ! + k~(kz + ( ~ - i ) y ~ yO _ ~ o (o 1)(x x0) + yO xo ~ ,n ~-o : _ ~ = t.

Since dc(t) > 0, the function x(t) is a monotone-increasing function and the t ime t = u of achievement of the turnpike is found from the relation

yO _ ox0 , = (~ - 1)(~ - Xo) + ky yO + y_.___.~o kz In Xxo- + lcu(kz + (C~yO- 1)Y~ In yO - mY: ' (36)

where �9 is determined by relations (29), (30). In conclusion, we present numerical values of the constants involved in Eq. (3) for the bacteria Pro-

pionibacterium shermanii growing on nutrient media with lactate as a source of carbon and inhibited by fermented salts of propionic and acetic acids, which were found experimentally [5]: /~,~ = 0.047 mole C per hour, k v = 0.108 mole C per hour, kz = 0.034 mole C per hour. According to N. M. Neronova, ~ = 6 + 2 and yO varies from 0 to 1.

REFERENCES

1. R. E. Bellman, I. Glicksberg, and O. A. Gross, Some aspects of the mathematical theory of control processes, Rand Corporation, Santa Monica, California (1958).

2. Yu. M. Svirezhev and E. Ya. Elizarov, "Mathematical modeling of biological systems," Probl. costa. biol., 20, 36-41 (1972).

3. A. B. Gorstko and G. A. Ugol'nitskii, Introduction to the modeling of ecological and economics systems [in Russian], Rostov University, Rostov-on-Don (1990).

4. O. I. Lyashenko, "Optimal harvesting models with a convex-upward function of population growth rate," Obchysl. Prykl. Mat., No. 77, 75-86 (1993).

5. N. D. Iyerusalimskii and N. M. Neronova, "A quantitative relation between the concentration of meta- bolic products and the growth rate of microorganisms," Dokl. Akad. Nauk SSSR, 161, No. 6, 1437-1440 (1965).

6. B. N. Belyanin and B. G. Kovrov, "On the mathematical model of biosynthesis in a light-limited culture of microorganisms," Dokl. Akad. Nauk SSSR, 179, No. 6, 1463 (1968).

7. I. N. Lyashenko and Sh. R. Redzhepova, Mathematical modeling and optimal solar power engineering systems [in Russian], Ylym, Ashgabad (1989).

8. N. V. Stepanova, Yu. M. Romanovskii, and N. D. Iyerusalimskii, "Mathematical modeling of the growth of microorganisms in a continuous culture," Dokl. Akad. Nauk SSSR, 163, No. 5, 1266-1269 (1965).

9. I. G. Minkevich, N. V. Stepanova, T. A. Fyodorova, and V. I. Shmal'gauzen, "On the fastest way of bringing of a cultivator to stable operating conditions," Biofizika, 15, No. 5, 867-872 (1970).

10. N. V. Stepanova, T. A. Fyodorova, and V. I. Shmal'gauzen, "Optimization of transient processes of growth of microorganisms in a continuous culture (a model allowing for the inhibition by products of secondary metabolism)," Biofizika, 16, No. 5, 841-848 (1971).

11. V. F. Krotov (ed.), Foundations of the theory of optimal control [in Russian], Vysshaya Shkola, Moscow (1990).

3938