real-time dynamic optimization of non-linear batch systems

338 THE CANADIAN JOURNAL OF CHEMICAL ENGINEERING VOLUME 84, JUNE 2006

INTRODUCTION

Industrial chemical processes can be divided into two catego-ries of production: continuous and fed-batch. Continuous processes are designed to run at steady-state. Examples

include oil refi ning, gas processing, and many chemical processes. To maximize the effi ciency and profi ts from these processes, it is necessary to keep the plant in the operating range under distur-bances. The optimization task required to operate such processes is usually performed to achieve disturbance rejection, designing controllers to reach and maintain set-point effectively, and keeping the down time to a minimum. Since the operating range is generally very narrow, the system dynamics can often be approximated by linear dynamics. Batch processing, however, provides some very unique challenges. Batch processes have a fi nite operating time, rather than a continuous operation. The control objective in batch processing is not to reach steady state, but to reach some desired objective by the end of the batch. This usually involves movement through a very wide operating range, and non-linearities in the system can be very strong. Batch optimization focuses on maximizing the performance objective by fi nding the corresponding input variable and state variable trajectories. Since batch production is usually of low volume,

Real-time Dynamic Optimization of Non-linear

Batch Systems

Nathaniel Peters, Martin Guay* and Darryl DeHaan

Department of Chemical Engineering, Queen’s University, Kingston, ON, Canada K7L 3N6

high value production, optimization of its operation is critical to make the process viable. Examples include pharmaceuticals, specialty chemicals, biological processes and food production.

A signifi cant amount of research has been done in the area of on-line (or real-time) optimization of batch processes. Numerous methods have been investigated including dynamic program-ming (Hestenes, 1966; Bellman, 1957), discretization (Cuthrell and Biegler, 1987) and parameterization (Visser et al., 2000). These techniques have been incorporated for a variety of model structures and optimization algorithms.

Dynamic or non-linear programming algorithms have been used in the development of on-line optimization techniques. The main challenge in the application of on-line optimization techniques is the signifi cant computing demands required for the computation of the updated profi les. This relatively high demand for computing time restricts the frequency of updates, and consequently these methods usually result in highly discre-tized on-line updates of the optimal trajectories (look at Noda et al., 2000; Zuo and Wu, 2000; Arpornwichanop et al., 2005;

In this paper, a methodology for designing and implementing a real time optimizing controller for non-linear batch processes is discussed. The controller is used to optimize the system input and state trajectories according to a cost function. An interior point method with penalty function is used to incorporate constraints into a modifi ed cost functional, and a Lyapunov-based extremum seeking approach is used to compute the trajectory parameters. Smooth trajectories were generated with reduced computing time compared to many optimizations in literature. In this paper, the theory is applied to general non-fl at non-linear systems in a true on-line optimization.

Dans cet article, on étudie une méthodologie pour la conception et l’implantation d’un contrôleur d’optimisation en temps réel de procédés discontinus non linéaires. Le contrôleur sert à optimiser l’entrée du système et les trajectoires d’état d’après une fonction de coûts. Pour calculer par ordinateur les paramètres de trajectoire, on recourt à une méthode à points intérieurs avec une fonction de pénalités pour incorporer des contraintes dans une méthode de recherche des extrêmes lyapunovienne et une fonctionnelle de coût modifi ée. Des trajectoires lisses ont été produites avec un temps de calcul réduit comparativement à de nombreuses optimisations de la littérature scientifi que. Dans cet article, la théorie est appliquée à des systèmes non linéaires non plats généraux dans une véritable optimisation en ligne.

Keywords: real-time optimization, dynamic optimization, batch systems

* Author to whom correspondence may be addressed.E-mail address: [email protected]

VOLUME 84, JUNE 2006 THE CANADIAN JOURNAL OF CHEMICAL ENGINEERING 339

Loeblein et al., 1999; Fournier et al., 1999; Zhang and Rohani, 2003). As a result, these techniques can often be referred to as “pseudo” on-line optimization. To compensate for the time required for computation, the profi les may be parameterized to known optimal trajectories (see Visser et al., 2000). This approach can be very effective in implementations, since the structure of the optimal profi les is known. However, the explicit knowledge of an optimal structure at the onset represents a considerable challenge in application. Since it is widely known that the solution set of optimal control problems are often discontinuous, there is no guarantee that any optimal structure can be applied successfully to another batch with varying initial conditions and parametric and modelling uncertainties, even if those effects remain small. The discontinuity of the solution set may cause some drastic changes in the solution and, hence, the appropriate choice of optimal structure.

In a series of papers by Palanki and Rahman (Palanki et al., 1993; Palanki and Rahman, 1994; Rahman and Palanki, 1996, 1998) a method is introduced that provides a geometric approach to handling batch optimization. They show how to develop feedback laws for end-point optimization problems under a variety of state space variations. This technique gives smooth optimal trajectories, however, this is only implementable for end-point optimization. Some off-line analysis is needed to handle the singular and non-singular sections of the feedback laws. Path constraints were not implemented.

Gattu and Zafi riou (1999) used a gradient method to defi ne a parameter update law for the input variables and state variables trajectories and then tracked it via a controller. This forms the basis for the theory presented here. One of the pitfalls of this cascade optimization is that it is essentially an optimization over the initial conditions, and, hence, not a true on-line optimiza-tion. The method discussed here will give a more mathemati-cally solid defi nition of the update law using a Lyapunov-based extremum seeking method. This paper will develop an optimiza-tion that minimizes the cost of the remaining batch, giving an optimization based on the current conditions.

While the development of this methodology comes from the extremum seeking literature, the methodology is quite close to that of Non-Linear Model Predictive Control (DeHaan and Guay, 2005; Fontes, 2001; Magni and Scattolini, 2004; Jadbabaie et al., 2001). Work has been done on the use of shrinking horizon NMPC for batch systems (Nagy and Braatz, 2003; Valappil and Georgakis, 2002).

One of the natural way to solve dynamic optimization problems is to approximate the optimal trajectories using a specifi c parametrization of the input variables and state variables trajectories. In doing so, many authors have considered the exploitation of system structure. One very effective approach can be developed for solving batch optimization problems for systems that are differentially fl at. Some results have been published on control of differentially fl at systems (e.g. Delaleau and Rudolph, 1998; van Nieuwstadt and Murray, 1998) and a number of fl atness-based dynamic optimization methods have been reported (e.g. Faiz et al., 2000; Oldenburg and Marquardt, 2002; Mahadevan et al., 2000; Mahadevan et al., 2001; Guay et al., 2001). Differential fl atness is a property introduced in Rouchon et al. (1995) of a special class of under-determined dynamical control systems whose dynamics can be freely assigned without differential constraints. For such systems, all input variables and state variables trajectories can be assigned by defi ning paths in the so-called fl at out-puts. The main advantage of fl atness-based approaches is the elimination of

numerical integration of the model in the solution of optimal control problems. This property has led to the development of effi cient algorithms (Mahadevan et al., 2000; Mahadevan et al., 2001). Mahadevan et al. (2001) proposed a fl atness-based model predictive technique for fed-batch reactors. A real-time optimiza-tion scheme is proposed that implements a model-predictive approach where the dynamic programming problem is solved at pre-specifi ed time intervals. At each subsequent interval, the dynamic programming problem is formulated and updated using the current state measurements and considering the optimiza-tion problem from the current time only. As in other fl atness-based approaches, the dynamic optimization problem is transformed to a non-linear optimization problem and solved using an appropriate non-linear optimization code. The resulting sequential approach allows one to respond in real-time to the effect of uncertainties and disturbances.

When the control system is not differentially fl at, alternative parametrization techniques are required to solve the optimiza-tion problem. The most natural method would consist in the parameterization of the input trajectory subject to a knowledge of the initial state measurements. In this work, this approach will be considered to develop an adaptive gradient-based methods for the solution of dynamic optimization problems in real-time. The key to the real-time optimization routine is to view the optimization task as an adaptive mechanism that allows the optimization to be performed at the same time-scale as the control system dynamics. This approach provides a systematic way to incorporate changes in processing conditions and real-time adjustments of the optimization task.

The paper is organized as follows. In the section On-Line Dynamic Optimization of Non-Linear Control Systems, the new method real-time optimization method is proposed and developed. Simulation results are presented in the Simulations section. The solution of benchmark problems that have appeared in the litera-ture has been emphasized. This is followed by a brief statement of the conclusions of this study and the main directions of future research in the last section.

ON-LINE DYNAMIC OPTIMIZATION OF NON-LINEAR CONTROL SYSTEMSIn this paper, we consider a general class of non-linear dynamical systems of the form:

!x f x u= ( , ) (1)

where x ∈ Rn are the state variables, u(t) = [u1(t), … , up(t)]T

∈ Rp is the vector of p input variables, f(x, u): Rn × Rp → Rn is a smooth vector fi eld on Rn. The input vector u(t) is assumed to belong to a set of admissible input trajectories U in the space of functions from the compact interval [0, T ] of R to Rp where T is the fi nal time of the batch process.

The optimization problem is to fi nd a system trajectory that minimizes some meaningful cost functional over the fi nite-time horizon [0, T ]. The dynamic optimization problem is stated as follows:

min ( , ) ,( )u t

TJ x u q x t u t dt= ( ) ( )( )∫0

(2)

subject to:

!x f x uw x t u t

= ( )( ) ( )( ) ≥

,, 0

(3)


x x0 0( ) = (4)

x T xf( ) = (5)

Assumption 1There exists a control, u(t) ∈ U, that can steer the states from x0 to xf over the batch interval t ∈ [0, T ].In this work, the input trajectories

u t u t u tp( ) = ( ) ( ) 1 … (6)

are parameterized as follows

u t ti i i( ) = ( )α θ , (7)

where θi = [θi1, … , θiN] for i = 1, … , p. It is assumed that the functions αi are continuously differentiable with respect to θi uniformly in t. Such functions could include polynomials, neural networks, wavelets, splines, etc. For the purpose of this paper, we focus on linearly parameterized input functions given by:

u t ti ij jj

n

( ) = ( )=∑θ Ξ

1 (8)

where Ξj are basis functions and θij for j = 1, … , N are the parameters to be determined. The state space equations can be rewritten in terms of θ and the initial conditions. If the input is defi ned as a polynomial then:

u tiT= ( )θ φ (9)

where the parameters and basis functions are expressed as follows

θ θ θ= [ ]1 # N (10)

φ t t tN( ) =

1 # . (11)

Remark 1It is important to note that the theory outlined in this paper is not restricted to linearly parameterized input trajectories. Any contin-uously differentiable time-varying function of the parameters can be treated. Typical parameterizations may include piecewise continuous inputs where both the parameters of the piecewise functions and the switching times are updated.

Once a specifi c parameterization of the input trajectories has been chosen, one can use the system dynamics (1) to compute the corresponding state trajectories subject to given initial conditions. The original dynamic optimization problem is re-written in parameterized form as follows:

min ,θ

θ φJ q x t dtTT= ( )( )∫0

(12)

subject to

(13)!x f x t

w x t t

T

T

= ( )( )( ) ( )( ) ≥

,

,

θ φ

θ φ 0

x xx T xf

0 0( ) =( ) =

(14)

The parameterization of the input trajectories transforms the dynamic optimization problem becomes a non-linear optimiza-tion problem. In general, the solution of such problems is very diffi cult and specialized techniques are needed. The objective of this paper is to develop a real-time optimization technique that allows one to compute optimal parameters and compute inputs that can steer the input variables and state variables trajectories along the optimal trajectories.

Remark 2In this context, the optimality is considered only with respect to the parameterized form of the dynamic optimization problem. As in any other situation where fi nite dimensional parameteriza-tions are used to approximate infi nite dimensional functionals, one cannot ensure that the chosen input parameterization approximates the true (infi nite dimensional) optimal input variables and state variables trajectories. This particularity is shared by all numerical optimization algorithms that are used to solve dynamic optimization problems.

We make the following assumptions for the parameterized optimization problem. In order to emphasize the dependence of the solutions of system (1), we let ξ(t, θ, x(t0)) represent the solution of system (1) at time t for an input defi ned by the parameters θ starting from x(t0).

Assumption 2The constraint set

ΩcTw t x t t= ∈ ( )( ) ( )( ) ≥ θ ξ θ θ φRm , , ,0 0 (15)

describes a convex subset of Rm. It is assumed that the parameters evolve on the compact convex subset ϒ = θ ∈ Rm | ||θ|| ≤ wm for some strictly positive constant wm > 0 such that ϒ ∩ Ωc ≠ 0/.

The cost functional J : ϒ → R is assumed to be continuously differentiable on ϒ. The gradient of Jip with respect to θ is assumed to be Lipschitz continuous on ϒ. The cost functional is assumed to locally convex on the set ϒ. The integrand of the cost functional q(ξ(t, θ, x(t0)), θ

T φ(t)) is assumed to be a suffi ciently smooth function of θ.

Assumption 2 indicates that the cost function is locally convex with respect to the parameters θ and that the optimiza-tion problem admits a local optimum over the parameter set ϒ. Furthermore, it assumes that the constraints are convex over ϒ. As in Guay and Zhang (2003), this assumption allows one to use an interior point method incorporating a log barrier function enforces the state and input constraints. In this study, the path constraints (14) are incorporated by introducing a log barrier function in the integrand of the cost function as follows

L t x t t q t x t t

w t

T T

i i

ξ θ θ φ ξ θ θ φ

µ ξ θ

, , , , , ,

, ,

0 0( )( ) ( )( ) = ( )( ) ( )( )− log xx t tT

ii

p

01

( )( ) ( )( ) +( )=∑ ,θ φ ε

(16)

The initial and terminal time constraints (4)–(5) are incorpora-ted by introducing a terminal penalty function in the unconstrai-ned cost functional with integrand (16) as follows:

J L t x t d M T x xipT

fT

= ( )( ) ( )( ) + ( )( ) −( )∫ ξ θ θ φ τ ξ θ, , , , ,0 02

0 (17)

where µi > 0 , εi > 0 and M > 0 are the tuning parameters of the cost functional, with µ and ε being taken as small as possible, and M taken as large as possible. It is assumed that ξ(0) = x0.


Using the redefi ned cost (17), the parameterized optimization problem becomes:

min ,θ

τ θ φ τ τJ L x d M x T xipT

fT

= ( ) ( )( ) + ( ) −( )∫2

0 (18)

subject to:

(19)!x f x t

x x

T= ( )( )( ) =

,θ φ0 0

Using standard argument from convex programming, it follows from Assumption 2 that the solution of the parameterized problem (18) approaches the solution of the constrained problem (14) as µi → 0 and as M → ∞. Since it is not possible in practice to choose M infi nitely large, the addition of penalty functions can only serve to approximate the optimization problem.

Remark 3At this point, it is important to dissociate the model predictions, x(t), from the value of the state variables of the batch process, x(t). At any given time t, the value of the current state x(t) represents the measured state variables for the batch process. In the technique outlined below, the value of the state variable x(t) provides the initial conditions for the predicted state trajectories given as a function of the input parameters by solving the ordinary differential equations

! ˆ,x f x T= ( )( )θ φ τ (20)with x(t) = x(t) and τ ∈ [t, T].

As stated above, the parameterized optimization problem (18) refl ects only the initial state measurements x(0). Assuming that state measurements x(t) are available throughout the batch, one restates the optimization problem taking into account both the path leading to the current state x(τ) (τ ∈ [0, t]) and the model prediction associate with the input parameters x(τ) (τ ∈ [t, T ]). The optimization problem is stated in terms of the modifi ed cost given by:

J L x u d L x d M x T xipm m f T

t

Ttf= ( ) ( )( ) + ( ) ( )( ) + ( ) −( )∫∫ τ τ τ τ θ φ τ τ, ,

0

2 (21)

where the fi rst integral represents the actual cost being calculated from the measured states, and the second integral is the predicted cost remaining using the current parameters. The measured cost can be thought of as another state of the system, z, such that

z t L x u dmt( ) = ( ) ( )( )∫ τ τ τ,

0 (22)

and!z L x t u tm= ( ) ( )( ), (23)

The modifi ed cost is then written as

J z t L x d M x T xipf T

ft

T= ( ) + ( ) ( )( ) + ( ) −( )∫ ˆ , ˆτ θ φ τ τ

2 (24)

The optimization problem becomes,

min ,θ

τ θ φ τ τJ z t L x d M x T xipf T

ft

T= ( ) + ( ) ( )( ) + ( ) −( )∫

2 (25)

subject to: (26)

! ˆ,ˆ

,

x f x

x t x t

z t L x u d

T

mt

= ( )( )( ) = ( )( ) = ( ) ( )( )∫

θ φ τ

τ τ τ0

In this study, we propose a Lyapunov-based approach as proposed in (Guay and Zhang, 2003) to solve the optimization problem (25).

We begin by proposing the following Lyapunov function candidate:

V Jip= (27)

That is, we consider the augmented unconstrained cost functional as a Lyapunov function for the closed-loop system. Its time derivative is given by:! !J J L Lip ip

mt

ft= ∇ + −θ θ (28)

where

∇ = ∂∂

∂∂

+∂∂

∂∂

+ ( ) −( ) ∂ (∫0 2J

Lx

x L

uu

d M x T xx T

ip

ff

t

Tfˆ

ˆˆ

ˆ

θ θτ

))∂θ

. (29)

The fi rst order sensitivities, ∂∂xθ

, are computed as the solution of

the following initial value problem:

ˆ ˆ,!x f x T= ( )( )θ φ τ (30)

ddt

x fx

x fu

u∂∂

= ∂∂

∂∂

+ ∂∂

∂∂

ˆˆ

ˆ

θ θ θ (31)

with initial conditions x(t) = x(t) and ∂∂xθ (0) = 0n×N. By

construction, it follows that Lm|t = Lf|t and hence, the time derivative of V reduces to:

! !J Jip ip= ∇ θ θ (32)

Following a standard Lyapunov stability analysis argument, one must choose the indeterminate ˙ θ such that V is strictly decreas-ing. A logical choice for the parameter update law is as follows:

!θ θ= − ∇k JipTΓ (33)

where Γ is any positive defi nite matrix. In general, the choice of Γ depending on the type of information available. If one applies Newton’s method for example, Γ should be chosen as the Hessian of Jip with respect to θ. For a steepest descent method, one chooses Γ = I. Then the fi nal form of the cost function is

!J k J Jip ipT

ip= − ∇ ∇θ θΓ (34)

The cost is strictly decreasing. By Lasalle’s invariance principle, it follows that the parameter estimates will converge to the set Ωθ = θ ∈ ϒ | ∇ θJip(θ) which is the set of local minimizers on ϒ. It follows that, by construction, any parameter path starting in ϒ asymptotically converges to a local minimizer.

Let θ∗ by a local minimizer of the optimization problem (25). Since, by Assumption 2, it follows that there exists a constant L such that the following inequality holds:

∇ ≤ −θ θ θJ Lip *

for all θ ∈ ϒ. As a result, the gradient of Jip is a bounded vector-valued function of θ on ϒ. However, there is no guarantee that the update law (33) can produce paths that lie entirely in ϒ. Such paths may lead to the divergence of the parameter update law where boundedness of the ||∇ θJip|| cannot be guaranteed.

^ ^

^ ^


The choice of input parameterization is extremely important. In most applications that have been considered, polynomial input parameterizations have been favoured. This choice is primarily motivated by the required smoothness of the input trajectories. As indicated above, any piecewise continuous function can be used with the proposed approach.

One of the main requirements of the technique is that the initial trajectories are feasible. That is, the initial parameters are chosen such that the input and state trajectories meet state constraints. This requirement can constitute a challenge in the application of the technique where feasible input trajectories are diffi cult to obtain. For existing or pilot scale processes, it is usually possible to obtain a suitable feasible parameterization. In other cases, it may be necessary to solve the optimization numerically off-line to obtain suitable starting values. The most effi cient way to achieve this is to take the fi rst set of parameter values leading to the feasible solution from a numerical optimi-zation technique. Otherwise, some trial and error may be required to obtain a feasible parameterization. When using polynomials, one simple approach is to fi nd low order (zeroth, fi rst or second order) feasible trajectories, setting all higher order terms to zero.

The value of optimization parameters µi, εi and M are chosen such that the constraints are met. As in any other method based on penalty methods, we choose a value of M that is large enough to ensure that the penalty functions vanish as a result of the optimization. In most applications that we have encountered, a value of M that is 2 or 3 orders of magnitude larger than the typical value of the cost function suffi ces to achieve the desired effect. The values of µi and εi are chosen to be suffi ciently small positive constant. Ideally, the value µi should be made to vanish at the optimum value of the parameters. This can be achieved by posing µi

as a monotonically decreasing function of time. It remains diffi cult to tune the value µi appropriately since, in this application, the measurement-based optimization is susceptible to sudden and unexpected changes. For this reason, we recommend the use of constant values for µi. Further develop-ments are required to alleviate these diffi culties.

The value of the optimization gain k is adjusted to vary the speed of convergence of the optimization scheme. In general, a larger value of k will make the optimization routine very sensitive to measurement noise and model uncertainties. As a result, the tuning of k should be done with care. In the examples given in the next section, the impact of measurement noise on the optimization routine is considered.

In the following, we document the application of the technique proposed above to two examples that have been studied in the literature. In each case, we provide a detailed account of the application of the technique in a practical situation.

SIMULATIONS

Bioreactor Simulation

Problem defi nitionThe fi rst optimization problem deals with the fed-batch bioreac-tor discussed in Mahadevan et al. (2001). The state space equations modelling the reaction are given as follows:

To avoid divergence of the update law, a projection algorithm is introduced to ensure the parameters remain in the convex set ϒ. The properties of the projection algorithm are discussed in (Krstic et al. 1995) and is given below

!θ θ

θ

θ θθ

θ

= − ∇( ) =

− ∇ <

< ∇ ( ) − ∇Proj k J

k J w

w and P k JipT

ipT

n

n i,

,

Γ

Γ

Γ

if or

0 ppT( ) ≤( )

0

ψ, otherwise

where ψθ θ

θθ θ θθ= − −

∇ ( ) ∇ ( )∇ ( )

∇ ( ) = − ≤k IP P

PJ P w

T

ipT T

m2 0Γ , , θ is the vector

of parameter estimates and ωm is chosen such that ||θ||≤ ωm.Geometrically, we see that in the situation where a parameter

value lies on the boundary of the set ϒ and the update law identi-fi es a direction that points to the outside of ϒ, the projection algorithm corrects the update law by projecting it to the space tangential to the boundary of the set ϒ.

By the properties of the projection algorithm, it follows that 1) the right-hand side of Equation (34) is Lipschitz continuous on ϒ, 2) the parameter functions remain in ϒ and 3) the function Jip is nonincreasing for all θ ∈ ϒ. The demonstration of these three basic properties is standard. The reader is referred to Krstic et al. (1995) for further details.

Remark 4Thus as the cost functional is nonincreasing, the cost functional Jip reaches a local optimum and the update law Equation (34) converges to a local optimizer θ∗ . This results in a fi nal cost Jip(T) given by:

J T L x t u t dt M x T xipm

fT

( ) = ( ) ( )( ) + ( ) −( )∫ ,2

0

that approaches the local minimum, but in terms of the measured process variables x and u.

Thus, in a case where, a local minimizer is identifi ed at an early stage of the batch, the routine ensures that the correspond-ing locally optimal operating strategy is achieved. However, in the presence of unforeseen disturbances, the real-time optimiza-tion scheme operates as a feedback controller which adjusts the remaining trajectory to achieve a local optimum. One of the contributions of this article is to demonstrate that the problem of real-time dynamic optimization can be treated satisfactorily using an adaptive gradient optimization technique, inspired by non-linear adaptive control design techniques.

DESIGN CONSIDERATIONSThe main advantage of the proposed technique is that it requires a minimal amount of pre-optimization analysis compared to other techniques. There is no explicit requirement for fl atness or the knowledge of a particular optimal structure. However, some tuning parameters and other computational requirements must be considered.

First of all, the technique requires, at every control move, the computation of the gradient term ∇ θJip. In most applications, the value of the derivative of the integrand of the cost function are calculated symbolically using, for example, Maple©. The sensitiv-ity of the state predictions with respect to the parameters requires the numerical solution of the initial value problem Equations (30)–(31). The numerical solution of such systems can be computed effi ciently using such packages as ODESSA (Leis and Kramer, 1988).


!

!

!

xu x xK x

uxx

xu S x

xu x x

Y K x

x u

m

m

f m

xs m

11 2

2

1

3

22

3

1 2

2

3

=+

−

=−( )

−+( )

=

where x1 and x2 are the concentrations of the biomass and substrate respectively; x3 is the volume, and u is the feed rate. The optimization scheme is to maximize the amount of biomass formed at the end of the batch. The optimization problem is described as follows:

minθ

µJ

x xK x

uxx

dt

u

x

u

m

m

T= −

+−

≥

≤

− ≥

∫ 1 2

2

1

30

1

3

1

0

10

10 0

The system parameters are summarized in Table 1.

Table 1. Parameters and initial conditions

Parameters Values

Sf 15g/L

Km 1.2g/L

Yxs 0.4g/g

µm 0.5/h

T 7.8 h

Initial conditions Values

x10 1g/L

x20 0g/L

x30 2L

Algorithm parameters Values

µ1 1E−20

ε 1E−8

k 10

ParameterizationIn (Mahadevan et al., 2001), the input profi le was parameterized by subdividing the batch into a number of intervals and representing each interval by a fi fth order polynomial. The optimization was modifi ed to include the constraint that the volume must reach its maximum value by the end of the batch. The initial guess of the parameters was taken from the solution of a highly constrained problem. The optimization was solved using non-linear programming. In this paper, a simple fi fth order polynomial profi le is selected for the input, with the initial parameters set to zero. The goal here is to show that the optimi-zation can be performed with reasonable computing time, and no off-line analysis of the feasible profi le or initial guesses. The input was defi ned as follows:

u tt

i

i

i

( ) =

−

=∑θ

7 8

1

1

6

..

Modifi ed costHaving selected the parameterization, the next step is to construct the modifi ed cost. The volume constraint is incorpo-rated as a terminal cost, while the input constraints are implemented with log barrier functions. The modifi ed cost function is described below

minlogθ εJ

u x xK x

uxx

u um

m= −

+

− +( )(1 2

2

1

31∫

T

0

log εu d+ − +( ))110 ττ + ( ) −( )x T32

10 .

The gradient was computed as discussed in the theory, the details are not included here. The next section will discuss the tuning and algorithm issues. The algorithm parameters for this simulation are in Table 1.

Algorithm and computing issuesComputing time is always an issue when an ODE solver is needed to determine the prediction and sensitivities. The Fortran package ODESSA was used to calculate the model prediction and the fi rst order sensitivities. MATLAB was used to perform the simulation of the closed-loop system.

The run time for this 7.8 h simulation was approximately 10 s, using a 1.6 GHz Intel Processor, using cost gradient as the update law.

ResultsThe dynamic optimization technique was applied to the nominal case presented above. The resulting simulation of the state and input variables profi les can be found in Figures 1 and 2. As in Mahadevan et al. (2001), the optimal fi nal biomass concentra-tion obtained was 4.8 g/L. The technique performs as expected without the need for complex parameterizations and the require-ment for partial fl atness.

Figure 1. Bioreactor varying growth rates state profi les


Figure 4. Bioreactor varying initial conditions input profi le

Finally the initial batch conditions were changed, including initial parameter values and initial conditions, and the optimiza-tion routine was applied again. Table 2 shows the initial and fi nal cost for several different initial conditions.

The results of the simulation are shown in Figures 3 and 4. For varying initial conditions, a local minimization was still achieved.

The results demonstrates the potentially dramatic effect of the changes in initial conditions on the achievable performance. It also confi rms the ability of the prescribed technique to adapt to various conditions.

Table 2. Cost summary

Case x10 x20 x30 Initial cost Final cost (g/L) (g/L) (L)

Nominal 1 0 2 63 −4.8

G.R.1 1 0 2 63 −2.5

G.R.2 1 0 2 63 −2.5

I.C.1 2 0.5 4 33.8 −4.5

I.C.2 0.5 0.7 0.5 89.7 −0.8

I.C.3 4 0.2 7 4.9 −4.7

The method used here provided results comparable to those used in Mahadevan et al. (2001). However, no initial understanding of the optimal structure was needed. A simpler structure was used with all the initial parameters set to zero. Less analysis was needed before running the batch, and the overall algorithm was simpler. This technique can be applied to a variety of problems, with minimal pre-batch analysis. Flatness was not required of the system, and the only restriction was that the cost function was at least once continuously differentiable. Consequently, this method is fl exible for a variety of batch systems. New optimal profi les were found for a variety of initial conditions, and robust-ness under parameter uncertainty was demonstrated.

Figure 2. Bioreactor varying growth rates input profi le

To show the robustness under uncertainty, the growth rate of the actual plant was reduced by half, while the modelled growth rate was unchanged (G.R. = 2). This was compared to the case where the new growth rate is known (G.R. = 1). The resulting profi les can be found in Figures 3 and 4. The new input profi le is modifi ed from the nominal case, as the measure-ment feedback detects the changing dynamics. The new input profi le does not approach the true optimal profi le, but the resulting cost is very similar.

Figure 3. Bioreactor varying initial conditions state profi les


Diketene Example

Problem defi nitionThe next example that was investigated was a time optimal system on the production of 2-acetoacetyle pyrrole. This was originally optimized in Loeblein et al. (1999). The process is a semi-batch reactor where pyrrole, diketene and pyridine (catalyst) are reacted to produce 2-acetoacetyle pyrrole and dehydroacetic acid. The following equations describe a simple model of the system.

!x kx

x xkx

x k xk

xx x

ux

c x

a Do

f

df

15

1 25

12

15

1 3

51

2= − − − −

+ −( )

!

!

!

xkx

x xux

x

xkx

x xk

xx x

ux

x

xkx

x

a

a f

d

25

1 25

2

35

1 25

1 35

3

45

12

= − −

= − −

= − uux

x

x u

54

5! =

where x1, x2, x3, x4 represent the concentrations of diketene, pyrrole, 2-acetoacetyl pyrrole, and dehydroacetic acid respec-tively. The volume of the reactor is x5, and the input u is the fl ow rate to the tank. The constants are given in Table 3.

Table 3. Parameters and initial conditions

Parameters Values

ka 0.0531/(mol min)

kd 0.1281/(mol min)

ko 0.0281/(mol min)

kf 0.0031/(mol min)

cdf 5.82 mol/L T 150 min

Initial conditions Values

x10 0.09 mol/L

x20 0.72 mol/L

x30 0.1 mol/L

x40 0.02 mol/L

x50 1.0L

Algorithm parameters Values

µ 0.01 M1 1E7 M2 1E11 ε 1E−5

k1 1E6

The optimization problem is a batch time minimization with the following constraints:

x T x T

x t

x T

u t

3 5

4

1

0 42

0 15

0 025

0

( ) ( ) ≥

( ) ≤

( ) ≤

( ) ≥

.

.

.

.

ParametrizationIn Loeblein et al. (1999), the input was parameterized as a piecewise constant function with eight sections. In this paper a fi fth-order polynomial was used for the input trajectory.

u tt

ii

i

( ) =

=

−

∑θ150

1

6 1

Modifi ed costThe modifi ed cost is defi ned incorporating the constraints as follows. The parameters are in Table 3.

θ µ ε εmin log logJ T u x= + − +( ) + − +( )1 30 15.((∫0

T

ε ε1 2log x log x log+ +( ) + +( ) + xx3 +( )ε

log x4+ +( )))ε τd

+ −( ) + −( )1 3 52

2 120 42 0 025M x x M x. .

The gradient and hessian are calculated as discussed above. An additional derivative is needed with respect to the batch time which is being minimized.

Algorithm and computing issuesAs in the previous example, ODESSA was used for the calcula-tion of the predictions and fi rst order sensitivities. The batch time is in the order of 140 min, while the simulation took 4 min and 22 s.

ResultsThe real-time dynamic optimization routine was applied to the nominal optimization problem stated above. The profi les of the constrained states are shown in Figure 5. The remaining state and input profi les are given in Figures 6 and 7. For the specifi c choice of input parameterization and nominal conditions, the minimum batch time was evaluated at 137.9 min. The results were very similar to those given in Loeblein et al. (1999).

Figure 5. Diketene constrained state profi les


Figure 6. Diketene unconstrained state profi les

Figure 7. Diketene input profi le

Some variations were found when batches were repeated for varying parameters values or initial estimates for the batch time. The results were comparable, although the batch time could vary by a few minutes. Table 4 shows the different batch times that were determined. Despite the varying batch times, the profi les were relatively similar. Plots of these profi les can be found in Figures 8, 9 and 10.

Table 4. Batch time

Initial batch time Final batch time (min) (min)

160 144

144 142

142 140

140 139

Figure 8. Diketene constrained state profi les (multiple runs)

Figure 9. Diketene unconstrained state profi les (multiple runs)

Figure 10. Diketene input profi le (multiple runs)


CONCLUSION AND FUTURE WORKIn this paper, we proposed a new on-line optimizing controller for non-linear dynamical systems. Smooth trajectories were generated on-line with feasible computing time to construct optimal trajectories without the need for off-line analysis.

The computational requirements for the technique proposed remains minimal as each control move requires of a unique computation of the state prediction and sensitivity equations. This provides a considerable gain over existing optimization based techniques where each control move can require many iterations from a numerical optimization routine. For more complex systems, the limiting factor remains the computing time the state predictions and sensitivity equations. The use of numerical integration tools such as ODESSA allows one to minimize the computing required in complex systems. In any application, however, there will be cases where the computing time required to calculate a control update exceeds the length of the integration step or the sampling time. Future work will be focused on the minimization of computing time and the application of approxi-mation techniques to reduce the complexity of the system.

The current work was based on the assumption that the state variables were measured and that the model of the system was suffi ciently accurate. The focus of future research will be on the application of the technique for batch processes with imperfect state measurements, parametric uncertainties and plant-model mismatch.

REFERENCESArpornwichanop, A., P. Kittisupakorn and I. M. Mujtaba,

“On-Line Dynamic Optimization and Control Strategy for Improving the Performance of Batch Reactors,” Chem. Eng. Proc. 44(1), 101–114 (2005).

Bellman, R. E., “Dynamic Programming,” Princeton University Press, Princeton (1957).

Cuthrell, J. E. and L. T. Biegler, “On the Optimization of Differential Algebraic Process Systems,” AIChE J. 33(8), 1257–1270 (1987).

DeHaan, D. and M. Guay, “A Real-Time Framework for Model Predictive Control of Continuous-Time Non-Linear Systems,” in “Proc. IEEE CDC ECC 05,” Seville, Spain (2005).

Delaleau, E. and J. Rudolph, “Control of Flat Systems by Quasi-Static Feedback of Generalized States,” International Journal of Control 71(5), 745–765 (1998).

Faiz, N., S. K. Agrawal and R. M. Murray, “Differential Flat Systems with Inequality Constraints: An Approach to Real-Time Feasible Trajectory Generation,” AIAA J. Guid. Nav. Control Dynamics 24(2), 219-227 (2001).

Fontes, F. , “A General Framework to Design Stabilizing Non-Linear Model Predictive Controllers,” Syst. Contr. Lett. 42(2), 127–143 (2000).

Fournier, F., M. A. Latifi and G. Valentin, “Methodology of Dynamic Optimization and Optimal Control of Batch Electrochemical Reactors,” Chem. Eng. Sci. 54(13–14), 2707–2714 (1999).

Gattu, G. and E. Zafi riou, “A Methodology for On-Line Setpoint Modifi cation for Batch Reactor Control in the Presence of Modeling Error,” Chem. Eng. J. 75(1), 21–29 (1999).

Guay, M. and T. Zhang, “Adaptive Extremum Seeking Control of Nonlinear Dynamic Systems with Parametric Uncertainties,” Automatica 39(7), 1283–1293 (2003).

Guay, M., S. Kansal and J. F. Forbes, “Trajectory Optimization for Flat Dynamical Systems,” Ind. Eng. Chem. Res. 40(9), 2089–2102 (2001).

Hestenes, M. R., “The Calculus of Variations and Optimal Control Theory,” Wiley, NY (1966).

Jadbabaie, A., J. Yu and J. Hauser, “Unconstrained Receding-Horizon Control of Nonlinear Systems,” IEEE Trans. Autom. Contr. 46(5), 776–782 (2001).

Krstic, M., I. Kanellakopoulos and P. Kokotovic, “Nonlinear and Adaptive Control Design,” John Wiley and Sons (1995).

Leis, J. R. and M. A. Kramer, “Algorithm 658 Odessa—An Ordinary Differential Equation Solver with Explicit Simultaneous Sensitivity Analysis,” ACM Trans. Math. Software 14(1), 61–67 (1988).

Loeblein, C., J. D. Perkins, B. Srinivasan and D. Bonvin, “Economic Performance Analysis in the Design of On-Line Batch Optimization Systems,” J. Proc. Control 9(1), 61–68 (1999).

Magni, L. and R. Scattolini, “Model Predictive Control of Continuous-Time Nonlinear Systems with Piecewise Constant Control,” IEEE Trans. Autom. Contr. 49(6), 900–906 (2004).

Mahadevan, R., S. K. Agrawal and F. J. Doyle III, “Differential Flatness Based Nonlinear Predictive Control of Fed-Batch Bioreactors,” Control Engineering Practice 9, 889–899 (2001).

Mahadevan, R., S. K. Agrawal and F. J. Doyle III, “A Flatness-Based Approach to Optimization in Fed-Batch Reactors,” in Proc. IFAC-ADCHEM 2000, Pisa, Italy (2000), pp. 111–116.

Nagy, Z. and R. Braatz, “Robust Nonlinear Model Predictive Control of Batch Processes,” AIChE J. 49(7), 1776–1786 (2003).

Noda, M., T. Chida, S. Hasebe and I. Hashimoto, “On-Line Optimization System of Pilot Scale Multi-Effect Batch Distillation System,” Comput. Chem. Eng. 24(2), 1577–1583 (2000).

Oldenburg, J. and W. Marquardt, “Dynamic Optimization Based on Higher Order Differential Model Representations,” Comput. Chem. Eng. 26(3), 385–400 (2002).

Palanki, S. and A. K. M. S. Rahman, “State Feedback Synthesis for On-Line Optimization of Batch Reactors with a Non-Affi ne Manipulated Input and Free Terminal Time,” in Proc American Control Conf., Vol. 3. (1994), pp. 3076–3079.

Palanki, S., C. Kravaris and H. Y. Wang, “Synthesis of Stated Feedback Laws for End-Point Optimization in Batch Processes,” Chem. Eng. Sci. 48(1), 135–151 (1993)

Rahman, A. K. M. S. and S. Palanki, “On-Line Optimization of Batch Processes with Nonlinear Manipulated Input,” Chem. Eng. Sci. 51(3), 449–459 (1996).

Rahman, A. K. M. S. and S. Palanki, “State Feedback Synthesis for On-Line Optimization of Batch Reactors with Multiple Manipulated Inputs,” Comp. Chem. Eng. 22(10), 1429-1439 (1998).

Rouchon, P., M. Fliess, J. Lévine and P. Martin, “Flatness and Defect of Non-Linear Systems: Introductory Theory and Examples,” Int. J. Control 61(6), 1327–1361 (1995).

Valappil, J. and C. Georgakis, “Nonlinear Model Predictive Control of End-Use Properties in Batch Reactors,” AIChE J. 48(9), 2006–2021 (2002).

van Nieuwstadt, M. and R. M. Murray, “Real-Time Trajectory Generation for Differentially Flat Systems,” Int. J. Robust Nonlinear Contr. 8(11), 995–1020 (1998).


Visser, E., B. Srinivasan, S. Palanki and D. Bonvin, “Feedback-Based Implementation Scheme for Batch Process Optimization,” J. Proc. Contr. 10(5), 399–410 (2000).

Zhang, G. P. and S. Rohani, “On-Line Optimal Control of a Seeded Batch Cooling Crystallizer,” Chem. Eng. Sci. 58(9), 1887–1896 (2003).

Zuo, K. and W. T. Wu, “Semi-Realtime Optimization and Control of a Fed-Batch Fermentation System,” Comp. Chem. Eng. 24(2), 1105–1109 (2000).

Manuscript received November 9, 2005; revised manuscript received January 26, 2005; accepted for publication February 16, 2006.

real-time dynamic optimization of non-linear batch systems

Documents