Introduction to Optimization

The maximizing or minimizing of a given function subject to

some type of constraints. Make your processes as effective as

possible.

In a typical chemical plant, there are many control variables for controlling the

process, such as maintaining a temperature, level, or flow.

Part 1: Process Control and

Optimization

Control has to do with

adjusting flow rates to

maintain the controlled

variables of the process at

specified set-points.

Optimization chooses the

values for key set-points

such that the process

operates at the “best”

economic conditions.

What is optimal operation

temperature?

Graphical Solution of Optimum Reactor Temperature, T*

AFACCBBAA VCQVCQVCQVCQ 0

VB > VC, VA, or VAF V is the chemical values.

At low T, little formation of B; At high T, too much of B reacts to

form C; Therefore, the exits an optimum reactor temperature, T*

Economic Objective Function

A B C k1(T) k2(T)

Use of “fmincon” in MATLAB for process optimization

[x fval] = fmincon(fun,x0,A,B,Aeq,Beq,lb,ub,nonlcon,options)

Estimated

parameters

Errors of predicted

and observed

results

Function that you

want to minimize,

i.e. the error

Initial guess of

the parameters

Boundaries of

the parameters

Choose the right solver

A and b describes the linear inequalities

among parameters

Aeq and beq describes the linear equalities

among parameters

ceq, and c describes the nonlinear

equalities/inequalities among parameters

X = fmincon(fun, X0, A, B, Aeq, Beq, lb, ub) defines a set of lower and upper bounds on the design

variables, X, so that a solution is found in the range LB <= X <= UB. Use empty matrices for LB and

UB if no bounds exist.

The function NONLCON accepts X and returns the vectors

C and Ceq, representing the nonlinear inequalities and

equalities respectively.

Example 1

Find values of x that minimize

starting at the point x = [10; 10; 10] and subject to

the constraints

X = fmincon(fun, X0, A, B, Aeq, Beq, lb, ub) minimizes FUN subject to the

linear equalities Aeq*X = Beq as well as A*X <= B. (Set A=[] and B=[] if no

inequalities exist.)

%file name example1

clc;close all;clear all;

int_guess=[10;10;10];% INITIAL GUESS FOR U'S

A=[-1 -2 -2;1 2 2]; % linear inequalities constraints

B=[0 72];

[u]=fmincon(@eg_1,int_guess,A,B) % CALL FOR OPTIMIZER

Use fmincon for optimization

%file name eg_1.m

function [err]=eg_1(u)

x1=u(1);

x2=u(2);

x3=u(3);

err=-x1*x2*x3;

Example 2 (dynamic optimization)

Consider the batch reactor with following reaction

A B C

Find the temperature, at which the product B is maximum

Mathematical Representation of the system is as :

clc;close all;clear all;

warning off

int_guess=300;% INITIAL GUESS FOR U'S

LB=298; % LOWER BOUND OF U

UB=398;% UPPER BOUND ON U

[u,FVAL]=fmincon(@reactor_problem,int_guess,[],[],[],[],LB,UB,[]); %

CALL FOR OPTIMIZER PROBABLY NOT TO CHANGE BE USER

[err]=reactor_problem(u);

load data_for_system

plot(t,Y(:,1),'k')

hold on

plot(t,Y(:,2),'m')

legend('opt c_A','opt c_B')

u

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

opt cA

opt cB

T=335.3244

function [err]=reactor_problem(u)

A=[1 0]; %initial condition

[t,Y]=ode15s(@reactor_ODE,[0 1],A,[],u); %tspan=1

err=-Y(end,2); %make the maximum B

save data_for_system t Y %just for plotting

function [YPRIME]=reactor_ODE(t,x,u)

YPRIME=zeros(2,1);

T=u;

k1=4000*exp(-2500/T);

k2=620000*exp(-5000/T);

YPRIME(1)=-k1*(x(1))^2;

YPRIME(2)=(k1*x(1)^2)-k2*x(2);

Maximize B

ODE calculation

Call ODE to solve the model

ode45 (@fun, slot, init, [opt], [par1, par2, ...])

Parameter fitting for dynamic model

Part 2: Inverse problem (from experimental

data to model construction)

An example for parameter fitting of

dynamic bioprocess model

A) Nonlinear Parameter Estimation via “fmincon”

in MATLAB

Mathematic

Model

Known

Parameters

Simulated

Results

References (e.g.

literature)

Experimental

Results

v.s.

Mathematic

Model

Unknown

Parameters

Simulated

Results

References (e.g.

literature)

Experimental

Results

v.s.

ε

“Inverse problem” “Forward problem”

“Parameter

estimation”

Nonlinear Parameter Estimation

1. Parameter estimation for linear systems

y = A∙x + b

y, x are the vector of dependent and independent variables,

A, b are parameters to be estimated

“regress” command in MATLAB (for linear model)

2. Parameter estimation for nonlinear systems

y = f (parameters, x)

f is the nonlinear function of the estimated parameters

For example, y=β1+β2∙sin(β3∙ t)

cftool box for more complicated model equations

3. Parameter estimation for dynamic ode models (“fmincon” , “nlinfit”, etc.

in MATLAB)

Example 1: parameter estimation in bioengineering

• Bioreactor model

SK

S

XXYdt

dS

XYdt

dP

Xdt

dX

S

SX

XP

max

/

/

1

Five parameters to be estimated:

YP/X, YX/S, µmax, Ks, and S(0)

Initial conditions:

X(0) = 0.05 g/L; P(0) = 0 g/L;

t X P S 0.0 0.51 0.28 14.86 3.0 0.15 0.47 14.93 6.0 0.42 0.21 14.30 9.0 0.56 0.61 14.00

12.0 0.33 0.12 13.80 15.0 0.09 0.09 12.00 18.0 1.01 0.16 11.11 21.0 1.60 0.34 6.00 24.0 2.62 0.52 3.50 27.1 3.73 1.13 0.70

Experimental data

We will use “fmincon” to find the

parameters

function [error,ypred] = reactor_model(beta)

global yobs

global t

%reactor model

S0=beta(5);

y0=[0.05 0 S0];

tspan=t; %we want y at every t

[t,y]=ode45(@ff,tspan,y0);

function dy = ff(t,y) %function that computes the dydt

umax=beta(1);

Ks=beta(2);

Ypx=beta(3);

Yxs=beta(4);

X=y(1);

P=y(2);

S=y(3);

u=umax.*S./(Ks+S);

dy(1)=u.*X;

dy(2)=Ypx.*u.*X;

dy(3)=-1/Yxs.*u.*X-u.*X;

dy=dy';

end

y1=y(:,1); y2=y(:,2); y3=y(:,3);

ypred=[y1; y2; y3];

error=sum((ypred-yobs).^2);

end

The Error Function

Input the observed data

Settings to run ODEs

The ODEs that describe

bioreactor behaviors

Calculate the error

%Use fmincon to find the parameters in reactor model

clear

clc

data =xlsread('reactor_model_data.xlsx');

t=data(:,1);

X=data(:,2);

P=data(:,3);

S=data(:,4);

x=t;

global yobs

global t

yobs=[X;P;S];

umax=0.1;

Ks=10;

Ypx=0.11;

Yxs=0.45;

S0=14.86;

beta0(1)=umax; %initial guess

beta0(2)=Ks; %initial guess

beta0(3)=Ypx; %initial guess

beta0(4)=Yxs; %initial guess

beta0(5)=S0; %initial guess

The fmincon Function

Set up the initial guess of

parameters

Read data from Excel

%fmincon

fun=@reactor_model;

A=[];

b=[];

Aeq=[];

beq=[];

lb=zeros(5,1);

ub=lb+20;

nonlcon=[];

x0=beta0;%lb+rand.*(ub-lb);

options=optimset('Algorithm','interior-point');

[param, fval] = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,options)

%get the predicted resutls

[error,ypred]=reactor_model(param);

n=length(t);

Xpred=ypred(1:n);

Ppred=ypred(n+1:2*n);

Spred=ypred(2*n+1:3*n);

figure(1)

set(gca, 'fontsize',14,'fontweight','bold');

hold on

plot(x,Xpred,'-r','linewidth',2.5);

plot(x,Ppred,'-g','linewidth',2.5)

plot(x,Spred,'-b','linewidth',2.5)

plot(x,X,'r+','markersize',10);

plot(x,P,'go','markersize',10)

plot(x,S,'bx','markersize',10)

xlabel('time')

ylabel('y')

legend('X g/L','P g/L','S g/L')

The fmincon Function (continued)

Settings to run fmincon

Run the model again to

get the predicted results

Plot the predicted v.s. observed

results

Q: How to find the global solution?

A: Randomly perturb the initial guess

Parameters Estimated via “fmincon”

param =

0.2248 3.8128 0.2606 0.3029 14.9987

fval =

4.3559

µmax = 0.2248 (1/h)

Ks = 3.8128 (g/L)

Ypx = 0.2606 (g/g)

Yxs = 0.3029 (g/g)

S0 = 14.9987 (g/L)

Estimating confidence intervals of parameters via

“fmincon”

Which one makes sense? Numbers in the brackets are the 95% confidence interval of µmax

µmax = 0.2248 [0.1700 0.2797] (1/h)

Or

µmax = 0.2248 [-0.1064 0.2874] (1/h)

µmax cannot be accurately estimated

Estimation of µmax is reliable

How to estimate the confidence intervals of parameters?

• “bootstrap” method: randomly resample the experimental observations; and

for each new set of sampled experimental observations, use fmincon to find a

new set of parameters.

• “Monte Carlo” method: randomly perturb the experimental observations within

the measurement errors (i.e. standard derivation); and for each new set of

perturbed experimental observations, use fmincon to find a new set of parameters.

B: Parameter fitting using nlinfit

Unlike forward problems, inverse problems require experimental data, and an

iterative solution. Because inverse problems require solving the forward

problem numerous time, the ode45 solver will be nested within a nonlinear

regression routine called “nlinfit.”

The syntax is: [param, r, J, COVB, mse] = nlinfit(X, y, fun, beta0);

returns the fitted coefficients param, the residuals r, the Jacobian J of function

fun, the estimated covariance matrix COVB for the fitted coefficients, and an

estimate MSE of the variance of the error term.

X is a matrix of n rows of the independent variable

y is n-by-1 vector of the observed data

fun is a function handle to a separate m-file to a function of this form:

yhat = fun(b,X)

where yhat is an n-by-1 vector of the predicted responses, and b is a vector of

the parameter values. beta0 is the initial guesses of the parameters.

Example 1: Model fitting ODE

equation

0 1 2 3 4 5 6 7 8 9 10-20

0

20

40

60

80

100

120

time (min)

y

ypred

yobs

ykdt

dy

Based on experimental

data, find y0 and k.

data =xlsread('exp_data.xls'); %read data from excel

yo=100; k=0.6;

beta0(1)=yo;

beta0(2)=k;

x=data(:,1); yobs=data(:,2);

[param,resids,J,COVB,mse] = nlinfit(x,yobs,'forderinv',beta0);

rmse=sqrt(mse); %root mean square error = SS/(n-p)

%R is the correlation matrix for the parameters, sigma is the standard error vector

[R,sigma]=corrcov(COVB);

%confidence intervals for parameters

ci=nlparci(param,resids,J);

%computed Cpredicted by solving ode45 once with the estimated parameters

ypred=forderinv(param,x);

%mean of the residuals

meanr=mean(resids);

Redisuals: the residual of an observed value is the difference between the observed value and the

estimated function value.

The Jacobian determinant carries

important information about the local

behavior of F.

The root-mean-square error (RMSE) is a measure of the differences between value (Sample and

population values) predicted by a model or an estimator and the values actually observed.

nlparci : The confidence interval calculation is valid when the length of RESID

exceeds the length of BETA, and J has full column rank. When J is ill-conditioned,

confidence intervals may be inaccurate.

The covariance matrix generalizes the notion of variance to multiple dimensions.

R = corrcov(C) computes the correlation matrix R that corresponds to the covariance matrix C.

[R,sigma]=corrcov(COVB);

figure

hold on

h1(1)=plot(x,ypred,'-','linewidth',3); %predicted y values

h1(2)=plot(x,yobs,'square', 'Markerfacecolor', 'r');

legend(h1,'ypred','yobs')

xlabel('time (min)')

ylabel('y')

%residual scatter plot

figure

hold on

plot(x, resids, 'square','Markerfacecolor', 'b');

YLine = [0 0];

XLine = [0 max(x)];

plot (XLine, YLine,'R'); %plot a straight red line at zero

ylabel('Observed y - Predicted y')

xlabel('time (min)‘)

Function with ode45

function y = forderinv(param,t)

%first-order reaction equation

tspan=t; %we want y at every t

[t,y]=ode45(@ff, tspan, param(1));

%param(1) is y(0)

function dy = ff(t, y) %function that

computes the dydt

dy(1)= -param(2)*y(1);

end

end

0 1 2 3 4 5 6 7 8 9 10-15

-10

-5

0

5

10

15

Observ

ed y

- P

redic

ted y

time (min)

0 1 2 3 4 5 6 7 8 9 10-20

0

20

40

60

80

100

120

time (min)

y

ypred

yobs

One ODE

Time y

0 107.2637

0.10101 102.9394

0.20202 92.71275

0.909091 75.15614

1.010101 70.4741

1.111111 69.83605

1.212121 57.00548

1.313131 60.03961

1.616162 54.01795

1.717172 56.79646

1.818182 53.81866

1.919192 49.67512

2.020202 42.0459

2.121212 40.42633

2.525253 41.06735

2.626263 39.48581

2.727273 34.28254

2.828283 30.3689

2.929293 31.69877

3.232323 25.20435

3.333333 27.20177

3.434343 19.71342

3.535354 26.31708

3.939394 23.20818

4.040404 15.5449

4.343434 19.08923

4.444444 16.08262

4.545455 19.26572

4.646465 23.76179

4.747475 11.85451

4.848485 7.628606

4.949495 7.998508

Time y

5.050505 8.541471

5.454545 12.46344

5.555556 6.944696

5.656566 15.90549

5.757576 5.717727

5.858586 9.637526

5.959596 4.531673

6.060606 5.446719

6.161616 7.203193

6.262626 7.02317

6.666667 16.22408

6.767677 5.286706

6.868687 11.7421

6.969697 -4.34103

7.070707 9.103846

417395

7.575758 9.998128

7.676768 6.737791

7.777778 7.460489

8.181818 4.723043

8.282828 8.394924

8.383838 -0.45654

8.484848 0.910225

8.888889 7.662295

8.989899 -0.04558

9.090909 2.102022

9.191919 1.454657

9.292929 4.797709

9.393939 9.16223

9.494949 -5.94742

9.59596 12.27148

9.69697 5.956534

10 3.329466

Raw data File name: exp_data.xls

/s

Yp/s=Yx/s*Yp/x

0 5 10 15 20 250

5

10

15

time (min)

y

S

X

P

Example 2: Three ODES for batch fermentation

Three ODEs parameter fitting

t X P S

0 0.05 0 15

2.7 0.082555 0.019533 14.86978

5.4 0.136259 0.051755 14.65496

8.1 0.224763 0.104858 14.30095

14 0.667768 0.370661 12.52893

20.5 2.151108 1.260665 6.595568

24.1 3.652464 2.161478 0.590144

File name: HW217.xls

clear all %clear all variables

global y0

data =xlsread('HW217.xls');

%initial conditions

y0=[0.05 0 15];

%initial guess

Umax=0.1; Ks=10; Ypx=0.11; Yxs=0.45;

beta0(1)=Umax; beta0(2)=Ks; beta0(3)=Ypx; beta0(4)=Yxs;

%Measured data

x=data(:,1);

yobsX=data(:,2);

yobsP=data(:,3);

yobsS=data(:,4);

yobs=[yobsX; yobsP; yobsS];

%nlinfit returns parameters, residuals, Jacobian (sensitivity coefficient matrix),

%covariance matrix, and mean square error. ode45 is solved many times

%iteratively

[param,resids,J,COVB,mse] = nlinfit(x, yobs,'forderinv2', beta0);

Fitting four parameters

Script

rmse=sqrt(mse); %root mean square error = SS/(n-p)

n=size(x); nn=n(1);

%confidence intervals for parameters

ci=nlparci(param,resids,J);

%computed Cpredicted by solving ode45 once with the estimated

parameters

ypred=forderinv2(param,x);

ypredX=ypred(1:nn);

ypredP=ypred(nn+1:2*nn);

ypredS=ypred(2*nn+1:3*nn);

%mean of the residuals

meanr=mean(resids);

Script

figure

hold on

plot(x, ypredX, x, ypredP, x, ypredS); %predicted y values

plot(x, yobsX, 'r+', x, yobsP, 'ro', x, yobsS, 'rx');

xlabel('time (min)')

ylabel('y')

%residual scatter plot

x3=[x; x; x];

figure

hold on

plot(x3, resids, 'square', 'Markerfacecolor', 'b');

YLine = [0 0];

XLine = [0 max(x)];

plot (XLine, YLine,'R'); %plot a straight red line at zero

ylabel('Observed y - Predicted y')

xlabel('time (min)‘)

Script

function y = forderinv2(param,t)

%first-order reaction equation

global y0;

tspan=t; %we want y at every t

[t,y]=ode45(@ff,tspan,y0); %param(1) is y(0)

function dy = ff(t,y) %function that computes the dydt

dy(1)= param(1)*y(3)/(param(2)+y(3))*y(1); %biomass

dy(2)= param(1)*param(3)*y(3)/(param(2)+y(3))*y(1); %product

dy(3)= -1/param(4)*dy(1)-1/(param(3)*param(4))*dy(2); %substrate

dy=dy';

end

% after the ode45, rearrange the n-by-3 y matrix into a 3n-by-1 matrix

% and send that back to nlinfit.

y1=y(:,1); y2=y(:,2); y3=y(:,3);

y=[y1; y2; y3];

end

Function and sub-function