engineers (chemical industries) by peter englezos [cap 9]

8/22/2019 Engineers (Chemical Industries) by Peter Englezos [Cap 9]

1/9

Constrained Parameter Estimation

I n some cases besides the governing algebraic or dif ferent ial equations, themathemat ica l model that descr ibes the physical system under investigation is ac-companied with a set of constraints. These are either equality or inequality con-straints that must be satisfied w h e n the parameters converge to their best values.The constraints may be s imply on the parameter values, e.g., a reaction rate con-stant m u s t be posi t ive, or on the response var iables. The latter are often encoun-tered in thermodynamic problems where th e parameters should be such that thecalculated thermophysica l propert ies satisfy all constraints imposed by thermody-namic laws. W e shal l first consider equal i ty constraints and subsequent ly inequal -it y constraints.

9 .1 E Q U A L I T Y C O N S T R A I N T SE qua l i t y constraints are ra ther seldom in parameter es t ima t ion . If there is anequal i ty constraint among the parameters , one should first at tempt to e l iminate oneof the u n k n o w n parameters s imply b y solving expl ic i t ly for one of the parametersand then substi tu t ing that relationship in the m o d e l equations. Such an action re-duces th e d imens iona l i t y o f t he parameter es t imat ion problem w h i c h aids s igni f i -cant ly in achieving convergence.If the equali ty constraint involves independent variables and parameters inan algebraic model , i.e., it is of the fo rm , c p ( x , k) = 0, and if we can solve expl ici t ly

for one of the u n k n o w n parameters, s imple substi tu t ion of the expression into themodel equ at ions reduces th e n u m b e r of u n k n o w n parameters by one.If the equal i ty constraint involves the response variables, then we have theopt ion to ei ther subst i tute the exper imenta l measurement of the respon se var iables

158

C i ht 2001 b T l & F i G LLC


2/9

Constrained Parameter Estimation 1 59

into th e cons t ra in t or to use the error- in-var iab les m etho d to o b ta in be s ide s t heparameters th e noise-f ree value of the response variables . The use of the error-in-variables method is discussed in Chapter 14 (parameter est imation with equat ionso f state). The genera l approach to h a n d l e an y equa l i t y constraint is t h r o u g h the useof Lagrange mul t ip l iers discussed nex t .

9 . 1 . 1 Lagrange Mul t ip l i e r sLet us consider constrained least squares est imation of u n k n o w n parameters

i n algebraic equation model s first. The problem can be formula ted as fol lows:G i v e n a set of data points {(x, ,y, ) , i=l , . . . ,N} and a mathemat ical mode l ofthe form, y = f (x ,k ) , the object ive is to de te rmine th e u n k n o w n parameter vector kby m i n i m i z i n g the least squares object ive function subject to the equali ty c o n -straint, namely

S L S (k ) = [yi - f ( X j , k ) ] T Q \y { -f(xn k)] (91inimizesubject to cp(xo,yo,k) = 0 (92The point whe re the constraint is satisfied, (x 0 ,yo), may or may not belong to

th e data se t { ( x j , y j ) > i = l , . . . , N } . T h e above constrained minimiza t ion problemca n be t rans formed into an unconstrained one by in t roduc ing the Lagrange mult i -pl ier , co an d augment ing th e least squares objective function to form the La-grang ian ,SLG (k,co) = S L S (k ) + tocp(x 0 ,v 0,k ) (93

Th e above uncons t ra ined es t imat ion problem can be solved b y a smal lmodif ication of the Gauss-Newton method . Let us assume that we have an esti-mate k0' of the parameters at th e j th iteration. Linearizat ion of the model equationand the const ra in t aroun d k"' y i e ld s ,

f d f T l T= f C x i . k O ) + ^ J A k ^ + (9.4)and

cp(x0 ,y0 ,k^ 1 )) = q > ( x 0 , y 0 , k ( - ' ) ) + f l A k ^ + 1 ) (95Copyright 2001 by Taylor & Francis Group, LLC
http://dk5985_ch14.pdf/http://dk5985_ch14.pdf/http://dk5985_ch14.pdf/


3/9

160 Chapter 9

Subs t i tu t ion o f the E q u a t i o n s 9.4 and 9 .5 in to the Lagrangian g iven b yE qu a t ion 9 .3 and use o f the s ta t ionary condi t ions

(9.6)c)ku"'an d5SLG(k(l+')'(0) = 0 (97(7(1)

y ie ld th e f o l l ow i ng system of l inear equa t io n sA A k '+ 1 ) = b (9.8a)

andwhere

c T A k ( i + 1 ) = - < p 0 (9.8b)

( 9 - 9 a )b = bGN - c (9.9b)

c p 0 =-cp(x0 , y o , kw) (9.9d)(^ A

c = (9.9e)( d k jEquat ion 9.8a is solved with respect to A k ^ + 1 ) to yield

Subsequen t subs t i tu t ion into E qua t io n 9.8b y ie ld s

c A" bGN - c A ~ c = -(p0 ( 9 - 1 1 )

Copyright 2001 by Taylor & Francis Group, LLC


4/9

Constrained Param eter Estimation 161

which upon rearrangement resul ts in

oo = 2 - ^r- (9.12)

subs t i tu t ing th e above exp re ss io n for the Lagrange m u l t i p l i e r into Equa t ion 9.8awe arrive at the fo l l o win g l inear equat ion for Ak(H),

A A k U + 1 ) = b ' p p + c1 A 'bGN '

GN (9.13)

Th e above equat ion leads to the f o l lowing steps of the m o di f i ed Gauss -N e w t o n method .

Modified Gauss-Newton Algorithm for Constrained Least SquaresT h e imp lemen ta t ion o f t he m o d i f i e d G a u s s -N e w t o n m e t h o d i s acco m pl i shedb y fo l lowing th e steps given be low,Step 1 . Generate /assume an init ial guess for the parameter vector k .Step 2. G i v e n th e current est imate of the parameters , k ci ), c o m p u t e theparameter sensi t ivi ty matr ix, G ;, the response variables f (x j ,k) ,and the constraint, c p 0 .Step 3. Set up matr ix A , vector b G N and comp ute vector c.Step 4 . Per form an e igenvalue decomposi t ion of A = V TA V and c o m p u t eA - ' = V T A - ' V .Step 5. C omp u t e the r ight hand side of Equat ion 9 . 1 3 and then solveEquat ion 9.13 with respect to A k ^ " ! ) .Step 6. Use the bisect ion ru le to de te rmine an acceptable step-size andt hen update the parameter est imates.Step 7. Ch eck for convergen ce. I f converged, est imate CO K(k *) and stop;else go back to Step 2.The above constrained parameter estimation problem becomes m u c h m o r echal leng ing i f the location w h e r e th e const ra in t m u s t b e sat is f ied, (x 0 ,y 0 ) , i s no tk n o w n a priori. T h i s s i tua t ion arises natural ly in the es t imat ion o f binary in terac-t ion parameters in cubic equat ions of state ( s e e Chap te r 1 4 ) . Furthermore , th e

above development can be readi ly extended to several constraints by in t roduc ingan equal num ber o f Lagrange mul t ip l ie r s .



5/9

162 Chapter 9

If th e mat hemat i ca l mode l i s descr ibed by a set of di f f e ren t ia l equat ions ,cons t ra ined param e te r es t imat ion b e c o m e s a fa ir ly compl ica t ed p r o b l e m . N o r m a l lywe may have two di f feren t types of constraints on the state variables: (i) isoperi-m etric constraints , expressed as an integral (or s u m m a t i o n ) constraint on the statevariables, and/or ( i i ) simple constraints on the state variables. In the first case, w egenerate the Langrangian by in t roducing the Lagrange mult ip l ier co which is ob-tained in a s imilar fashion as the descr ibed p rev io us l y . If however , w e have a s im-p le constraint on the state variables, the Lagrange mul t ip l ier co , becomes an un-k n o w n functional , co(t), that must be de te rmined. This problem can be tackled us-ing e leme nts f rom the calculus of variat ions but it is beyond the scope of th is bookand it is not considered here .

9 .2 I N E Q U A L I T Y C O N S T R A I N T SThere are two t ypes of inequal i ty constraints . Those that i nvo lve only th eparameters (e.g., the parameters m u s t be pos i t ive and less than one ) and those thatinvolve not only the parameters bu t also the dependent variables (e.g., th e pre-dicted concentrat ions of al l species must always be posi t ive or zero and the un-k n o w n reaction rate constan ts m u s t al l be pos i t ive) .W e shall e x a m i n e each case independent ly .

9 .2 .1 O p t im u m Is Internal PointMost o f the constrained parameter estimation prob lems be long to th i s case.Based on scient i f ic considerat ions, w e arrive qui te often at constraints that th eparameters o f t he math ema t ica l m o d e l sh ou l d sa t i s fy . M o st o f the t i m e these are ofthe f o rm ,k mi n.i < k, < kmax .i ; i= \,...,p (9 .14)

and our objective is to ensure that the op t ima l parameter est imates satisfy theabove constraints. I f our ini t ial guesses are very poor , dur ing the early i terationsof th e Gauss -Newton m e t h o d the parameters m ay reach beyond these boundar ieswhere the parameter values m ay have no physica l meaning or the mathematicalmode l breaks d o w n w h i c h in turn may lead to severe convergence prob lems . I f theop t imum is indeed an internal poin t , we can readi ly solve the prob lem us ing one ofth e f o l l ow i ng three app ro aches .

9.2 .1 .1 ReparameterizationThe simplest way to deal with constraints on the parameters is to ignoret hem ! We use the unconstrained algori thm of our choice and i f the converged pa-



6/9

Constrained Parameter Estimation 1 63

rameter es t imates sat isfy th e constraints no f u r the r act ion is r equ i r ed . Unfor tu-nately we cannot a lways use this approach. A smart way of imposing s imple con-straints on the parameters is th rough reparameterizat ion. For example , i f we havethe s imple constraint (often encountered in engineering problems) ,k, > 0 (9.15)

th e fo l lowing reparameter izat ion always enforces the above constraint (Bates andWatts , 1988),k i = e x / > ( K i ) (9.16)

By conduct ing our search over K , regardless of its value, exp(K0 an d hence k;is always posi t ive. For the m o r e general case of the in terval constraint on the pa-rameters given b y Equa t ion 9.14, we can perform the fo l l o win g t ransformation,

k - k , , ^ m a x , ! ^ m i n . i , ._,k j =kminj+=r-r- (9-17)\ + exp(K {)Using th e above t r an s fo rm a t io n , we are able to per fo rm an unconst ra inedsearch over K J . For any va lue of K J , the or ig ina l pa ramete r k , r em ain s wi t h in itsl imits . W h e n K , approaches very large values ( tends to inf ini ty) , k j approaches it slower l imit , k m n u wh ereas when K J approaches very large negative values ( tends tom i n u s inf ini ty) , k : approaches its upper l imit , k m a X ) i . Obvious ly , the above trans-formation increases the complexi ty o f the mathemat ica l model; however , there areno constraints on the parameters.

9.2.1.2 Penalty FunctionI n th is case instead of reparameter iz ing the prob lem, we augm ent the objec-

t ive funct ion b y adding extra te rms that tend to explode w h e n th e parameters ap -proach near the bou ndary and becom e negl ig ib le w h e n the parameters are far.O necan easily construct such functions.One of the s imples t and yet very effective penalty funct ion that keeps th eparameters in the interval (k m n w , k m ax ,) is

(9.18)~

Th e func t ions essentia l ly place an equal ly weighted penal ty for small orlarge-valued parameters on the overal l object ive function. I f penalty funct ions forCopyright 2001 by Taylor & Francis Group, LLC


7/9

164 Chapter 9

cons t ra in t s on al l u n k n o w n param e te r s are added to the ob jec t ive f u nc t i on , w eobta in ,

Sp(k,Q = (9 .19)or

s p ( k , g = k k ^ m a x . i ~ K m m , iK. j K. m m , i k " (9.20)The user suppl ied weig h t in g constant , C , (>0) , sho u l d have a large value

dur ing the early i terat ions of the Gauss-Newton method when the parameters areaway from the i r opt imal values. As the parameters approach th e o p t i m u m , ^should be reduced so that the contribut ion of the penalty funct ion is essentiallynegl igible (so that no bias is introdu ced in the parameter estimates).With a few minor modif ica t ions , th e Gauss-Newton method presented inChapter 4 can be used to obtain the u n k n o w n parameters. I f we consider Taylorseries expans ion of the penal ty funct ion around the cur ren t est imate of the pa-rameter we have,

A k < J + 1 )

where

and

50 k k^ m x ^

Ak^ +1 ) (9.21)

(9.22)

= 2 . r , . i, . 1sx i ~ i n i n i m 3x i ~ (9.23)

S u b s e q u e n t use of the stationary condi t ion (3Sp/9k ( i+1))=0 , yie lds the normalequa t ions



8/9

Constrained Parameter Estimation 165

AAk(H) = b (9.24)Th e diagonal e l e m e n t s of matr ix A are g iven by

( 2 ^A ( i , i ) = A G N ( i , i )+p-^ - (9.25)[S k i 2 }and e l em en t s o f vec tor b are given by

db ( i ) = bGN (i) - J- (9.26)

where matrix A G N and vector b G N are those given in Chap te r 4 for the Gauss -Newton method. The above equat ions apply equa l ly wel l to dif ferent ia l equat ionm o d e l s . In th is case A G N and boN are t hose g iven i n Ch ap te r 6 .

9.2 .1 .3 Bisection R u l eIf we are certain that the op t imum parameter est imates lie wel l wi th in theconstraint boundar ies , the s imples t way to ensure that the parameters stay wi th inthe bound ar ies is th rou gh the use of the bisection ru le . Nam ely , du r ing each itera-tion of the Gauss -Newton method, i f anyone of the new parameter estimates liebeyond its boundar ies , then vector A k ^ + 1 ) is halved, unt i l all the parameter con-straints are sat isf ied. Once the constraints are satisfied, w e proceed with the de-te rmina t ion of the step-size that wil l yie ld a reduct ion in the ob ject ive funct ion asalready discussed in Chapters 4 and 6.Our exper ience wi th a lgebra ic and dif ferent ial equat ion mode ls has shownthat th i s is indeed the easiest and most effect ive approach to u se . It has been im -p lemented in the comp uter programs provided with th i s book .

9.2.2 T h e K u h n - T u c k e r ConditionsTh e most general case is covered by the w e l l - k n o w n Kuhn-Tucher condi -t ions f or opt imal i ty . Let us assume the most general case where w e seek the un-k n o w n parameter vector k , that wi l l

Minimize S ( k ) = e ^ Q . e , ( 9 . 27 a )

http://dk5985_ch04.pdf/http://dk5985_ch04.pdf/http://dk5985_ch06.pdf/http://dk5985_ch06.pdf/http://dk5985_ch04.pdf/


9/9

166 Chapter 9

Subject to c p i ( k ) = 0 ; i = l , 2 , . . . , n p (9.27b)> 0 ; i = n , p + l , 1^+2,...,^+^ (9.27c)

This constrained m in im iz a t io n problem is again solved by in t roducing n ,pLagrange mul t ip l i e r s for the equal i ty constraints , n v Lagrange mul t ip l ie r s for theinequal i ty constraints and by fo rm in g th e augmented objective funct ion (Langran-gian)N n (p nv|/

SLG(k,co) = Ve T Q j e i + Y o o j c p , - Y(o n +i\|/n +i (9.28)LUV t J / , 1 1 1 /J 1T1 / 1 H(p+l Y ri(p-t-l V Ji = l i= l i= l

The necessary condi t ions for k* to be the op t ima l parameter values corre-spond ing to a m i n i m u m of the augmented object ive funct ion S LG (k,co) are givenby Edgar and H i m m e l b l a u (1988) and G i l l et al . (1981) and are brief ly presentedhere . T h e Langrang ian func t ion m ust be a t a stat ionary po in t , i . e . ,

a tk*,co* (9.29)5kThe constraints are satisfied at k*, i . e . ,

q > i ( k * ) = 0 ; i = l , 2 , . . . , n 9 (9.27b)an d v | / i ( k * ) > 0 ; 1=^+1,^+2,...,^+^ (9.27c)Th e Lagrange mul t ip l ie r s corresponding to the inequal i ty constraints (to;,i= n () )+l, . . . , n ^+n ^ ) are non-negat ive , in part icular ,

C 0 i > 0 ; for all active inequal i ty constraints (when \ |/;(k*)=0) (9.30a)andC O ; = 0 ; for all inactive inequal i ty constraints (when v|/j(k*)>0) (9.30b)

Based on the above, we can develop an "adaptive" Gauss -Newton me thodfor param e te r es t imat ion wi t h equa l i t y cons t ra in t s w h e r e b y the set of act ive c o n -straints (which are all equal i t ies ) is updated at each i teration. A n example is p r o -vided in Chapte r 14 where w e examine the est imation of binary interactions pa-rameters in cubic equat ions of state subject to predict ing the correct phase behav-ior (i .e. , avoiding er roneous tw o-phase split predict ions un der certain condit ions) .


engineers (chemical industries) by peter englezos [cap 9]

Documents