Linear Quadratic Gaussian Control and ModelPredictive Control (LQG and MPC)Sachin C. PatwardhanDept. of Chemical Engineering,I. I. T. Bombay, Powai, Mumbai 400 076Email: sachinp@iitb.ac.in1 Dynamic Model for Controller SynthesisLet us consider a stochastic process described by the following linear discrete state spacemodelx(/ + 1) = dx(/) +Iu(/)+v(/) (1)y(/) = Cx(/)+v(/) (2)where x 1nrepresents state variables, u 1mrepresents manipulated inputs, y 1rrepresents measured outputs and v 1nand v 1rrepresent unmeasured disturbances(state noise) and measurement noises, respectively. Here the vectors v(/) and v(/) areassumed to be zero mean white noise sequences such thatH1= 1_v(/)v(/)T(3)H12= 1_v(/)v(/)T(4)H2= 1_v(/)v(/)T(5)Such a model can be derived using either from linearization of a of rst principles (or greybox model or state realization of a time series model developed from input-output data.1.1 Linearization of First Principles / Grey Box ModelLinear discrete perturbation model of the formx(/ + 1) = dx(/) +Iu(/)+Idd(/) (6)y(/) = Cx(/)+v(/) (7)1can be developed in the neighborhood of an operating point starting from a nonlinear rstprinciples (or grey box) model. Here d 1drepresents vector of (physical) unmeasuredsuch as uctuations in feed concentrations.If it is further assumed that d(/) with knowncovariance matrix, say Q. then we havev(/) = Idd(/) (8)1 [v(/)] = 0 ; Co [v(/)] = 11 = IdQITd(9)When a state space model derived from rst principles is used for inferential control, the setof controlled outputs will dier from the set of measured output. We represent controlledoutputs yr 1ras followsyr(/) = Crx(/)Example: Consider a CSTR in which the state variable vector is perturbations in reactorconcentration and reactor temperaturex = _ oCAo1 _TIf reactor temperature is measured and it is desired to control reactor concentration, thenC = _ 0 1 _andCr = _ 1 0 _1.2 State Realization of Time Series ModelThe innovations form of state space model of the formx(/ + 1) = dx(/) +Iu(/) +L1o(/) (10)y(/) = Cx(/) +o(/) (11)can be obtained from time series models (ARX/ARMAX) models identied from input-output data. Here, the innovations (or residuals) o(/) are a zero mean Gaussian whitenoise sequence with covariance matrix Ye and K represents the corresponding steady stateKalman gain. The above model can be re-written asx(/ + 1) = dx(/) +Iu(/) +v(/) (12)y(/) = Cx(/) +v(/) (13)2where the vectors v(/) 1nand v(/) 1rare zero mean white noise sequences such thatH1= 1_v(/)v(/)T = L1YTeLT1(14)H12= 1_v(/)v(/)T = L1Ye(15)H2= 1_v(/)v(/)T = Ye(16)It may be noted that, when a state space model is identied from data, typically we haveCr = C, i.e., the set of outputs that can be controlled is identical to the set of measuredoutputs.2 Quadratic Optimal ControlThe above model is used as a basis for design of a state feedback controller. Steps involvedin design of any state feed-back controller are as follows1. Solve regulator / controller design problem under the assumption that full state isavailable for feedback2. Design state estimator and implement control law using estimated states.In this section, we rst describe the method for designing optimal state feedback controllawand later showhowcontrol lawcan be implemented using optimal state observer (KalmanFilter).2.1 Linear Quadratic Optimal Regulator DesignWe rst discuss the state regulator design problem, where it is desired to bring the systemform non-zero initial state to zero initial state (the origin of state space).Non-zero initialstate can result from impulse like disturbances, which are suciently spaced in time. Wewant to device a state feedback regulator of typeu(/) = Gx(/)which will bring the system to the origin in a optimal fashion. For designing the regulator,we only consider the deterministic part of the model, i.e.x(/ + 1) = dx(/) +Iu(/) (17)y(/) = Cx(/) (18)3The regulator design problem is posed as an optimization problem where it is desired todetermine control sequence u(0). u(1).....u(` 1) such that objective functionJ = 1_N1

k=0_x(/)TVxx(/) +u(/)TVuu(/)+x(`)TVNx(`)_(19)is minimized. Here 1(.) represents the expectation and r(`) represents the state at naltime `1. Vx. Vu and VN are symmetric positive denite matrices. This optimizationproblem is solved using the Bellmans method of dynamic programming. In order to derivethe control law, we start optimization from time / = ` and work backwards in time.Letus dene S(`) = VN andJ(/) = minu(k):::::u(N1)1_N1

i=k_x(i)TVxx(i) +u(i)TVuu(i)+x(`)TVNx(`)_(20)For / = `.J(`) = x(`)TVNx(`) (21)Then, for / = ` 1.J(` 1) =minu(N1)_x(` 1)TVxx(` 1) +u(` 1)TVuu(` 1) + J(`)_(22)ButJ(`) = x(`)TVNx(`)= [x(` 1) + u(` 1)]TS(`) [x(` 1) + u(` 1)] (23)Thus,J(`1) =minu(N1)_x(` 1)T _Vx + TS(`)x(` 1) +x(` 1)T

TS(`)u(` 1)+u(` 1)T

TS(`)x(` 1) +u(` 1)T _

TS(`) +Vuu(` 1)_(24)Note that the rst term x(` 1)T _Vx + TS(`)x(` 1) in J(` 1)cannot be inu-enced by u(` 1).We solve the problem of minimizing the last three terms in J(` 1)by method of competing squares. In order to see how this can be achieved, consider a scalarquadratic function1 (u) = uTAu +zTu +uTz (25)= uTAu +zTu +uTz +zTA1z zTA1z (26)= _u +A1z_TA_u +A1z_zTA1z (27)4where A is a positive denite matrix. The rst term on the right hand side in the aboveequation is always non-negative. This implies that minimum of 1 (c)with respect to u isattained atu = A1z (28)and its minimum value is1min (u) = zTA1z (29)In the minimization problem at hand, with some rearrangement, we haveA = _

TS(`) +Vu(30)z = TS(`)x(` 1) (31)and the optimal solution can be expressed asu(` 1) = G(` 1)x(` 1) (32)G(` 1) = _Vu + TS(`)_1

TS(`) (33)which gives minimum loss function,J(` 1) = x(` 1)TS(` 1)x(` 1) (34)where,S(` 1) = TS(`) +VxG(` 1)T _Vu + TS(`)G(` 1) (35)Similar arguments give,J(` 2) =minu(N2)_x(` 2)TVxx(` 2) +u(` 2)TVuu(` 2) + J(` 1)_(36)This is same as the earlier optimization problem, but with the time argument shifted. Theprocedure can be repeated backward in time. The equation,S(/) = [ G(/)]TS(/ + 1) [ G(/)] +Vx +G(/)TVuG(/) (37)is called the discrete time Riccati equation. The matrices S(`) = VN and Vu are assumedto be positive denite and symmetric. This implies that S(/) is positive denite/semi-deniteand symmetric and this condition guarantees optimality at each stage. When horizon `becomes large, S(/) tends to a constant matrix S(/) S1 which can be computed bysolving the algebraic Riccati equation (ARE)G1 = _Vu + TS1

_1

TS1 (38)5S1 = [ G1]TS1 [ G1] +Vx +GT1VuG1(39)This ARE has several solutions. However, if (. ) is reachable and if (. X) is observablepair, where Vu = XTX, then there exists a unique, symmetric, non-negative denite solutionto the ARE. The corresponding state feedback control law can be formulated asu(/) = G1x(/) (40)Further, when (. ) is controllable and objective function is symmetric and positive def-inite, the LQ controller will always give asymptotically stable closed loop behavior. Byselecting Vx and Vu appropriately, it is easy to compromise between speed of recovery andmagnitude of control signals.2.2 Stability of LQ controllerTheorem 1 (Stability of the closed loop system) : Consider the time invariant systemgiven by equation (1) the loss function for the optimal control is given by equation (19).Assume that a positive-denite steady state solution S1 exists for Riccati equation (39).Then the steady state optimal strategyu(/) = G1x(/) = _Vu + TS1

1

TS1x(/)gives an asymptotically stable closed-loop systemx(/ + 1) = ( G1)x(/)Proof: Theorem A.3 in Appendix can be used to show that the closed loop system isasymptotically stable (see Appendix for denitions of stability). It is sucient to show thatthe function\ (x(/)) = xT(/)S1x(/)is a Lyapunov function. \ is positive denite and\ (x(/)) = xT(/ + 1)S1x(/ + 1) xT(/)S1x(/)= xT(/)( G1)TS1( G1)x(/) xT(/)S1x(/)= xT(/)[Vx + 1TVu1]x(/)Because Vx + GT1VuG1 is positive denite, \is negative denite. The closed loopsystem is thus asymptotically stable.6The poles of the closed loop system can be obtained in several ways. When the design iscompleted, the poles are obtained fromdct(`1 + G1) = 0It is possible to show that the poles are the : stable eigenvalues of the generalizedeigenvalue problem.__1 0Vx

T_` _V1u 0 1__ = 0This equation is called the Euler equation of the LQ problem. Theorem 1 shows that LQcontroller gives a stable closed loop system, i.e. all poles ofG1 are inside unit circle.2.3 Linear Quadratic Gaussian Control2.3.1 State EstimationThe model (1-2) can be used to develop the optimal state predictor as followso(/) = y(/) C x(/[/ 1) (41) x(/ + 1[/) = d x(/[/ 1) +Iuu(/) +Lpo(/) (42)where Lp is the solution of the steady state Riccati equationLp= _dI1CT+H12 _CI1CT+H21(43)I1= dI1dT+H1Lp_CI1CT+H2 LTp(44)Here, matrix I1 denotes steady state covariance of error in state estimation. It can be shownthat the residual (or innovation) o(/) is a zero mean Gaussian white noise process withcovariance matrix Y1.The state feedback LQ control law designed above is implementedtogether with the Kalman predictor using estimated states as followsu(/) = G1 x(/[/ 1) (45)Alternatively, model (1-2) can be used to develop the optimal current state estimator asfollowso(/) = y(/) C x(/[/ 1) (46) x(/[/ 1) = d x(/ [/ 1) +Iuu(/ 1) (47) x(/[/) = x(/[/ 1) +Lco(/) (48)7where Lc is the solution of the steady state Riccati equationI1;1= dI0;1dT+H1(49)Lc= _dI1;1CT+H12 _CI1;1CT+H21(50)I0;1= [I LcC] I1;1(51)We can then use Kalman ler (current state estimator) to implement control law as followsu(/) = G1 x(/[/) (52)2.3.2 Separation PrincipleTo assess the nominal stability of the closed loop generated by observer - regulator pair, theobserver and the plant dynamics has be considered together. For example, the closed loopsystem with Kalman predictor can be described by following set of equationsx(/ + 1) = dx(/) +Iu(/) +v(/) (53)y(/) = Cx(/) +v(/) (54) x(/ + 1[/) = d x(/[/ 1) +Iu(/) +Lp [y(/) C x(/[/ 1)] (55)By introducing state estimation error(/[/ 1) = x(/) x(/[/ 1) (56)the above set of equations can be rearranged as_ x(/ + 1)(/ + 1[/)_=_ [dIG1] IG1[0] [dLpC]__ x(/)(/[/ 1)_+_ II_v(/) +_ [0]Lp_v(/) (57)Thus, dynamics of LQG controller is determined by dynamics of LQ controller (i.e. eigenvalues of [dIG1]) and dynamics of the state observers (i.e. eigen values of [dLpC] ).Thus, designing state feedback controller and state observer to be individually stable ensuresstability of the closed loop (separation principle). As a consequence, if the system underconsideration is observable and controllable, then the resulting closed loop is guaranteedto be stable if weighting matrices in LQ formulation are chosen to be positive denite /semi-denite. It is straight forward to show that separation principle also holds in this case.82.4 Linear Quadratic Optimal Controller2.4.1 State Augmentation for Unmeasured Disturbance ModelingLinear quadratic regulator designed above can generate oset if (a) the unmeasured dis-turbances are non-stationary, i.e. they have slowly drifting behavior (b) mismatch existsbetween the plant and the model. In order to deal with such a situation and introduceintegral action in the controller in the face of plant-model mismatch, the state space model(1-2) is augmented with extra articial states as followsx(/ + 1) = dx(/) +Iuu(/) +I

(/) +v(/) (58)(/ + 1) = (/) +v

(/) (59)(/ + 1) = (/) +v

(/) (60)y(/) = Cx(/) +C

(/) +v(/) (61)where Hsand Htare articially introduced input and output disturbance vectorswhile vectors v

Hsand v

Htare zero mean white noise sequences with covariances Q

and Q

. respectively. The model coecient matrices (I

, C

) and noise covariances matrices(Q

, Q

) are treated as tuning parameters, which can be chosen to achieve the desired closedloop disturbance rejection characteristics. Note that the total number of extra states cannotexceed the number of measurements due to the requirement that the additional states shouldbe observable. Typical choices of the articial state variables and the corresponding couplingmatrices are as follows Output bias formulation: A simple approach is to view the drifting disturbancesas causing a bias in the measured outputs, i.e., we can chooseI

= [0] ; Q

= [0] ; C

= 1 ; Q

= o21 Input Bias Formulation: The elements of vector can be viewed as bias in : ma-nipulated inputs. When number of manipulated inputs equals the number of measuredoutputs (: = :), then we can chooseI

= Iu; Q

= o21 ; C

= [0] ; Q

= [0]If number of manipulated inputs exceeds the number of measurements, then : linearlyindependent columns of Iu can be selected as I

. Disturbance bias formulation: When the state space model is derived from rstprinciples, it is possible to chooseI

= d ; Q

= o219provided number of disturbance variables (d) = :. If d:. then : linearly independentcolumns of d can be chosen as

.In all the above cases, o2is treated as a tuning parameter. The above set of equationscan be combined into an augmented state space model of the formxa(/ + 1) = daxa(/) +Iuau(/) +va(/) (62)y(/) = Caxa(/) +v(/) (63)wherexa(/) =__ x(/)(/)/)__; va(/) =__ v(/)v

(/)v

/)__da=__ d I

[0][0] I

[0][0] [0] I

__; Iua =_ Iu0_Ca= _ C[0] C

_H1a= 1_va(/)va(/)T =__ H1[0] [0][0] Q

[0][0] [0] Q

__H12a= 1_va(/)v(/)T =_ H12[0]_H2a= 1_v(/)v(/)T = H2This augmented model can be used for developing a Kalman predictor of the formoa(/) = y(/) C xa(/[/ 1) (64) xa(/ + 1[/) = da xa(/[/ 1) +Iuau(/ 1) +Laoa(/) (65)where the steady state Kalman gain is obtained by solving the corresponding steady stateRiccati equationsLa= _daIa1CTa +H12a _CaIa1CTa +H21(66)Ia1= daIa1dTa +H1La_CaIa1CTa +H2 LTa(67)In order to maintain the observability of the articially introduced states, the number of ad-ditional states introduced in the augmented model should not exceed the number of measuredoutputs . When the state space model (10-11) is observable and stable with no integratingmodes, the augmented state space model will be observable (detectable) in most of the cases.102.4.2 Quadratic Optimal Servo ControlThe problem of tracking a setpoint is solved by modifying the regulatory control law asfollowsu(/) us(/) = G [x(/) xs(/)] (68)u(/) = us(/) G [x(/) xs(/)] (69)where xs(/) represent the nal steady state target corresponding to the setpoint, say r(/).and us(/) represents the steady state input necessary to reach this steady state target. Atany instant /.these vectors are computed by solving steady state equationxs(/) = dxs(/) +Iuus(/) +I

(/) (70)r(/) = Crxs(/) +C

(/) (71)Eliminating xs(/) from above equations, we haver(/)= Kuus(/) +K

(/) +C

(/) (72)where steady state gain matrices Ku and K

are dened as followsKu = Cr (I d)1

u; K

= Cr (I d)1

When number of inputs (:) equals the number of controlled outputs (:), we can re-arrangeabove equations as followsus(/) = K1u[r K

(/) C

(/)] (73)xs(/) = (I d)1_

uK1u[r(/) C

(/)] +_

uK1u K

_(/)(74)When number of the manipulated inputs (:) is not equal to number of controlled outputs(:), matrix K1uin the above expression should be replaced by Kyu. i.e., pseudo-inverse ofthe steady state gain matrix Ku. For the case : = :. two special cases of quadratic optimaltracking control law are as follows Output bias formulation: In this case we have I

= [0] . C

= 1. which impliesthat K

= [0] and computation for xs(/). us(/) reduces tous(/) = K1u[r(/) (/)] (75)xs(/) = (I d)1

uK1u[r(/) (/)] (76)11 Input Bias Formulation:In this case we have I

= Iu . C

= [0]. which impliesthat K

= Ku and computation for xs(/). us(/) reduces tous(/) = K1u r(/) (/) (77)xs(/) = (I d)1

uK1u r(/) (78)3 Model Predictive ControlLQG formulation described above provides a systematic approach to designing a control lawfor linear multi-variable systems. However, main diculty associated with the classical LQGformulation is inability to handle constraints explicitly. Operating constraints, such as limitson manipulated inputs, limits on controlled outputs arising out of product quality or safetyconsiderations are commonly encountered in any real problem. Also, it is dicult to dealwith plant-model mismatch in classical LQG framework.Model Predictive Control (MPC) refers to a class of multi-variable control algorithmsdeveloped in process industry to deal with operating constraints together with multi-variableinteractions. MPC can be viewed as modied versions of LQ (or LQG) formulation, whichcan deal with plant-model mismatch and operating constraints in a systematic manner. Thisapproach was rst proposed independently by two industrial groups Dynamic Matrix Control (DMC): Proposed by Cutler and Ramaker from Shell, USAin 1978. Model Algorithmic Control (MAC): proposed by Richalet (1978, France)These initial versions were based on nite impulse models (FIR) models. In this section,we rst present DMC formulation starting from equations (17-18). We later proceed todevelop the recent version of MPC formulation based on Kalman lter, which is bettersuited for dealing with unmeasured disturbances.3.1 Model Based Prediction of Future BehaviorThe main component of DMC is the model describing deterministic contributions to processdynamics given by equations (17-18). This model is used on-line to performfuture predictionsof plant dynamics. When the system under consideration is open loop stable, this model canbe used to develop an open loop state observer as follows x(/[/ 1) = d x(/ 1[/ 2) +Iuu(/ 1) (79)12where x(/[/ 1) denotes estimate of state x(/) based on information up to time instant(/ 1).In a typical DMC formulation, at each sampling instant, the state estimator (79) is usedfor predicting future behavior of the plant over a nite future time horizon of length j (calledas the prediction horizon) starting from the current time instant /. Let us assume that atany instant /. we are free to choose only future manipulated input movesu(/[/). u(/ + 1[/)......u(/ + 1[/)with the following constraints on the remaining of the future input movesu(/ + [/) = u(/ + + 1[/) = ...... = u(/ + j 1[/) = u(/ + 1[/) (80)where is called as the control horizon. The predicted estimates of the state variables canbe generated by recursively using the state estimator (41-42) as follows x(/ + 1[/) = d x(/[/ 1) +Iuu(/[/) (81) x(/ + , + 1[/) = d x(/ + ,[/) +Iuu(/ + ,[/) (82)A predictive control formulation based on the above open loop observer predictions canpose practical diculties in the presence of mismatch between plant and the model. Sincethe model is identied once in the initial phase of the MPC implementation, discrepanciesbetween plant and model parameters can arise over the period of time due to shift in theoperating point, changes in the disturbance characteristics etc. Thus, there is a need tointroduce some mechanism to account for the plant model mismatch, which, in turn, intro-duces integral action in the controller formulation. In DMC formulation, this is achieved bycorrecting the future predictions as follows y(/ + , + 1[/) = Cr x(/ + ,[/) + (/ + ,[/) (83) (/ + , + 1[/) = (/ + ,[/) (84) (/[/) = y(/) C x(/[/ 1) (85)(for , = 1....j) (86)Note that vector (/) contain information about unknown / unmeasured disturbances aswell as plant model mismatch. It is easy to see that this mismatch compensation strategy isequivalent to the output bias formulation discussed in the previous section.133.2 Future Setpoint Trajectory GenerationIn addition to predicting the future output trajectory, at each instant, a ltered futuresetpoint trajectory is generated using a reference system of the formxr(/ + , + 1[/) = drxr(/ + ,[/) +Ir [r(/) yr(/)] (87)yrf(/ + , + 1[/) = y(/) +Crfxr(/ + , + 1[/) (88)for , = 0. 1. ....j 1with initial condition xr(/) = 0. Here, r(/) 1rrepresents the setpoint vector. Thecoecient matrices of the reference system are tuning parameters which can be selectedto achieve the desired closed loop tracking performance. In order to ensure the free servoresponses for step changes in the setpoint, the coecient matrices of the reference systemshould be selected such that the steady state gain of the reference system is equal to theidentity matrix, i.e.Crf(I dr)1Ir = ITypically, the reference system is selected such that its transfer function matrix is diagonalwith unit gain rst (or higher) order low pass lters on the main diagonal.3.3 Base Dynamic Matrix Control (DMC) FormulationGiven future desired setpoint trajectory yrf(/ + ,[/) : , = 1. 2. ....j at instant /. the modelpredictive control problem at the sampling instant / is dened as a constrained optimizationproblem whereby the future manipulated input moves u(/[/). u(/ + 1[/)......u(/ + 1[/)are determined by minimizing an objective function dened as =p

j=1of(/ + ,[/)Tveof(/ + ,[/) + q1

j=0u(/ + ,[/)Tvuu(/ + ,[/) (89)whereof(/ + ,[/) = yrf(/ + ,[/) y(/ + ,[/) (90), = 1. 2. ....j (91)represents future predicted error andu(/ + ,[/) = u(/ + ,[/) u(/ + , 1[/) (92), = 1. ... 1u(/[/) = u(/[/) u(/ 1) (93)14The above minimization problem is subject to following constraintsyL_ yc(/ + ,[/) _ yH(94), = 1. 2. ....juL_ u(/ + ,[/) _ uH(95)uL_uf(/ + ,[/) _ uH(96), = 0. 1. 2. .... 1Here, ve represents positive denite the error weighting matrix and vu represents positivesemi-denite the input move weighting matrix. The closed loop stability and the desiredclosed loop performance can be achieved by judiciously selecting the prediction horizon j.control horizon . Typically, prediction horizon is selected close to the open loop settlingtime while control horizon () is chosen signicantly smaller (say between 1 to 5). Theweighting matrices ve and vu are typically chosen diagonal and can be eectively used tospecify relative importance of errors and manipulated input moves. The resulting constrainedoptimization problem can be solved using any standard nonlinear programming method suchas SQP (Sequential Quadratic Programming). For ecient on-line implementation, it ispossible to re-arrange above problem as a QP problem.The controller is implemented in a moving horizon frame work. Thus, after solving theoptimization problem, only the rst move uopt(/[/) is implemented on the plant, i.e.u(/) = uopt(/[/)and the optimization problem is reformulated at the next sampling instant based on theupdated information from the plant. Note that the use of open loop observer limits applica-bility of DMC to open loop stable systems. Thus, if a process has unstable modes, thenthese modes are stabilized rst by introducing a local feedback loop and DMC is applied asa master controller in a cascade conguration.3.4 Unconstrained DMC Control LawDening the future input vector lf(/) and the predicted output vector (/) over the futurehorizon aslf(/) = _ u(/[/)Tu(/ + 1[/)T....... u(/ + 1[/)T _T(97)(/) = _ y(/ + 1[/)T y(/ + 2[/)T..... y(/ + j[/)T _(98)the above prediction model, together with the constraints (118) on the future inputs, can beexpressed as(/) = Sx x(/,/ 1) +Sulf(/) +S

(/) (99)15whereSx =__CdCd2......Cdp__; S

=__II......I__(100)Su =__CIu[0] [0] .... [0]CdIuCIu[0] .... [0]..... ..... .... ... [0]Cdq1IuCdq2Iu.... ... CIuCdqIuCdq1Iu... .... C(d+ 1)Iu..... ...... .... .... ....Cdp1IuCdp2Iu... ... C(dpq+ .... + 1)Iu__(101)Here, matrix Su is called dynamic matrix of the system. Dening the future referencetrajectory vector H(/) asH(/) = _ yrf(/ + 1[/)Tyrf(/ + 2[/)T..... yrf(/ + j[/)T _T(102)the prediction error vector E(/) at instant / can be computed asE(/) = H(/) (/) (103)The unconstrained version of the DMCcontrol problemstated above can be re-cast as followsminUf(k)E(/)TVEE(/) + lf(/)TVU lf(/) (104)where VE and VU represents error weighting and input move weighting matrices, respec-tively, and are dened asVE= dicq_ veve.... ve_(105)VU= dicq_ vuvu.... vu_(106)Here, lf(/) is dened aslf(/) =__u(/,/) u(/ 1)u(/ + 1,/) u(/,/)......u(/ + 1,/) u(/ + 2,/)__ = Alf(/) A0u(/ 1) (107)16whereA =__1 [0] [0] [0]1 1 [0] [0]... ... ... ...[0] .... 1 1__; A0 =__1[0]...[0]__(108)This unconstrained problem can be solved analytically to compute a closed form control law.The unconstrained optimization problem can be reformulated asminUf(k)12lf(/)THlf(/) +lf(/)Tz(/) (109)whereH= 2(STuVESu +ATVUA) (110)z(/) = 2_(H(/) Sx x(/,/ 1) Sd (/))TVESu + (A0uk1)TVUA_(111)The least square solution to above minimization problem is[lf(/)]opt = H1z(/)Since only the rst input move is implemented on the processuopt(/[/) = AT0lf(/) = AT0H1z(/)With some algebraic manipulations, the above control law can be rearranged as followsuopt(/[/) = Guu(/ 1) Gx x(/[/ 1) +G

(/[/) +GrH(/)From the above rearranged equation, it is easy to see that the unconstrained DMC controllaw is a special type of state feedback controller.3.5 QP FormulationThe base constrained DMC formulation stated in Section 3.3 can be re-cast as followsminUf(k)E(/)TVEE(/) + lf(/)TVU lf(/) (112)subject to the following constraintsL_ (/) _ H(113)lL_ lf(/) _ lH(114)lL_lf(/) _ lH(115)17Note that above formulation has a quadratic objective function and linear constraints. Thus,for improving the computational eciency, the above problem can be transformed into anequivalent quadratic programming (QP) formulation as followsminUf(k)12lf(/)THlf(/) +zTlf(/) (116)Subject toAlf(/) _ I (117)whereA =__IqmIqmAASuSu__; I =__lHlLlH+A0uk1lLA0uk1Sx x(/,/ 1) Sdd(/) +HSx x(/,/ 1) +Sdd(/) H__Here, Iqm represents identity matrix of dimension :. QP formulation is more suitablefor on-line implementation as ecient algorithms exist for solving QP.3.6 MPC Formulation Based on Kalman FilterThe DMC controller discussed above gives good setpoint tracking performance. However.the main limitation of DMC formulation arises from poor handling of unmeasured distur-bance and plant model mismatch. Thus, the research eorts over last decade were directedtoward improving disturbance handing capabilities of MPC formulations. This section illus-trates how MPC can be formulated using Kalman predictor to improve disturbance rejectioncapabilities of MPC.The main component of the improved MPC formulation is the model describing deter-ministic and stochastic contributions to process dynamics (equations 1-2), which is used todevelop the Kalman predictor (equations 41-44). At each sampling instant, the Kalman stateestimator is used for predicting future behavior of the plant over a nite future time horizonof length j (called as the prediction horizon) starting from the current time instant /. Let usassume that at any instant /. we are free to choose only future manipulated input movesu(/[/). u(/ + 1[/)......u(/ + 1[/)with the following constraints on the remaining of the future input movesu(/ + [/) = u(/ + + 1[/) = ...... = u(/ + j 1[/) = u(/ + 1[/) (118)18where is called as the control horizon. As the expected value of future innovations is zero,the optimal predicted estimates of the state variables can be generated by recursively usingthe state estimator (41-42) as follows x(/ + 1[/) = d x(/[/ 1) +Iuu(/[/) +Lo(/) (119) x(/ + , + 1[/) = d x(/ + ,[/) +Iuu(/ + ,[/) (120)(for , = 1....`p1)A predictive control formulation based on the above optimal predictions can pose practicaldiculties in the presence of mismatch between plant and the model. Since the model isidentied once in the initial phase of the MPC implementation, discrepancies between plantand model parameters can arise over the period of time due to shift in the operating point,changes in the disturbance characteristics etc. Thus, there is a need to introduce somemechanism to account for the plant model mismatch, which, in turn, introduces integralaction in the controller formulation. This can be achieved by augmenting the state spacemodel with articially introduced input and / or output disturbance variables, which behaveas integrated white noise sequences. The resulting augmented model can be written asx(/ + 1) = dx(/) +Iuu(/) +I

(/) +v(/) (121)(/ + 1) = (/) +v

(/) (122)(/ + 1) = (/) +v

(/) (123)y(/) = Cr(/) +C

(/) +v(/) (124)where Hsand Htare articially introduced input and output disturbance vectorswhile vectors v

Hsand v

Htare zero mean white noise sequences with covariancesQ

and Q

. respectively. The model coecient matrices (I

, C

) and noise covariancesmatrices (Q

, Q

) are treated as tuning parameters, which can be chosen to achieve thedesired closed loop disturbance rejection characteristics. Typical choices of the articialstate variables and the corresponding coupling matrices have been discussed in Section 2.3.This augmented model can be used for developing a Kalman predictor of the formoa(/) = y(/) C xa(/[/ 1) (125) xa(/ + 1[/) = da xa(/[/ 1) +Iuau(/) +Laoa(/) (126)where La represents the steady state Kalman gain is obtained by solving the correspondingsteady state Riccati equations.The optimal predictions of states based on the augmented19state space model can be generated as follows xa(/ + 1[/) = da xa(/[/ 1) +Iuau(/[/) +Laoa(/) (127) xa(/ + , + 1[/) = da xa(/ + ,[/) +Iuau(/ + ,[/) (128)(for , = 1....j 1) (129) y(/ + ,[/) = Ca xa(/ + ,[/) (for , = 1....j) (130)Using the above augmented state observer, the MPC controller is formulated as an optimiza-tion problem at every instant as discussed in Section 3.3. It is also possible to derive closedform unconstrained control law and a QP formulation suitable for on-line implementation asdiscussed in section 3.4 and 3.5, respectively.4 Appendix: Stability4.1 DenitionsIt is assumed that the notion of stability is known from the basic texts in control theory.Only the basic denitions are given here.Consider the discrete time, state space equation(possibly nonlinear and time varying)x(/ + 1) = F(x(/). /) (131)Let x

(/) and x(/) be solutions of Eq. (131) when the initial conditions are x

(/0) andx(/0), respectively. Further let | . |denote a vector norm.Denition A.1 (Stability): The solution x

(/) of Eq. (131) if stable if for any givenc0, there exists a o(c. /0)0 such that all solutions with | x(/0) x

(/0) |< o are suchthat | x(/) x

(/) |< c and for all / = /0.Denition A.2 (Asymptotic stability): The solution x

(/) of Eq.131 is asymp-totically stable if it is stable and if | x(/) x

(/) | 0 when / provided that| x(/0) x

(/0) | is small enough.From the denitions, it follows that stability in general is dened for a particular solutionand not for the system.Theorem A.1. Asymptotic stability of linear systems. A discrete-time, linear, time-invariant system,x

(/ + 1) = x

(/). x

(0) = c0(132)is asymptotically stable if and only if all eigen values ofare strictly inside the unit circle.20Stability with respect to disturbances in the initial value has already been dened. Othertypes of stability concepts are also of interest. In control we often work with input-outputmodels of the formy(/) = F[y(/ 1). y(/ 2). .... y(/ :y). u(/ 1). ..... u(/ :u)] +o(/) (133)In the context of such models, we are interested in knowing whether the dynamic systemwill generate bounded output sequence, i.e. |y(/)|1 _ `y < for all / if the system issubjected to a bounded input sequence, i.e. |u(/)|1 _ `u < for all /.Denition A.3. BIBO stability: A linear, time-invariant system is bounded-input-bounded-output (BIBO) stable if a bounded input gives a bounded output for every initialvalue.From the denition it follows that asymptotic stability is the strongest concept. Thefollowing theorem is a result.Theorem A.2. Asymptotic stability implies stability and BIBO stability.4.2 Lyapunovs Second MethodLyapunovs second method is a useful tool for determining the stability of non-liner dynamicsystems. (Analyzing stability using eienvalues is often referred to as Lyapunovs rst method.) Lyapunov developed the theory for dierential equation, but a corresponding theory canalso be derived for dierence equations.Denition A.4. Lyapunov function: Let \ (x) represent a Lyapunov function for thesystem,x(/ + 1) = F(x(/)) ; F(0) = 0 (134)If:1. \ (x) is continuous in x and \ (0) = 0.2. \ (x) is positive denite.3. \ (x) = \ [x(/ + 1)] \ [x(/)] = \ [F(x(/))] \ [x(/)] is negative denite for all /.A simple geometric illustration of the denition is given in Figure (4.2). The level curvesof a positive denite continuous function \ [x(/)] are closed curves in the neighborhood ofthe origin. Let each curve be labelled by the value of the function. Condition 3 above impliesthat the dynamics of the system are such that the solution always moves toward curves with21lower values.All the curves encircle the origin and do not intersect any other level curve.It thus seems that the existence of a lyapunov function ensures asymptotic stability. Thefollowing theorem is a precise statement of this fact.Theorem 3. Stability theorem of Lyapunov: The solution x(/) = 0 is asymp-totically stable if there exists a Lyapunov function for the system given by equation (134).Further, if there exists (| x |) such that0 < (| x |) < \ (x)where (| x |) as | x |. then the solution is asymptotically stable for all initialconditions.The main obstacle to using the Lyapunov theory is nding a suitable Lyapunov function.This is in general a dicult problem; however, for the linear system of Eq.132, it is straight-forward to determine quadratic Lyapunov functions. Take \ (x) = xTIx as a candidate forLyapunov function. The increment of \ is then given by\ (x) = \ (x) \ (x) = xT

TIx xTIx= xT[TI 1]x = xTQxFor \to be a Lyapunov function, it is thus necessary and sucient that there exists apositive denite matrix 1 that satises the equation

TI I = Q (135)22where Q is positive denite. Eq.135 is called Lyapunovs equation. It can be shown thatthere is always a solution to the Lyapunov equation when the linear system is stable. Thematrix I is positive denite if Q is positive denite.5 ReferencesAstrom, K. J. and B. Wittenmark, Computer Controlled Systems, Prentice Hall, 1990.Franklin, G. F. and J. D. Powell, Digital Control of Dynamic Systems, Addison-Wesley,1989.23