chapter 8 the maximum principle: discrete time
DESCRIPTION
Chapter 8 The Maximum Principle: Discrete Time. 8.1 Nonlinear Programming Problems We begin by starting a general form of a nonlinear programming problem. y : be an n -component column vector, a : be an r -component column vector, b : be an s -component column vector. h: E n E 1 , - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/1.jpg)
Chapter 8 The Maximum Principle: Discrete Time
8.1 Nonlinear Programming Problems We begin by starting a general form of a nonlinearprogramming problem.
y: be an n-component column vector,a: be an r-component column vector,b: be an s-component column vector.h: En E1,g: En Er,w: En Es be given functions.
![Page 2: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/2.jpg)
We assume functions g and w to be column vectors with components r and s , respectively. We considerthe nonlinear programming problem:
subject to
![Page 3: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/3.jpg)
8.1.1 Lagrange MultipliersSuppose we want to solve (8.1) without imposingconstraint (8.2) or (8.3). The problem is now theclassical unconstrained maximization problem ofcalculus, and the first-order necessary conditions forits solution are
The points satisfying (8.4) are called critical points.With equality constraints, the Lagrangian is
where is an r-component row vector.
![Page 4: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/4.jpg)
The necessary condition for y* to be a (maximum)solution to be (8.1) and (8.2) is that there exists an r- component row vector such that
Suppose (y*, *) is a solution of (8.6) and (8.7). Note that y* depends on a, i.e., y*=y*(a). Now
is the optimum value of the objective function. The Lagrange multipliers satisfy the relation
which means that *i is the negative of the imputed value
of the unit of ai.
![Page 5: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/5.jpg)
Example 8.1 Consider the problem:
Solution.
From the first two equations we get
solving this with the last equation yields the quantities
![Page 6: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/6.jpg)
8.1.2 Inequality Constraints
Note that (8.10) is analogous to (8.6). Also (8.11)repeats the inequality constraint (8.3) in the sameway that (8.7) repeated the equality constraint (8.2).However, the conditions in (8.12) are new and areparticular to the inequality-constrained problem.
![Page 7: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/7.jpg)
Example 8.2
Solution. We form the Lagrangian
The necessary conditions (8.10)-(8.12) become
![Page 8: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/8.jpg)
Case 1:From (8.13) we get x = 4, which also satisfies (8.14).Hence, this solution, which makes h(4)=16, is a possible candidate for the maximum solution.
Case 2: Here from (8.13) we get = - 4, which does not satisfythe inequality 0 in (8.15).
From these two cases we conclude that the optimumsolution is x* = 4 and
![Page 9: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/9.jpg)
Example 8.3 solve the problem:
Solution. The Lagrangian is
The necessary conditions are
![Page 10: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/10.jpg)
Case 1: = 0From (8.16) we obtain x = 4 , which does not satisfy(8.17), thus, infeasible.
Case 2: x=6(8.17) holds. From (8.16) we get = 4, so that (8.18)holds. The optimal solution is then
since it is the only solution satisfying the necessaryconditions.
![Page 11: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/11.jpg)
Example 8.4 Find the shortest distance between thepoint (2.2) and the upper half of the semicircle ofradius one, whose center is at the origin. In order tosimplify the calculation, we minimize h , the square ofthe distance:
![Page 12: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/12.jpg)
The Lagrangian function for this problem is
The necessary conditions are
From (8.24) we see that either =0 or x2+y2 =1,i.e., weare on the boundary of the semicircle. If =0, we seefrom (8.20) that x=2. But x=2 does not satisfy (8.22) forany y , and hence we conclude >0 and x2+y2 =1.
![Page 13: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/13.jpg)
Figure 8.1 Shortest Distance from a Point to a Semi-Circle
![Page 14: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/14.jpg)
From (8.25) we conclude that either or y =0. If , then from (8.20), (8.21) and >0, we get x = y. Solving the latter with x2+y2 =1 , gives
If y =0, then solving with x2+y2 =1 gives
These three points are shown in figure 8.1. Of thethree points found that satisfy the necessaryconditions, clearly the point found in (a) isthe nearest point and solves the closest-pointproblem. The point (-1,0) in (c) is in fact the farthestpoint; and the point (1,0) in (b) is neither the closestnor the farthest point.
![Page 15: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/15.jpg)
Example 8.5 Consider the problem:
subject to
The set of points satisfying the constraints is shownshaded in figure 8.2. From the figure it is obvious thatthe solution point (0,1) maximizes the value of y. Let us see if we can find it using the above procedure.
![Page 16: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/16.jpg)
Figure 8.2: Graph of Example 8.5
![Page 17: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/17.jpg)
The Lagrangian is
The necessary conditions are
together with (8.27) and (8.28). From (8.30) we get =0, since x-1/3 is never 0 in the range -1 x 1. Butsubstitution of =0 into (8.31) gives = - 1 < 0, which fails to solve (8.33).
![Page 18: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/18.jpg)
Example 8.6 Consider the problem:
subject to
The constraints are now differentiable, and theoptimum solution is (x*,y*)=(0,1) and h*=1. But onceagain the Kuhn-Tucker method fails, as we will see.The Lagrangian is
![Page 19: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/19.jpg)
so that the necessary conditions are
together with (8.35) and (8.36). From (8.41) we get,either y =0 or =0. Since y =0 minimizes the objectivefunction, we choose =0. From (8.38) we get either =0 or x=0. Since substitution of = =0 into (8.39)shows that it is unsolvable, we choose x =0, 0. Butthen (8.40) gives (1-y)3 = 0 or y =1. However, =0 and y =1 means that once more there is no solution to (8.39).
![Page 20: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/20.jpg)
The reason for failure of the method in Example 8.6 isthat the constraints do not satisfy what is called theconstraint qualification, which will be discussed in the next section.In order to motivate the definition we illustrate two different situation in figure 8.3. In figure 8.3(a) we show two boundary curves and intersecting the boundary point . The two tangentsto these curves are shown, and is a vector lyingbetween the two tangents. Starting at , there is adifferentiable curve drawn so that it liesentirely within the feasible set Y, such that its initialslope is equal to .
![Page 21: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/21.jpg)
Figure 8.3: Kuhn-Tucker Constraint Qualifications
![Page 22: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/22.jpg)
Whenever such a curve can be drawn from everyboundary point in Y and every contained betweenthe tangent lines, we say that the constraints definingY satisfy the Kuhn-Tucker constraint qualification.Figure 8.3(b) illustrates a case of a cusp at which theconstraint qualification does not hold. Here the twotangents to the graphs and coincide,so that 1 and 2 are vectors lying between these twotangents. Notice that for vector 1 , it is possible to findthe differentiable curve satisfying the abovecondition, but for vector 2 no such curve exists.Hence, the constraint qualification does not hold forthe example in figure 8.3(b).
![Page 23: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/23.jpg)
8.1.3 Theorems from Nonlinear ProgrammingWe first state the constraint qualification symbolically.For the problem defined by (8.1), (8.2), and (8.3), let Ybe the set of all feasible vectors satisfying (8.2) and(8.3), i.e.,
Let be any point of Y and let be the vector of tight constraints at point , i.e., z includes all the gconstraints in (8.2) and those constraints in (8.3)which are satisfied as equalities.
Define the set
![Page 24: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/24.jpg)
Then, we shall say that the constraints set Y satisfiesthe Kuhn-Tucker constraint qualification at ifz is differentiable at and if, for every , there exists a differentiable curve defined for such that
The Lagrangian function is
![Page 25: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/25.jpg)
The Kuhn-Tucker conditions at for this problemare
where and are row vectors of multipliers to bedetermined.
![Page 26: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/26.jpg)
Theorem 8.1 (Sufficient Conditions).If h,g, and w are differentiable, h is concave, g isaffine, w is concave, and solve the conditions(8.44)-(8.47), then is a solution to the maximization problem (8.1)-(8.3).
Theorem 8.2 (Necessary Conditions).If h,g, and w are differentiable, and solve the maximization problem, and the constraint qualificationholds at , then there exist multipliers and suchthat satisfy conditions (8.44)-(8.47).
![Page 27: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/27.jpg)
8.1.4 A Discrete-Time Optimal Control Problem
Here, the sate xk is assumed to be measured at thebeginning of period k and control uk is implementedduring period k. this convention is depicted in figure8.4.
![Page 28: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/28.jpg)
We also define continuously differentiable functions f: F:g: S:Then, a discrete-time optimal problem in the Bolzaform is
subject to the difference equations
In (8.49) the term is known as thedifference operator.
![Page 29: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/29.jpg)
8.1.5 A Discrete Maximum PrincipleThe Lagrangian function of the problem is
We now define the Hamiltonian function Hk to be
Using (8.52) we can rewrite (8.51) as
![Page 30: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/30.jpg)
If we differentiate (8.53) with respect to xk for k =1,2,…,T-1 , we obtain
which upon rearranging terms becomes
If we differentiate (8.53) with respect to xT , we get
The difference equations (8.54) with terminal boundaryconditions (8.55) are called adjoint equations.
![Page 31: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/31.jpg)
If we differentiate L with respect to k and state thecorresponding Kuhn-Tucker conditions for themultiplies k and constraint (8.50), we have
and
![Page 32: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/32.jpg)
We note that, if Hk is concave in k, isconcave in k, and the constraint qualification holds,then conditions (8.56) and (8.57) are precisely thenecessary and sufficient conditions for solving thefollowing Hamiltonian maximization problem:
![Page 33: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/33.jpg)
Theorem 8.3. If for every k , Hk in (8.52) and g(uk,k) arc concave in uk , and the constraint qualificationholds, then the necessary conditions for uk* ,k =0,1,…,T-1, to be an optimal control for the problem(8.48)-(8.50) are
![Page 34: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/34.jpg)
Example 8.7 Consider the discrete-time optimalcontrol problem:
subject to
We shall solve this problem for T=6 and T 7.Solution. The Hamiltonian is
![Page 35: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/35.jpg)
Let us assume, as we did in Example 2.3, that k < 0as long as xk is positive so that k=-1. Given this assumption, (8.61) becomes , whose solutionis
By differentiating (8.63), we obtain the adjoint equation
Let us assume T =6. Substitute (8.65) into (8.66) toobtain
![Page 36: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/36.jpg)
From Appendix A.11, we find the solution to be
where c is a constant. Since 6 = 0, we can obtain thevalue of c by setting k =6 in the above equation. Thus,
so that
A sketch of the value for k and xk appears in figure 8.5. Note that 5 =0, so that the control 4 is singular.However, since x4 =1, we choose 4 =-1 in order tobring x5 down to 0.
![Page 37: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/37.jpg)
Figure 8.5: Sketch of xk and k
![Page 38: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/38.jpg)
The solution of the problem for T 7 is carried out inthe same way that we solved example 2.3. Namely,observe that x5 =0 and 5 = 6 =0, so that the controlis singular. We simply make k =0 for k 7 so that k =0 for all k 7 . It is clear without a formal proof That this maximizes (8.60).
Example 8.8 Let us consider a discrete version of theproduction-inventory example of Section 6.1; seeKleindorfer (1975). Let Ik, Pk and Sk be the inventory,production, and demand at time k , respectively.Let I0 be the initial inventory, let and be the goal
![Page 39: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/39.jpg)
levels of inventory and production, and let h and c beinventory and production cost coefficients. Theproblem is:
subject to
Form the Hamiltonian
where the adjoint variable satisfies
![Page 40: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/40.jpg)
To maximize the Hamiltonian, let us differentiate (8.70)to obtain
Since production must be nonnegative, we obtain theoptimal production as
Expressions (8.69), (8.71), and (8.72) determine atwo-point boundary value problem. For a given set ofdata, it can be solved numerically by using aspreadsheet software EXCEL; see Section 2.5. If theconstraint Pk 0 is dropped it can be solvedanalytically by the method of Section 6.1, withdifference equations replacing the differential equations used there.
![Page 41: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/41.jpg)
8.2 A General Discrete Maximum Principle
subject to
Assumptions required are:(i) and are continuouslydifferentiable in xk for every uk and k .(ii) The sets are b-directionallyconvex for every x and k, where b =(-1,0,…,0). That is,given and w in and 0 1 , there exists
![Page 42: Chapter 8 The Maximum Principle: Discrete Time](https://reader036.vdocuments.us/reader036/viewer/2022062222/568167a1550346895ddceceb/html5/thumbnails/42.jpg)
such that
and
for every x and k. It should be noted that convexityimplies b-directional convexity, but not the converse.(iii) satisfies the Kuhn-Tucker constraintqualification.