nonlinear programming: concepts, algorithms and applications
TRANSCRIPT
![Page 1: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/1.jpg)
Nonlinear Programming: Concepts, Algorithms and
Applications
L. T. BieglerChemical Engineering Department
Carnegie Mellon UniversityPittsburgh, PA
![Page 2: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/2.jpg)
2
Introduction
Unconstrained Optimization• Algorithms• Newton Methods• Quasi-Newton Methods
Constrained Optimization• Karush Kuhn-Tucker Conditions• Special Classes of Optimization Problems• Reduced Gradient Methods (GRG2, CONOPT, MINOS)• Successive Quadratic Programming (SQP)• Interior Point Methods
Process Optimization• Black Box Optimization• Modular Flowsheet Optimization – Infeasible Path• The Role of Exact Derivatives
Large-Scale Nonlinear Programming• Data Reconciliation• Real-time Process Optimization
Further Applications• Sensitivity Analysis for NLP Solutions• Multiperiod Optimization Problems
Summary and Conclusions
Nonlinear Programming and Process Optimization
![Page 3: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/3.jpg)
3
Introduction
Optimization: given a system or process, find the best solution to this process within constraints.
Objective Function: indicator of "goodness" of solution, e.g., cost, yield, profit, etc.
Decision Variables: variables that influence process behavior and can be adjusted for optimization.
In many cases, this task is done by trial and error (through case study). Here, we are interested in a systematic approach to this task - and to make this task as efficient as possible.
Some related areas:
- Math programming
- Operations Research
Currently - Over 30 journals devoted to optimization with roughly 200 papers/month - a fast moving field!
![Page 4: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/4.jpg)
4
Optimization Viewpoints
Mathematician - characterization of theoretical properties of optimization, convergence, existence, local convergence rates.
Numerical Analyst - implementation of optimization method for efficient and "practical" use. Concerned with ease of computations, numerical stability, performance.
Engineer - applies optimization method to real problems. Concerned with reliability, robustness, efficiency, diagnosis, and recovery from failure.
![Page 5: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/5.jpg)
5
Optimization LiteratureEngineering
1. Edgar, T.F., D.M. Himmelblau, and L. S. Lasdon, Optimization of Chemical Processes, McGraw-Hill, 2001.
2. Papalambros, P. and D. Wilde, Principles of Optimal Design. Cambridge Press, 1988.
3. Reklaitis, G., A. Ravindran, and K. Ragsdell, Engineering Optimization, Wiley, 1983.
4. Biegler, L. T., I. E. Grossmann and A. Westerberg, Systematic Methods of Chemical Process Design, Prentice Hall, 1997.
Numerical Analysis
1. Dennis, J.E. and R. Schnabel, Numerical Methods of Unconstrained Optimization,Prentice-Hall, (1983), SIAM (1995)
2. Fletcher, R. Practical Methods of Optimization, Wiley, 1987.
3. Gill, P.E, W. Murray and M. Wright, Practical Optimization, Academic Press, 1981.
4. Nocedal, J. and S. Wright, Numerical Optimization, Springer, 1998
![Page 6: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/6.jpg)
6
Scope of optimizationProvide systematic framework for searching among a specified space of alternatives to identify an “optimal” design, i.e., as a decision-making tool
PremiseConceptual formulation of optimal product and process design corresponds to a mathematical programming problem
nyn yRx
yxg
yxhst
yxf
}1 ,0{
0),(
0),(
),(min
∈∈≤=
Motivation
![Page 7: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/7.jpg)
7
xxHybrid
xxNonlinear MPC
xLinear MPC
xxReal-time optimization
xxxSupply Chain
xxxxScheduling
xxFlowsheeting
xxx Equipment Design
x xxxReactors
xxSeparations
xxxxxxMENS
xxxxxxHENS
SA/GANLPLP,QPGlobalMINLPMILP
Optimization in Design, Operations and Control
![Page 8: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/8.jpg)
8
Unconstrained Multivariable Optimization
Problem: Min f(x) (n variables)
Equivalent to: Max -f(x), x ∈ Rn
Nonsmooth Functions- Direct Search Methods- Statistical/Random Methods
Smooth Functions- 1st Order Methods- Newton Type Methods- Conjugate Gradients
![Page 9: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/9.jpg)
9
Example : Optimal Vessel Dimensions
Min TC 2π D
2 + S C π DL = cost
s.t. V - 2πD L
4 = 0
Min TC 2π D
2 + S C
4V
D = cost
d(cost)
dD = TC π D - s4VC
2D = 0
D = 4V
π SC
TC
1/ 3
L =4V
π
1/3TC
SC
2/3
What is the optimal L/D ratio for a cylindrical vessel?Constrained Problem
(1)
Convert to Unconstrained (Eliminate L)
(2)
==> L/D = CT/CSNote:- What if L cannot be eliminated in (1) explicitly? (strange shape)- What if D cannot be extracted from (2)?
(cost correlation implicit)
![Page 10: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/10.jpg)
10
Two Dimensional Contours of F(x)
Convex Function Nonconvex Function Multimodal, Nonconvex
Discontinuous Nondifferentiable (convex)
![Page 11: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/11.jpg)
11
Local vs. Global Solutions
•Find a local minimum point x* for f(x) for feasible region defined by constraint functions: f(x*) � f(x) for all x satisfying the constraints in some neighborhood around x* (not for all x ∈ X)
•Finding and verifying global solutions will not be considered here.
•Requires a more expensive search (e.g. spatial branch and bound).
•A local solution to the NLP is also a global solution under the following sufficient conditions based on convexity.
• f(x) is convex in domain X, if and only if it satisfies:
f(α y + (1-α) z) � α f(y) + (1-α)f(z)
for any α, 0 � α � 1, at all points y and z in X.
![Page 12: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/12.jpg)
12
Linear Algebra - Background
Some Definitions• Scalars - Greek letters, α, β, γ• Vectors - Roman Letters, lower case• Matrices - Roman Letters, upper case• Matrix Multiplication:
C = A B if A ∈ ℜ n x m, B ∈ ℜ m x p and C ∈ ℜ n x p, Cij = Σk A ik Bkj
• Transpose - if A ∈ ℜ n x m, interchange rows and columns --> AT∈ ℜ m x n
• Symmetric Matrix - A ∈ ℜ n x n (square matrix) and A = AT
• Identity Matrix - I, square matrix with ones on diagonal and zeroes elsewhere.
• Determinant: "Inverse Volume" measure of a square matrix
det(A) = Σi (-1)i+j A ij A ij for any j, or
det(A) = Σj (-1)i+j A ij A ij for any i, where A ij is the determinantof an order n-1matrix with row i and column j removed. det(I) = 1
• Singular Matrix: det (A) = 0
![Page 13: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/13.jpg)
13
∇ f =
∂f /1∂x
∂f /2∂x
......
∂f / n∂x
2∇ f(x) =
2∂ f
12∂x
2∂ f
1∂x 2∂x⋅ ⋅ ⋅ ⋅
2∂ f
1∂x n∂x
.... .... ....2∂ f
n∂x 1∂x
2∂ f
n∂x 2∂x⋅ ⋅ ⋅ ⋅
2∂ f
n2∂x
2∂ f
∂ x j∂xi
2∂ f
j∂x i∂x
Gradient Vector- (∇ f(x))
Hessian Matrix(∇ 2f(x) - Symmetric)
Note: =
Linear Algebra - Background
![Page 14: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/14.jpg)
14
• Some Identities for Determinantdet(A B) = det(A) det(B); det (A) = det(AT)
det(αA) = αn det(A); det(A) = Π i λ i(A)
• Eigenvalues:det(A- λ I) = 0, Eigenvector: Av = λ vCharacteristic values and directions of a matrix.For nonsymmetric matrices eigenvalues can be complex, so we often use singular values,σ = λ(ATΑ)1/2 ≥ 0
• Vector Norms
|| x ||p = { Σi |xi|p} 1/p
(most common are p = 1, p = 2 (Euclidean) and p = ∞ (max norm = maxi|xi|))• Matrix Norms
||A|| = max ||A x||/||x|| over x (for p-norms)
||A||1 - max column sum of A, maxj (Σi |Aij|)||A||∞ - maximum row sum of A, maxi (Σj |Aij|)||A||2 = [σmax(Α)] (spectral radius)
||A||F = [Σi Σj (A ij)2]1/2 (Frobenius norm)κ(Α) = ||A|| ||A-1|| (condition number) = σmax/σmin(using 2-norm)
Linear Algebra - Background
![Page 15: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/15.jpg)
15
Find v and λ where Avi = λ i vi, i = i,nNote: Av - λv = (A - λI) v = 0 or det (A -λI) = 0For this relation λ is an eigenvalueand v is an eigenvectorof A.
If A is symmetric, all λ i are realλ i > 0, i = 1, n; A is positivedefiniteλ i < 0, i = 1, n; A is negativedefiniteλ i = 0, some i: A is singular
QuadraticFormcan be expressed in CanonicalForm(Eigenvalue/Eigenvector)xTAx ⇒ A V = V ΛV - eigenvector matrix (n x n)Λ - eigenvalue (diagonal) matrix = diag(λ i)
If A is symmetric, all λ i are realand V can be chosen orthonormal(V-1 = VT). Thus, A = V Λ V-1 = V Λ VT
For QuadraticFunction: Q(x) = aTx + ½ xTAx
Define: z = VTx and Q(Vz) = (aTV) z + ½ zT (VTAV)z= (aTV) z + ½ zT Λ z
Minimum occurs at (if λ i > 0) x = -A-1a or x = Vz = -V(Λ-1VTa)
Linear Algebra - Eigenvalues
![Page 16: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/16.jpg)
16
Positive (or Negative) CurvaturePositive (or Negative) Definite Hessian
Both eigenvalues are strictly positive or negative• A is positive definiteor negative definite• Stationary points are minima or maxima
![Page 17: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/17.jpg)
17
Zero Curvature Singular Hessian
One eigenvalue is zero, the other is strictly positive or negative• A is positive semidefiniteor negative semidefinite• There is a ridge of stationary points (minima or maxima)
![Page 18: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/18.jpg)
18
One eigenvalue is positive, the other is negative• Stationary point is a saddle point• A is indefinite
Note: these can also be viewed as two dimensional projections for higher dimensional problems
Indefinite Curvature Indefinite Hessian
![Page 19: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/19.jpg)
19
Eigenvalue Example
Min Q(x) =1
1
T
x + 1
2xT 2 1
1 2
x
AV = VΛ with A = 2 1
1 2
VT AV = Λ =1 0
0 3
with V =
1/ 2 1/ 2
-1/ 2 1/ 2
• All eigenvalues are positive
• Minimum occurs at z* = -/-1VTa
z=VT x = (x1 − x2) / 2
(x1 + x2) / 2
x =Vz= (x1 + x2) / 2
(−x1 + x2) / 2
z* =0
2 /(3 2)
x* =
1/3
1/3
![Page 20: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/20.jpg)
20
1. Convergence Theory• Global Convergence - will it converge to a local optimum (or stationary point) from a poor starting point?• Local Convergence Rate - how fast will it converge close to the solution?
2. Benchmarks on Large Class of Test Problems
Representative Problem (Hughes, 1981)
Min f(x1, x2) = α exp(-β)u = x1 - 0.8v = x2 - (a1 + a2 u2 (1- u)1/2 - a3 u)α = -b1 + b2 u2 (1+u)1/2 + b3 uβ = c1 v2 (1 - c2 v)/(1+ c3 u2)
a = [ 0.3, 0.6, 0.2]b = [5, 26, 3]c = [40, 1, 10]x* = [0.7395, 0.3144] f(x*) = 5.0893
Comparison of Optimization Methods
![Page 21: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/21.jpg)
21
Three Dimensional Surface and Curvature for Representative Test Problem
Regions where minimum eigenvalue is less than:
[0, -10, -50, -100, -150, -200]
![Page 22: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/22.jpg)
22
What conditions characterize an optimal solution?
x1
x2
x*
Contours of f(x)
Unconstrained Local MinimumNecessary Conditions
∇ f (x*) = 0pT∇ 2f (x*) p � 0 for p∈ℜ n
(positive semi-definite)
Unconstrained Local MinimumSufficient Conditions
∇ f (x*) = 0pT∇ 2f (x*) p > 0 for p∈ℜ n
(positive definite)
Since ∇ f(x*) = 0, f(x) is purelyquadraticfor x close to x*
*)*)((*)(2
1*)(*)(*)()( 2 xxxfxxxxxfxfxf TT −∇−+−∇+=
For smooth functions, why are contours around optimum elliptical?Taylor Seriesin n dimensions about x*:
![Page 23: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/23.jpg)
23
Taylor Series for f(x) about xk
Take derivative wrt x, set LHS § 0
0 §∇ f(x) = ∇ f(xk) + ∇ 2f(xk) (x - xk)⇒ (x - xk) ≡ d = - (∇ 2f(xk))-1 ∇ f(xk)
• f(x) is convex (concave) if for all x ∈ℜ n, ∇ 2f(x) is positive (negative) semidefinitei.e. minj λ j ≥ 0 (maxj λ j � 0)
• Method can fail if:- x0 far from optimum- ∇ 2f is singular at any point- f(x) is not smooth
• Search direction, d, requires solution of linear equations.• Near solution:
k+1x - x * = K kx - x * 2
Newton’s Method
![Page 24: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/24.jpg)
24
0. Guess x0, Evaluate f(x0).
1. At xk, evaluate ∇ f(xk).
2. Evaluate Bk = ∇ 2f(xk) or an approximation.
3. Solve: Bk d = -�f(xk)If convergence error is less than tolerance:e.g., ||∇ f(xk) || ≤ ε and ||d|| ≤ ε STOP, else go to 4.
4. Find α so that 0 < α ≤ 1 and f(xk + α d) < f(xk)sufficiently (Each trial requires evaluation of f(x))
5. xk+1 = xk + α d. Set k = k + 1 Go to 1.
Basic Newton Algorithm - Line Search
![Page 25: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/25.jpg)
25
Newton’s Method - Convergence Path
Starting Points[0.8, 0.2] needs steepest descent steps w/ line search up to ’O’, takes 7 iterations to ||∇ f(x*)|| � 10-6
[0.35, 0.65] converges in four iterations with full steps to ||∇ f(x*)|| � 10-6
![Page 26: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/26.jpg)
26
• Choice of Bk determines method.- Steepest Descent: Bk = γ I- Newton: Bk = ∇ 2f(x)
• With suitable Bk, performance may be good enough if f(xk + αd)is sufficiently decreased (instead of minimized along line search direction).
• Trust region extensionsto Newton's method provide very strong global convergence properties and very reliable algorithms.
• Local rate of convergence depends on choice of Bk.
Newton’s Method - Notes
Newton−Quadratic Rate : limk→∞
xk+1 − x *
xk − x *2 = K
Steepest descent− Linear Rate : limk→∞
xk+1 − x *
xk − x *<1
Desired?− Superlinear Rate : limk→∞
xk+1 − x *
xk − x *= 0
![Page 27: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/27.jpg)
27
k+1B = kB + y - kB s( ) Ty + y y - kB s( )T
Ty s -
y - kB s( )Ts y TyTy s( ) Ty s( )
k+1
Bk+1( )-1 = H = kH +
TssTs y
- kH y Ty kH
ky H y
Motivation: • Need Bk to be positive definite. • Avoid calculation of ∇ 2f. • Avoid solution of linear system for d = - (Bk)-1 ∇ f(xk)
Strategy: Define matrix updating formulas that give (Bk) symmetric, positive definite andsatisfy:
(Bk+1)(xk+1 - xk) = (∇ fk+1 - ∇ fk) (Secant relation)
DFP Formula:(Davidon, Fletcher, Powell, 1958, 1964)
where: s = xk+1- xk
y = ∇ f (xk+1) - ∇ f (xk)
Quasi-Newton Methods
![Page 28: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/28.jpg)
28
k+1B = kB + Tyy
Ts y -
kB s Ts kBks B s
k+1B( )−1= k+1H = kH +
s - kH y( ) Ts + s s - kH y( )TTy s
- y - kH s( )T y s Ts
Ty s( ) Ty s( )
BFGS Formula(Broyden, Fletcher, Goldfarb, Shanno, 1970-71)
Notes:1) Both formulas are derived under similarassumptionsand have
symmetry2) Both have superlinearconvergenceand terminate in n steps on
quadratic functions. They are identical if α is minimized. 3) BFGS is more stable and performs better than DFP, in general.4) For n ≤ 100, these are the bestmethods for general purpose
problems if second derivatives are not available.
Quasi-Newton Methods
![Page 29: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/29.jpg)
29
Quasi-Newton Method - BFGSConvergence Path
Starting Point[0.8, 0.2] starting from B0 = I , converges in 9 iterations to ||∇ f(x*)|| � 10-6
![Page 30: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/30.jpg)
30
Harwell (HSL)
IMSL
NAg - Unconstrained Optimization Codes
Netlib (www.netlib.org)
•MINPACK
•TOMS Algorithms, etc.
These sources contain various methods
•Quasi-Newton
•Gauss-Newton
•Sparse Newton
•Conjugate Gradient
Sources For Unconstrained Software
![Page 31: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/31.jpg)
31
Problem: Minx f(x)s.t. g(x) ≤ 0
h(x) = 0where:
f(x) - scalar objective functionx - n vector of variables
g(x) - inequality constraints, m vectorh(x) - meqequality constraints.
Sufficient Condition for Unique Optimum- f(x) must be convex, and- feasible region must be convex,
i.e. g(x) are all convexh(x)are all linear
Except in special cases, ther is noguaranteethat a localoptimumis globalif sufficient conditions are violated.
Constrained Optimization(Nonlinear Programming)
![Page 32: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/32.jpg)
32
23
1
A
B
y
x
1x , 1y ≥ 1R 1x ≤ B- 1R , 1y ≤ A - 1R
2, x 2 y ≥ 2R 2x ≤ B- 2R , 2 y ≤ A - 2R
3,x 3 y ≥ 3R 3x ≤ B- 3R , 3 y ≤ A - 3R
1x - 2x( )2 + 1y - 2y( )2 ≥ 1R + 2R( )2
1x - 3x( )2 + 1y - 3y( )2 ≥ 1R + 3R( )2
2x - 3x( )2 + 2y - 3y( )2 ≥ 2R + 3R( )2
Example: Minimize Packing Dimensions
What is the smallest box for three round objects?Variables: A, B, (x1, y1), (x2, y2), (x3, y3)Fixed Parameters: R1, R2, R3
Objective: Minimize Perimeter = 2(A+B)Constraints: Circles remain in box, can’t overlapDecisions: Sides of box, centers of circles.
no overlapsin box
x1, x2, x3, y1, y2, y3, A, B ≥ 0
![Page 33: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/33.jpg)
33
Min
Linear Program
Min
Linear Program(Alternate Optima)
Min
Min
Min
Convex Objective FunctionsLinear Constraints
Min
Min
Min
Nonconvex RegionMultiple Optima
MinMin
Nonconvex ObjectiveMultiple Optima
Characterization of Constrained Optima
![Page 34: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/34.jpg)
34
What conditions characterize an optimal solution?
Unconstrained Local MinimumNecessary Conditions
∇ f (x*) = 0pT∇ 2f (x*) p � 0 for p∈ℜ n
(positive semi-definite)
Unconstrained Local MinimumSufficient Conditions
∇ f (x*) = 0pT∇ 2f (x*) p > 0 for p∈ℜ n
(positive definite)
![Page 35: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/35.jpg)
35
Optimal solution for inequality constrained problem
Min f(x)s.t. g(x) � 0
Analogy: Ball rolling down valley pinned by fenceNote: Balance of forces (∇ f, ∇ g1)
![Page 36: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/36.jpg)
36
Optimal solution for general constrained problem
Problem: Min f(x)s.t. g(x) � 0
h(x) = 0Analogy: Ball rolling on rail pinned by fencesBalance of forces: ∇ f, ∇ g1, ∇ h
![Page 37: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/37.jpg)
37
Necessary First Order Karush Kuhn - Tucker Conditions
∇ L (x*, u, v) = ∇ f(x*) + ∇ g(x*) u + ∇ h(x*) v = 0(Balance of Forces)u � 0 (Inequalities act in only one direction)g (x*) � 0, h (x*) = 0 (Feasibility)uj gj(x*) = 0 (Complementarity: either gj(x*) = 0 or uj = 0)u, vare "weights" for "forces," known as KKT multipliers, shadow prices, dual variables
“To guarantee that a local NLP solution satisfies KKT conditions, a constraint qualification is required. E.g., the Linear Independence Constraint Qualification(LICQ) requires active constraint gradients, [∇ gA(x*) ∇ h(x*)], to be linearlyindependent. Also, under LICQ, KKT multipliers are uniquely determined.”
Necessary (Sufficient) SecondOrderConditions- Positive curvature in "constraint" directions.- pT∇ 2L (x*) p ≥ 0 (pT∇ 2L (x*) p > 0)
where p are the constrained directions: ∇ gA(x*)Tp = 0, ∇ h(x*)Tp = 0
Optimality conditions for local optimum
![Page 38: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/38.jpg)
38
Single Variable Example of KKT Conditions
-a a
f(x)
x
Min (x)2 s.t. -a � x � a, a > 0x* = 0 is seen by inspection
Lagrange function :L(x, u) = x2 + u1(x-a) + u2(-a-x)
First Order KKT conditions:∇ L(x, u) = 2 x + u1 - u2 = 0u1 (x-a) = 0u2 (-a-x) = 0-a � x � a u1, u2 � 0
Consider three cases:• u1 > 0, u2 = 0 Upper bound is active, x = a, u1 = -2a, u2 = 0• u1 = 0, u2 > 0 Lower bound is active, x = -a, u2 = -2a, u1 = 0• u1 = u2 = 0 Neither bound is active, u1 = 0, u2 = 0, x = 0
Second order conditions (x*, u1, u2 =0) ∇ xxL (x*, u*) = 2 pT∇ xxL (x*, u*) p = 2 (∆x)2 > 0
![Page 39: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/39.jpg)
39
Single Variable Example of KKT Conditions - Revisited
Min -(x)2 s.t. -a � x � a, a > 0x* = ±a is seen by inspection
Lagrange function :L(x, u) = -x2 + u1(x-a) + u2(-a-x)
First Order KKT conditions:∇ L(x, u) = -2x + u1 - u2 = 0u1 (x-a) = 0u2 (-a-x) = 0-a � x � a u1, u2 � 0
Consider three cases:• u1 > 0, u2 = 0 Upper bound is active, x = a, u1 = 2a, u2 = 0• u1 = 0, u2 > 0 Lower bound is active, x = -a, u2 = 2a, u1 = 0• u1 = u2 = 0 Neither bound is active, u1 = 0, u2 = 0, x = 0
Second order conditions (x*, u1, u2 =0) ∇ xxL (x*, u*) = -2 pT∇ xxL (x*, u*) p = -2(∆x)2 < 0
a-a
f(x)
x
![Page 40: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/40.jpg)
40
For x = a or x = -a, we require the allowable direction to satisfy the active constraints exactly. Here, any point along the allowable direction, x* must remain at its bound.
For this problem, however, there are no nonzero allowable directions that satisfy this condition. Consequently the solution x* is defined entirely by the active constraint. The condition:
pT ∇ xxL (x*, u*, v*) p > 0for all allowable directions, is vacuously satisfied - because there are noallowable directions that satisfy ∇ gA(x*)T p = 0. Hence, sufficientsecond order conditions are satisfied.
As we will see, sufficient second order conditions are satisfied by linear programs as well.
Interpretation of Second Order Conditions
![Page 41: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/41.jpg)
41
Role of KKT Multipliers
a-a
f(x)
x a + ∆a
Also known as:• Shadow Prices• Dual Variables• Lagrange Multipliers
Suppose a in the constraint is increased to a + ∆af(x*) = (a + ∆a)2
and[f(x*, a + ∆a) - f(x*, a)]/∆a = 2a + ∆a
df(x*)/da = 2a = u1
![Page 42: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/42.jpg)
42
Linear Programming:Min cTxs.t. Ax � b
Cx = d, x � 0Functions are all convex ⇒ global min.Because of Linearity, can prove solution will always lie at vertex of feasible region.
x2
x1
Simplex Method- Start at vertex- Move to adjacent vertex that offers most improvement- Continue until no further improvement
Notes: 1) LP has wide uses in planning, blending and scheduling2) Canned programs widely available.
Special Cases of Nonlinear Programming
![Page 43: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/43.jpg)
43
Simplex MethodMin -2x1 - 3x2 Min -2x1 - 3x2
s.t. 2x1 + x2 � 5 ⇒ s.t. 2x1 + x2 + x3 = 5x1, x2 � 0 x1, x2, x3 � 0
(add slack variable)Now, define f = -2x1 - 3x2 ⇒ f + 2x1 + 3x2 = 0Set x1, x2 = 0, x3 = 5 and form tableau
x1 x2 x3 f b x1, x2 nonbasic2 1 1 0 5 x3 basic2 3 0 1 0
To decrease f, increase x2. How much? so x3 � 0x1 x2 x3 f b2 1 1 0 5-4 0 -3 1 -15
f can no longer be decreased! Optimal
Underlined terms are -(reduced gradients); nonbasicvariables (x1, x3), basic variable x2
Linear Programming Example
![Page 44: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/44.jpg)
44
Problem: Min aTx + 1/2 xT B xA x � bC x = d
1) Can be solved using LP-like techniques: (Wolfe, 1959)
⇒ Min Σj (zj+ + zj-)s.t. a + Bx + ATu + CTv = z+ - z-
Ax - b + s = 0Cx - d = 0s, z+, z- � 0{u j sj = 0}
with complicating conditions.
2) If B is positive definite, QP solution is unique.If B is pos. semidefinite, optimum value is unique.
3) Other methods for solving QP’s (faster)- Complementary Pivoting (Lemke)- Range, Null Space methods (Gill, Murray).
Quadratic Programming
![Page 45: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/45.jpg)
45
iµ = 1
T ir
t=1
T
∑ (t)
Definitions:xi - fraction or amount invested in security iri (t) - (1 + rate of return) for investment i in year t.µi - average r(t) over T years, i.e.
Note: maximize average return, no accounting for risk.
Portfolio Planning Problem
. ,0
1 .t.
etcx
xs
xMax
i
ii
iii
≥
=∑∑µ
![Page 46: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/46.jpg)
46
ijS{ } = ij2σ =
1
T ir (t) - iµ( )
t =1
T
∑ jr (t) - jµ( )
S =
3 1 - 0.5
1 2 0.4
-0.5 0.4 1
Definition of Risk - fluctuation of ri(t) over investment (or past) time period.To minimize risk, minimize variance about portfolio mean (risk averse).
Variance/Covariance Matrix, S
Example: 3 investmentsµj
1. IBM 1.32. GM 1.23. Gold 1.08
Portfolio Planning Problem
. ,0
1 .t.
etcx
Rx
xs
SxxMax
i
iii
ii
T
≥
≥
=
∑∑
µ
![Page 47: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/47.jpg)
47
SIMPLE PORTFOLIO INVESTMENT PROBLEM (MARKOWITZ)45 OPTION LIMROW=0;6 OPTION LIMXOL=0;78 VARIABLES IBM, GM, GOLD, OBJQP, OBJLP;910 EQUATIONS E1,E2,QP,LP;1112 LP.. OBJLP =E= 1.3*IBM + 1.2*GM + 1.08*GOLD;1314 QP.. OBJQP =E= 3*IBM**2 + 2*IBM*GM - IBM*GOLD15 + 2*GM**2 - 0.8*GM*GOLD + GOLD**2;1617 E1..1.3*IBM + 1.2*GM + 1.08*GOLD =G= 1.15;1819 E2.. IBM + GM + GOLD =E= 1;2021 IBM.LO = 0.;22 IBM.UP = 0.75;23 GM.LO = 0.;24 GM.UP = 0.75;25 GOLD.LO = 0.;26 GOLD.UP = 0.75;2728 MODEL PORTQP/QP,E1,E2/;2930 MODEL PORTLP/LP,E2/;3132 SOLVE PORTLP USING LP MAXIMIZING OBJLP;3334 SOLVE PORTQP USING NLP MINIMIZING OBJQP;
Portfolio Planning Problem - GAMS
![Page 48: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/48.jpg)
48
S O L VE S U M M A R Y**** MODEL STATUS 1 OPTIMAL**** OBJECTIVE VALUE 1.2750RESOURCE USAGE, LIMIT 1.270 1000.000ITERATION COUNT, LIMIT 1 1000BDM - LP VERSION 1.01A. Brooke, A. Drud, and A. Meeraus,Analytic Support Unit,Development Research Department,World Bank,Washington D.C. 20433, U.S.A.
Estimate work space needed - - 33 KbWork space allocated - - 231 KbEXIT - - OPTIMAL SOLUTION FOUND.
LOWER LEVEL UPPER MARGINAL- - - - EQU LP . . . 1.000- - - - EQU E2 1.000 1.000 1.000 1.200
LOWER LEVEL UPPER MARGINAL- - - - VAR IBM 0.750 0.750 0.100- - - - VAR GM . 0.250 0.750 .- - - - VAR GOLD . .. 0.750 -0.120- - - - VAR OBJLP -INF 1.275 +INF .**** REPORT SUMMARY : 0 NONOPT
0 INFEASIBLE0 UNBOUNDED
SIMPLE PORTFOLIO INVESTMENT PROBLEM (MARKOWITZ)Model Statistics SOLVE PORTQP USING NLP FROM LINE 34MODEL STATISTICSBLOCKS OF EQUATIONS 3 SINGLE EQUATIONS 3BLOCKS OF VARIABLES 4 SINGLE VARIABLES 4NON ZERO ELEMENTS 10 NON LINEAR N-Z 3DERIVITIVE POOL 8 CONSTANT POOL 3CODE LENGTH 95GENERATION TIME = 2.360 SECONDSEXECUTION TIME = 3.510 SECONDS
Portfolio Planning Problem - GAMS
![Page 49: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/49.jpg)
49
S O L VE S U M M A R YMODEL PORTLP OBJECTIVE OBJLPTYPE LP DIRECTION MAXIMIZESOLVER MINOS5 FROM LINE 34**** SOLVER STATUS 1 NORMAL COMPLETION**** MODEL STATUS 2 LOCALLY OPTIMAL**** OBJECTIVE VALUE 0.4210RESOURCE USAGE, LIMIT 3.129 1000.000ITERATION COUNT, LIMIT 3 1000EVALUATION ERRORS 0 0M I N O S 5.3 (Nov. 1990) Ver: 225-DOS-02B.A. Murtagh, University of New South Walesand
P.E. Gill, W. Murray, M.A. Saunders and M.H. WrightSystems Optimization Laboratory, Stanford University.
EXIT - - OPTIMAL SOLUTION FOUNDMAJOR ITNS, LIMIT 1FUNOBJ, FUNCON CALLS 8SUPERBASICS 1INTERPRETER USAGE .21NORM RG / NORM PI 3.732E-17
LOWER LEVEL UPPER MARGINAL- - - - EQU QP . . . 1.000- - - - EQU E1 1.150 1.150 +INF 1.216- - - - EQU E2 1.000 1.000 1.000 -0.556
LOWER LEVEL UPPER MARGINAL- - - - VAR IBM . 0.183 0.750 .- - - - VAR GM . 0.248 0.750 EPS- - - - VAR GOLD . 0.569 0.750 .- - - - VAR OBJLP -INF 1.421 +INF .**** REPORT SUMMARY : 0 NONOPT
0 INFEASIBLE0 UNBOUNDED0 ERRORS
SIMPLE PORTFOLIO INVESTMENT PROBLEM (MARKOWITZ)Model Statistics SOLVE PORTQP USING NLP FROM LINE 34EXECUTION TIME = 1.090 SECONDS
Portfolio Planning Problem - GAMS
![Page 50: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/50.jpg)
50
Motivation: Build on unconstrained methods wherever possible.
Classification of Methods:
•Reduced Gradient Methods - (with Restoration) GRG2, CONOPT •Reduced Gradient Methods - (without Restoration) MINOS•Successive Quadratic Programming - generic implementations•Penalty Functions - popular in 1970s, but fell into disfavor. Barrier Methods have been developed recently and are again popular.•Successive Linear Programming - only useful for "mostly linear" problems
We will concentrate on algorithms for first four classes.
Evaluation: Compare performance on "typical problem," cite experience on process problems.
Algorithms for Constrained Problems
![Page 51: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/51.jpg)
51
Representative Constrained Problem (Hughes, 1981)
g1 Š 0
g2 Š 0
Min f(x1, x2) = α exp(-β)g1 = (x2+0.1)2[x1
2+2(1-x2)(1-2x2)] - 0.16 � 0g2 = (x1 - 0.3)2 + (x2 - 0.3)2 - 0.16 � 0x* = [0.6335, 0.3465] f(x*) = -4.8380
![Page 52: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/52.jpg)
52
Min f(x) Min f(z)
s.t. g(x) + s = 0 (add slack variable) ‘⇒ s.t. h(z) = 0
c(x) = 0 a � z � b
a � x � b, s � 0
• Partition variables into:
zB - dependent or basicvariables
zN - nonbasicvariables, fixed at a bound
zS - independent or superbasic variables
Analogy to linear programming. Superbasics required only if nonlinear problem
• Solve unconstrained problem in space of superbasicvariables.
Let zT = [zST zB
T zNT] optimize wrtzS with h(zS, zB , zN) = 0
Calculateconstrained derivativeor reduced gradientwrt zS.
•Dependent variables are zB ∈ Rm
•Nonbasic variables zN (temporarily) fixed
Reduced Gradient Method with Restoration(GRG2/CONOPT)
![Page 53: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/53.jpg)
53
•By remaining feasible always, h(z) = 0, a � z � b, one can apply an unconstrained algorithm (quasi-Newton) using (df/dzS)
•Solve problem in reduced space of zS variables
Definition of Reduced Gradient
[ ]
[ ]B
zzSS
zzBSS
B
B
T
BS
T
S
BS
B
SS
z
fhh
z
f
dz
df
hhz
h
z
h
dz
dz
dzz
hdz
z
hdh
zh
z
f
dz
dz
z
f
dz
df
BS
BS
∂∂∇∇−
∂∂=
∇−∇=
∂∂
∂∂−=
=
∂∂+
∂∂=
=∂∂+
∂∂=
−
−−
1
1
1
: toleads This
0
:have we,0)( Because
![Page 54: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/54.jpg)
54
If ∇ hT is (m x n); ∇ zShT is m x (n-m); ∇ zBhT is (m x m)
(df/dzS) is the change in f along constraint direction per unit change in zS
Example of Reduced Gradient
[ ]
[ ] ( ) 2/322-432
Let
2]- 2[ 4], 3[
2443 ..
2
11
11
1
21
1
21
22
1
+=−=
∂∂∇∇−
∂∂=
==
=∇=∇
=+−
−
−
xxdx
df
z
fhh
z
f
dz
df
x, zxz
xfh
xxts
xxMin
Bzz
SS
BS
TT
BS
![Page 55: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/55.jpg)
55
Sketch of GRG Algorithm
1.1. Initialize problem and obtain a feasible point at zInitialize problem and obtain a feasible point at z00
2.2. At feasible point At feasible point zzkk, partition variables , partition variables zz into into zzNN, , zzBB, , zzSS
3.3. Calculate reduced gradient, Calculate reduced gradient, ((df/dzdf/dzSS))
4.4. Evaluate search direction for Evaluate search direction for zzSS, , d = Bd = B--11(df/dz(df/dzSS) )
5.5. Perform a line search.Perform a line search.
•• Find Find αα∈ (0,1] with zzSS := := zzSSkk + + αα dd
•• Solve for Solve for h(zh(zSSkk + + αα d, d, zzBB, , zzNN) = 0) = 0
•• If If f(zf(zSSkk + + αα dd, , zzBB, , zzNN) ) < < f(zf(zSS
kk, , zzBB, , zzNN), ), setsetzzSS
k+1 k+1 ==zzSSkk + + αα d, k:= k+1d, k:= k+1
6.6. If ||If ||((df/dzdf/dzSS)||<)||< ε, ε, Stop. Else, go to 2. Stop. Else, go to 2.
![Page 56: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/56.jpg)
56
1. GRG2 has been implemented on PC’s as GINO and is very reliable and
robust. It is also the optimization solver in MS EXCEL.
2. CONOPT is implemented in GAMS, AIMMS and AMPL
3. GRG2 uses Q-N for small problems but can switch to conjugate
gradients if problem gets large. CONOPT uses exact second derivatives.
4. Convergence of h(zS, zB , zN) = 0 can get veryexpensive because ∇ h is
required
5. Safeguards can be added so that restoration (step 5.) can be dropped
and efficiency increases.
Representative Constrained Problem Starting Point [0.8, 0.2]
• GINO Results- 14 iterations to ||∇ f(x*)|| � 10-6
• CONOPT Results- 7 iterations to ||∇ f(x*)|| � 10-6 from feasible point.
GRG Algorithm Properties
![Page 57: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/57.jpg)
57
Motivation: Efficient algorithms are available that solve linearly constrained optimization problems (MINOS):
Min f(x)s.t. Ax � b
Cx = d
Extend to nonlinear problems, through successive linearization
Develop major iterations (linearizations) and minor iterations (GRG solutions) .
Reduced Gradient Method without Restoration (MINOS/Augmented)
Strategy: (Robinson, Murtagh & Saunders)1. Partitionvariables into basic, nonbasic
variables and superbasic variables..
2. Linearizeactive constraints at zk
Dkz = ck
3. Let ψ = f (z) + vTh (z) + β (hTh)(Augmented Lagrange),
4. Solve linearly constrained problem:Min ψ (z)s.t. Dz = c
a � z � busing reduced gradients to get zk+1
5. Set k=k+1, go to 3.6. Algorithm terminates when no
movement between steps 3) and 4).
![Page 58: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/58.jpg)
58
1. MINOS has been implemented very efficiently to take care of linearity. It becomes LP Simplex method if problem is totally linear. Also, very efficient matrix routines.
2. No restoration takes place, nonlinear constraints arereflected in ψ(z) during step 3). MINOS is more efficient than GRG.
3. Major iterations (steps 3) - 4)) converge at a quadraticrate.4. Reduced gradient methods are complicated, monolithic codes:
hard to integrate efficiently into modeling software.
Representative Constrained Problem– Starting Point [0.8, 0.2]MINOS Results: 4 major iterations, 11 function calls
to ||∇ f(x*)|| � 10-6
MINOS/Augmented Notes
![Page 59: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/59.jpg)
59
Motivation:• Take KKT conditions, expand in Taylor series about current point.• Take Newton step (QP) to determine next point.
Derivation– KKT Conditions∇ xL (x*, u*, v*) = ∇ f(x*) + ∇ gA(x*) u* + ∇ h(x*) v* = 0
h(x*) = 0 gA(x*) = 0, where gA are the activeconstraints.
Newton- Step
xx∇ LA
g∇ ∇ h
Ag∇ T 0 0
∇ hT 0 0
∆x
∆u
∆v
= -
x∇ L kx , ku , kv( )Ag kx( )h kx( )
Requirements:• ∇ xxL must be calculated and should be ‘regular’•correct active set gA
•good estimates of uk, vk
Successive Quadratic Programming (SQP)
![Page 60: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/60.jpg)
60
1. Wilson (1963)- active set can be determined by solving QP:
Min ∇ f(xk)Td + 1/2 dT ∇ xx L(xk, uk, vk) d
ds.t. g(xk) + ∇ g(xk)
T d � 0h(xk) + ∇ h(xk)
T d = 0
2. Han (1976), (1977), Powell (1977), (1978)- approximate ∇ xxL using a positive definite quasi-Newton update (BFGS)- use a line search to converge from poor starting points.
Notes:- Similar methods were derived using penalty (not Lagrange) functions.- Method converges quickly; very few function evaluations.- Not well suited to large problems (full space update used).
For n > 100, say, use reduced space methods (e.g. MINOS).
SQP Chronology
![Page 61: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/61.jpg)
61
What about ∇ xxL?• need to get second derivatives for f(x), g(x), h(x).• need to estimate multipliers, uk, vk; ∇ xxL may not be positive
semidefinite⇒ Approximate ∇ xxL (xk, uk, vk) by Bk, a symmetric positive
definite matrix.
BFGS Formula s = xk+1 - xk
y = ∇ L(xk+1, uk+1, vk+1) - ∇ L(xk, uk+1, vk+1)• second derivatives approximated by change in gradients• positive definite Bk ensures unique QP solution
Elements of SQP – Hessian Approximation
k+1B = kB + Tyy
Ts y -
kB s Ts kBks B s
![Page 62: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/62.jpg)
62
How do we obtain search directions?• Form QP and let QP determine constraint activity• At each iteration, k, solve:
Min ∇ f(xk) Td + 1/2 dT Bkdd
s.t. g(xk) + ∇ g(xk) T d � 0h(xk) + ∇ h(xk) T d = 0
Convergence from poor starting points• As with Newton's method, choose α (stepsize) to ensure progress
toward optimum: xk+1 = xk + α d.• α is chosen by making sure a merit function is decreased at each
iteration.Exact Penalty Functionψ(x) = f(x) + µ [Σ max (0, gj(x)) + Σ |hj (x)|]
µ > maxj {| uj |, | vj |}Augmented Lagrange Functionψ(x) = f(x) + uTg(x) + vTh(x)
+ η/2 {Σ (hj (x))2 + Σ max (0, gj (x))2}
Elements of SQP – Search Directions
![Page 63: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/63.jpg)
63
Fast Local ConvergenceB = ∇ xxL Quadratic∇ xxL is p.d and B is Q-N 1 step SuperlinearB is Q-N update, ∇ xxL not p.d 2 step Superlinear
Enforce Global ConvergenceEnsure decrease of merit function by taking α � 1Trust region adaptations provide a stronger guarantee of global convergence - but harder to implement.
Newton-Like Properties for SQP
![Page 64: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/64.jpg)
64
0. Guessx0, Set B0 = I (Identity). Evaluate f(x0), g(x0) and h(x0).
1. At xk, evaluate ∇ f(xk), ∇ g(xk), ∇ h(xk).
2. If k > 0, updateBk using the BFGS Formula.
3. Solve: Mind ∇ f(xk)Td + 1/2 dTBkd
s.t. g(xk) + ∇ g(xk)Td ≤ 0h(xk) + ∇ h(xk)Td = 0
If KKT error less than tolerance:||∇ L(x*)|| ≤ ε, ||h(x*)|| ≤ ε,
||g(x*)+||≤ ε. STOP, else go to 4.
4. Find α so that 0 < α ≤ 1 and ψ(xk + αd) < ψ(xk) sufficiently
(Each trial requires evaluation of f(x), g(x)and h(x)).
5. xk+1 = xk + α d. Set k = k + 1Go to 2.
Basic SQP Algorithm
![Page 65: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/65.jpg)
65
Nonsmooth Functions- ReformulateIll-conditioning- Proper scalingPoor Starting Points – Trust Regions can helpInconsistent Constraint Linearizations- Can lead to infeasible QP's
x2
x1
Min x2
s.t. 1 + x1 - (x2)2 � 01 - x1 - (x2)2 � 0
x2 � -1/2
Problems with SQP
![Page 66: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/66.jpg)
66
SQP Test Problem
1.21.00.80.60.40.20.00.0
0.2
0.4
0.6
0.8
1.0
1.2
x1
x2
x*
Min x2s.t. -x2 + 2 x1
2 - x13 � 0
-x2 + 2 (1-x1)2 - (1-x1)3 � 0x* = [0.5, 0.375].
![Page 67: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/67.jpg)
67
SQP Test Problem – First Iteration
1.21.00.80.60.40.20.00.0
0.2
0.4
0.6
0.8
1.0
1.2
x1
x2
Start from the origin (x0 = [0, 0] T) with B0 = I , form:
Min d2 + 1/2 (d12 + d2
2)s.t. d2 � 0
d1 + d2 � 1d = [1, 0]T. with µ1 = 0 and µ2 = 1.
![Page 68: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/68.jpg)
68
1.21.00.80.60.40.20.00.0
0.2
0.4
0.6
0.8
1.0
1.2
x1
x2
x*
From x1 = [0.5, 0]T with B1 = I (no update from BFGS possible), form:
Min d2 + 1/2 (d12 + d2
2)s.t. -1.25 d1 - d2 + 0.375 � 0
1.25 d1 - d2 + 0.375 � 0d = [0, 0.375]T with µ1 = 0.5 and µ2 = 0.5
x* = [0.5, 0.375]T is optimal
SQP Test Problem – Second Iteration
![Page 69: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/69.jpg)
69
Representative Constrained Problem SQP Convergence Path
g1 Š 0
g2 Š 0
Starting Point[0.8, 0.2] - starting from B0 = I and staying in bounds
and linearized constraints; converges in 8 iterations to ||∇ f(x*)|| � 10-6
![Page 70: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/70.jpg)
70
Barrier Methods for Large-Scale Nonlinear Programming
0
0)(s.t
)(min
≥=
ℜ∈
x
xc
xfnx
Original Formulation
0)(s.t
ln)()( min1
=
−= ∑=ℜ∈
xc
xxfxn
ii
x nµϕµBarrier Approach
Can generalize for
bxa ≤≤
⇒ As µ Î 0, x*(µ) Î x* Fiacco and McCormick (1968)
![Page 71: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/71.jpg)
71
Solution of the Barrier Problem
⇒ Newton Directions (KKT System)
0 )(
0
0 )()(
==−=−+∇
xc
eXVe
vxAxf
µλ
⇒ Solve
−
−+∇−=
−
− eSv
c
vAf
d
d
d
XV
A
IAW x
1
0
00
µ
λ
ν
λ
⇒ Reducing the Systemxv VdXveXd 11 −− −−=µ
∇−=
Σ++ c
d
A
AW x
T
µϕλ
0
VX 1−=Σ
IPOPT Code IPOPT Code –– www.coinwww.coin --or.orgor.org
![Page 72: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/72.jpg)
72
Global Convergence of Newton-based Barrier Solvers
Merit Function
Exact Penalty: P(x, η) = f(x) + η ||c(x)||
Aug’d Lagrangian:L*(x, λ, η) = f(x) + λTc(x) + η ||c(x)||2
Assess Search Direction (e.g., from IPOPT)
Line Search – choose stepsizeα to give sufficient decrease of merit function using a ‘step to the boundary’ rule with τ ~0.99.
• How do we balanceφ (x) and c(x) with η?
• Is this approach globally convergent? Will it still be fast?
)(
0)1(
0)1(
],,0( for
1
1
1
kkk
kvkk
kxk
xkk
vdvv
xdx
dxx
λλαλλτα
ταααα
−+=>−≥+=
>−≥++=∈
++
+
+
![Page 73: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/73.jpg)
73
Global Convergence Failure(Wächter and B., 2000)
0 ,
01)(
02
1 ..
)(
32
22
1
31
≥=−−
=−−
xx
xx
xxts
xfMin
xx11
xx22
0
0)()(
>+
=+
xk
kx
Tk
dx
xcdxA
α
Newton-type line search ‘stalls’even though descent directions exist
Remedies:
•Composite Step Trust Region (Byrd et al.)
•Filter Line Search Methods
![Page 74: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/74.jpg)
74
Line Search Filter Method
Store (φk, θk) at allowed iterates
Allow progress if trial point is acceptable to filter with θ margin
If switching condition
is satisfied, only an Armijo line search is required on φk
If insufficient progress on stepsize, evoke restoration phase to reduce θ.
Global convergence and superlinearlocal convergence proved (with second order correction)
22,][][ >>≥−∇ bad bk
aTk θδφα
φ(x)
θ(x) = ||c(x)||
![Page 75: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/75.jpg)
75
Implementation Details
Modify KKT (full space) matrix if nonsingular
� δ1 - Correct inertia to guarantee descent direction� δ2 - Deal with rank deficient Ak
KKT matrix factored by MA27
Feasibility restoration phase
Apply Exact Penalty Formulation
Exploit same structure/algorithm to reduce infeasibility
−
+Σ+IA
AWTk
kkk
2
1
δδ
ukl
Qk
xxx
xxxcMin
≤≤
−+ 21 ||||||)(||
![Page 76: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/76.jpg)
76
Details of IPOPT Algorithm
Options
Line Search StrategiesLine Search Strategies
- l2 exact penalty merit function
- augmented Lagrangianfunction
- Filter method (adapted from Fletcher and Leyffer)
Hessian Calculation Hessian Calculation
- BFGS (reduced space)
- SR1 (reduced space)
- Exact full Hessian (direct)
- Exact reduced Hessian (direct)
- Preconditioned CG
Comparison
34 COPS Problems
(600 - 160 000 variables)
486 CUTE Problems
(2 – 50 000 variables)
56 MIT T Problems
(12097 – 99998 variables)
Performance MeasurePerformance Measure
- rp, l = (#iterp,l)/ (#iterp, min)
- P(τ) = fraction of problems with log2(rp, l) < τ
![Page 77: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/77.jpg)
77
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
tau
rho
CPU time (CUTE)
IPOPTKNITROLOQO
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
tau
rho
CPU time (COPS)
IPOPTKNITROLOQO
IPOPT Comparison with KNITRO and LOQO
•IPOPT has better performance, robustness on CUTE, MITT and COPS problem sets
•Similar results appear with iteration counts
•Can be downloaded from http://www.coin-or.org
•See links for additional information
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
tau
rho
CPU time (MITT)
IPOPTKNITROLOQO
![Page 78: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/78.jpg)
78
Recommendations for Constrained Optimization
1. Best current algorithms• GRG 2/CONOPT• MINOS• SQP• IPOPT
2. GRG 2 (or CONOPT)is generally slower, but is robust. Use with highly nonlinear functions. Solver in Excel!
3. For small problems (n � 100) with nonlinear constraints, use SQP.4. For large problems (n � 100) with mostly linear constraints, use MINOS.
==> Difficulty with many nonlinearities
SQP MINOSCONOPT
Small, Nonlinear Problems- SQP solves QP's, not LCNLP's, fewer function calls.Large, Mostly Linear Problems - MINOS performs sparse constraint decomposition. Works efficiently in reduced space if function calls are cheap!Exploit Both Features– IPOPT takes advantages of few function evaluations and large-scale linear algebra, but requires exact second derivatives
Fewer Function
Evaluations
Tailored Linear
Algebra
![Page 79: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/79.jpg)
79
SQP RoutinesHSL, NaG and IMSL (NLPQL) RoutinesNPSOL – Stanford Systems Optimization LabSNOPT – Stanford Systems Optimization Lab (rSQP discussed later)IPOPT – http://www.coin-or.org
GAMS ProgramsCONOPT - Generalized Reduced Gradient method with restorationMINOS - Generalized Reduced Gradient method without restorationA student version of GAMS is now available from the CACHE office. The cost for this package including Process Design Case Students, GAMS: A User’s Guide, and GAMS - The Solver Manuals, and a CD-ROM is $65 per CACHE supporting departments, and $100 per non-CACHE supporting departments and individuals. To order please complete standard order form and fax or mail to CACHE Corporation. More information can be found on http://www.che.utexas.edu/cache/gams.html
MS Excel Solver uses Generalized Reduced Gradient method with restoration
Available Software for Constrained Optimization
![Page 80: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/80.jpg)
80
1) Avoid overflows and undefined terms, (do not divide, take logs, etc.) e.g. x + y - ln z = 0Î x + y - u = 0
exp u - z = 02) If constraints must alwaysbe enforced, make sure they are linear or bounds.
e.g. v(xy - z2)1/2 = 3 Πvu = 3u��- (xy - z2) = 0, u � 0
3) Exploit linear constraints as much as possible, e.g. mass balancexi L + yi V = F zi Î l i + vi = f i
L – ∑ l i = 04) Use bounds and constraints to enforce characteristic solutions.
e.g. a � x � b, g (x) � 0to isolate correct root of h (x) = 0.
5) Exploit globalproperties when possibility exists. Convex (linear equations?)Linear Program? Quadratic Program? Geometric Program?
6) Exploit problem structure when possible.e.g. Min [Tx - 3Ty]s.t. xT + y - T2 y = 5
4x - 5Ty + Tx = 70 � T � 1
(If T is fixed ⇒ solve LP) ⇒ put T in outer optimization loop.
Rules for Formulating Nonlinear Programs
![Page 81: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/81.jpg)
81
State of Nature and Problem Premises
Restrictions: Physical, LegalEconomic, Political, etc.
Desired Objective: Yield, Economic, Capacity, etc.
Decisions
Process Model Equations
Constraints Objective Function
Additional Variables
Process OptimizationProblem Definition and Formulation
Mathematical Modeling and Algorithmic Solution
![Page 82: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/82.jpg)
82
Large Scale NLP AlgorithmsMotivation: Improvement of Successive Quadratic Programming as Cornerstone Algorithm
Î process optimization for design, control and operations
Evolution of NLP Solvers:
1981-87: Flowsheet optimization over 100 variables and constraints
1988-98: Static Real-time optimizationover 100 000 variables and constraints
1999- : Simultaneous dynamic optimizationover 1 000 000 variables and constraints
SQP rSQP IPOPT
rSQP++
Current: Tailor structure, architecture and problems
![Page 83: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/83.jpg)
83
In Out
Modular Simulation ModePhysical Relation to Process
- Intuitive to Process Engineer- Unit equations solved internally - tailor-made procedures.
•Convergence Procedures - for simple flowsheets, often identifiedfrom flowsheet structure
•Convergence "mimics" startup.•Debugging flowsheets on "physical" grounds
Flowsheet Optimization Problems - Introduction
![Page 84: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/84.jpg)
84
C
13
2 4
Design SpecificationsSpecify # trays reflux ratio, but would like to specify overhead comp. ==> Control loop -Solve Iteratively
•Frequent block evaluation can be expensive•Slow algorithms applied to flowsheet loops.•NLP methods are good at breaking looks
Flowsheet Optimization Problems - Features
Nested Recycles Hard to HandleBest Convergence Procedure?
![Page 85: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/85.jpg)
85
Chronology in Process Optimization
Sim. Time Equiv.1. Early Work - Black Box Approaches
Friedman and Pinder (1972) 75-150Gaddy and co-workers (1977) 300
2. Transition - more accurate gradientsParker and Hughes (1981) 64Biegler and Hughes (1981) 13
3. Infeasible Path Strategy for Modular SimulatorsBiegler and Hughes (1982) <10Chen and Stadtherr (1985)Kaijaluoto et al. (1985)
and many more 4. Equation Based Process Optimization
Westerberg et al. (1983) <5Shewchuk (1985) 2DMO, NOVA, RTOPT, etc. (1990s) 1-2
Process optimization should be as cheap and easy as process simulation
![Page 86: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/86.jpg)
86
Aspen Custom Modeler (ACM)
Aspen/Plus
gProms
Hysim/Hysys
Massbal
Optisim
Pro/II
ProSim
ROMeo
VTPLAN
Process Simulators with Optimization Capabilities (using SQP)
![Page 87: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/87.jpg)
87
4
3 2
1
5
6h(y) = 0
w(y) y
f(x, y(x))
x
Simulation and Optimization of Flowsheets
Min f(x), s.t. g(x) � 0For single degree of freedom:• work in space defined by curve below. • requires repeated (expensive) recycle convergence
![Page 88: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/88.jpg)
88
Expanded Region with Feasible Path
![Page 89: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/89.jpg)
89
"Black Box" Optimization Approach• Vertical steps are expensive (flowsheet convergence)• Generally no connection between x and y.• Can have "noisy" derivatives for gradient optimization.
![Page 90: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/90.jpg)
90
SQP - Infeasible Path Approach• solve and optimize simultaneously in x and y• extended Newton method
![Page 91: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/91.jpg)
91
Architecture- Replace convergence with optimization block- Problem definition needed (in-line FORTRAN)- Executive, preprocessor, modules intact.
Examples1. Single Unit and Acyclic Optimization- Distillation columns & sequences
2. "Conventional" Process Optimization - Monochlorobenzene process- NH3 synthesis
3. Complicated Recycles & Control Loops- Cavett problem- Variations of above
Optimization Capability for Modular Simulators(FLOWTRAN, Aspen/Plus, Pro/II, HySys)
![Page 92: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/92.jpg)
92
S06HC1
A-1ABSORBER
15 Trays(3 Theoretical Stages)
32 psia
P
S04Feed80
oF
37 psia
T
270o
F
S01 S02
Steam360
oF
H-1U = 100
MaximizeProfit
Feed Flow RatesLB Moles/Hr
HC1 10Benzene 40MCB 50
S07
S08
S05
S09
HC1
T-1
TREATER
F-1FLASH
S03
S10
25psia
S12
S13S15
P-1
C
1200F
T
MCB
S14
U = 100 CoolingWater80oF
S11
Benzene,0.1 Lb Mole/Hr
of MCB
D-1DISTILLATION
30 Trays(20 Theoretical Stages)
Steam360oF
12,000Btu/hr- ft
2
90oF
H-2U = 100
Water80o
F
PHYSICAL PROPERTY OPTIONSCavett Vapor PressureRedlich-Kwong Vapor FugacityCorrected Liquid FugacityIdeal Solution Activity CoefficientOPT (SCOPT) OPTIMIZEROptimal Solution Found After 4 IterationsKuhn-Tucker Error 0.29616E-05Allowable Kuhn-Tucker Error 0.19826E-04Objective Function -0.98259
Optimization Variables32.006 0.38578 200.00 120.00Tear Variables0.10601E-19 13.064 79.229 120.00 50.000Tear Variable Errors (Calculated Minus Assumed)-0.10601E-19 0.72209E-06-0.36563E-04 0.00000E+00 0.00000E+00-Results of infeasible path optimization-Simultaneous optimization and convergence of tear streams.
Optimization of Monochlorobenzene Process
![Page 93: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/93.jpg)
93
H2
N2
Pr
Tr
To
T Tf fν
Product
Hydrogen and Nitrogen feed are mixed, compressed, and combined with a recycle stream and heated to reactor temperature. Reaction occurs in a multibed reactor (modeled here as an equilibrium reactor) to partially convert the stream to ammonia. The reactor effluent is cooled and product is separated using two flash tanks with intercooling. Liquid from the second stage is flashed at low pressure to yield high purity NH3 product. Vapor from the two stage flash forms the recycle and is recompressed.
Ammonia Process Optimization
Hydrogen Feed Nitrogen FeedN2 5.2% 99.8%H2 94.0% ---CH4 0.79 % 0.02%Ar --- 0.01%
![Page 94: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/94.jpg)
94
Optimization Problem
Max {Total Profit @ 15% over five years}
s.t. • 105 tons NH3/yr.• Pressure Balance• No Liquid in Compressors• 1.8 � H2/N2 � 3.5• Treact� 1000o F• NH3 purged � 4.5 lb mol/hr• NH3 Product Purity � 99.9 %• Tear Equations
Performance Characterstics
• 5 SQP iterations. • 2.2 base point simulations. • objective function improves by $20.66 x 106 to $24.93 x 106. • difficult to converge flowsheetat starting point
691.4691.788. Feed 2 (lb mol/hr)
2632.02629.77. Feed 1 (lb mol/hr)
20002163.56. Reactor Press. (psia)
0.010.00855. Purge fraction (%)
10780.524. Inlet temp. rec. comp. (oF)
35353. Inlet temp. 2nd flash (oF)
65652. Inlet temp. 1st flash (oF)
4004001. Inlet temp. reactor (oF)
20.65924.9286Objective Function($106)
Starting pointOptimum
Ammonia Process Optimization
![Page 95: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/95.jpg)
95
Recognizing True Solution• KKT conditions and Reduced Gradients determine true solution• Derivative Errors will lead to wrong solutions!
Performance of AlgorithmsConstrained NLP algorithms are gradient based (SQP, Conopt, GRG2, MINOS, etc.)
Global and Superlinear convergence theory assumes accurate gradients
Worst Case Example(Carter, 1991)Newton’s Method generates an ascent directionand fails for any ε !
How accurate should gradients be for optimization?
)(
)()()(
)( ]11[
/1/1
/1/1
)(
01
00
000
xgAd
Oxfxg
xxfx
A
AxxxfMin
T
T
−−=
+∇==∇=
+−−+
=
=
εε
εεεεεεεε
−−
![Page 96: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/96.jpg)
96
Implementation of Analytic Derivatives
Module Equations
c(v, x, s, p, y) = 0
Sensitivity
Equations
x y
parameters, p exit variables, s
dy/dxds/dxdy/dpds/dp
Automatic Differentiation Tools
JAKE-F, limited to a subset of FORTRAN (Hillstrom, 1982) DAPRE, which has been developed for use with the NAG library (Pryce, Davis, 1987)ADOL-C, implemented using operator overloading features of C++ (Griewank, 1990)ADIFOR, (Bischof et al, 1992) uses source transformation approach FORTRAN code .TAPENADE, web-based source transformation for FORTRAN code
Relative effort needed to calculate gradients is not n+1 but about 3 to 5 (Wolfe, Griewank)
![Page 97: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/97.jpg)
97
S1 S2
S3
S7
S4S5
S6
P
Ratio
Max S3(A) *S3(B) - S3(A) - S3(C) + S3(D) - (S3(E))2 2 3 1/2
Mixer Flash
1 20
100
200
GRG
SQP
rSQP
Numerical Exact
CP
U S
econ
ds (
VS
3200
)Flash Recycle Optimization(2 decisions + 7 tear variables)
1 20
2000
4000
6000
8000
GRG
SQP
rSQP
Numerical Exact
CP
U S
econ
ds (V
S 32
00)
Reactor
Hi P
Flash
Lo P
Flash
Ammonia Process Optimization(9 decisions and 6 tear variables)
![Page 98: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/98.jpg)
98
Min f(z) Min ∇ f(zk)T d + 1/2 dT Wk ds.t. c(z)=0 s.t. c(zk) + (Αk)T d = 0
zL � z � zU zL � zk + d � zU
Characteristics• Many equations and variables (� 100 000)• Many bounds and inequalities (� 100 000)
Few degrees of freedom (10 - 100)Steady state flowsheet optimizationReal-time optimizationParameter estimation
Many degrees of freedom (� 1000)Dynamic optimization (optimal control, MPC)State estimation and data reconciliation
Large-Scale SQP
![Page 99: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/99.jpg)
99
• Take advantage of sparsity of A=∇ c(x)• project W into space of active (or equality constraints)• curvature (second derivative) information only needed in space of degrees of
freedom• second derivatives can be applied or approximated with positive curvature
(e.g., BFGS)• use dual space QP solvers
+ easy to implement with existing sparse solvers, QP methods and line search techniques
+ exploits 'natural assignment' of dependent and decision variables (some decomposition steps are 'free')
+ does not require second derivatives
- reduced space matrices are dense- may be dependent on variable partitioning- can be very expensive for many degrees of freedom- can be expensive if many QP bounds
Few degrees of freedom => reduced space SQP (rSQP)
![Page 100: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/100.jpg)
100
Reduced space SQP (rSQP)Range and Null Space Decomposition
∇−=
+ )(
)(
0 k
k
Tk
kk
xc
xfd
A
AW
λ
Assume no active bounds, QP problem with n variables and mconstraints becomes:
• Define reduced space basis, Zk∈ ℜ n x (n-m)with (Ak)TZk = 0• Define basis for remaining space Yk∈ ℜ n x m, [Yk Zk]∈ℜ n x n
is nonsingular. • Let d = Yk dY + Zk dZ to rewrite:
[ ] [ ] [ ]
∇
−=
+)(
)(
0
0
0
0
00
0 k
kTkk
Z
Ykk
Tk
kkTkk
xc
xf
I
ZYd
d
I
ZY
A
AW
I
ZY
λ
![Page 101: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/101.jpg)
101
Reduced space SQP (rSQP)Range and Null Space Decomposition
• (ATY) dY =-c(xk) is square, dY determined from bottom row.
• Cancel YTWYand YTWZ; (unimportant as dZ, dY --> 0)
• (YTA) O = -YT∇ f(xk), λ can be determined by first order estimate
• Calculate or approximate w= ZTWY dY, solve ZTWZ dZ =-ZT∇ f(xk)-w
• Compute total step: d = Y dY + Z dZ
∇∇
−=
+ )()(
)(
00
0k
kTk
kTk
Z
Y
kTk
kkTkkkTk
kTkkkTkkkTk
xcxfZ
xfY
d
d
YA
ZWZYWZ
AYYWYYWY
λ
00 00
![Page 102: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/102.jpg)
102
Range and Null Space Decomposition• SQP step (d) operates in a higher dimension • Satisfy constraints using range space to get dY
• Solve small QP in null space to get dZ
• In general, same convergence properties as SQP.
Reduced space SQP (rSQP) Interpretation
ddYY
ddZZ
![Page 103: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/103.jpg)
103
1. Apply QRfactorization to A. Leads to dense but well-conditioned Y and Z.
2. Partition variables into decisions u and dependents v. Create orthogonal Y and Z with embedded identity matrices (ATZ = 0, YTZ=0).
3. Coordinate Basis - same Z as above, YT = [ 0 I ]
• Bases use gradient information already calculated.• Adapt decomposition to QP step• Theoretically same rate of convergence as original SQP.• Coordinate basis can be sensitive to choice of u and v. Orthogonal is not.• Need consistent initial point and nonsingular C; automatic generation
Choice of Decomposition Bases
[ ]
=
=
00
RZY
RQA
[ ] [ ]
=
−
=
=∇∇=−
− I
CNY
NC
IZ
CNccATT
Tv
Tu
T
1
![Page 104: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/104.jpg)
104
1. Choose starting point x0.2. At iteration k, evaluate functions f(xk), c(xk) and their gradients.3. Calculate bases Y and Z.4. Solve for step dY in Range space from
(ATY) dY =-c(xk)5. Update projected Hessian Bk ~ ZTWZ(e.g. with BFGS), wk (e.g., zero)6. Solve small QP for step dZ in Null space.
7. If error is less than tolerance stop. Else8. Solve for multipliers using (YTA) λ = -YT∇ f(xk)9. Calculate total step d = Y dY + Z dZ.10. Find step size α and calculate new point, xk+1 = xk + 11. Continue from step 2 with k = k+1.
rSQP Algorithm
UZYk
L
ZkT
ZZTkkT
xZdYdxxts
dBddwxfZMin
≤++≤
++∇
..
2/1))((
![Page 105: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/105.jpg)
105
rSQP Results: Computational Results for General Nonlinear Problems
Vasantharajan et al (1990)
![Page 106: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/106.jpg)
106
rSQP Results: Computational Results for Process ProblemsVasantharajan et al (1990)
![Page 107: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/107.jpg)
107
Coupled Distillation Example - 5000 EquationsDecision Variables - boilup rate, reflux ratio
Method CPU Time Annual Savings Comments1. SQP* 2 hr negligible Base Case 2. rSQP 15 min. $ 42,000 Base Case3. rSQP 15 min. $ 84,000 Higher Feed Tray Location4. rSQP 15 min. $ 84,000 Column 2 Overhead to Storage5. rSQP 15 min $107,000 Cases 3 and 4 together
18
10
1
QVKQVK
Comparison of SQP and rSQP
![Page 108: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/108.jpg)
108
REACTOR EFFLUENT FROM LOW PRESSURE SEPARATOR
PR
EFL
ASH
MA
IN F
RA
C.
RECYCLEOIL
REFORMERNAPHTHA
C3
/C4
SP
LITT
ER
nC4
iC4
C3
MIXED LPG
FUEL GAS
LIGHT NAPHTHA
AB
SO
RB
ER
/
STR
IPP
ER
DEB
UTA
NIZ
ER
DIB
• square parameter case to fit the model to operating data. • optimization to determine best operating conditions
Existing process, optimization on-line at regular intervals: 17 hydrocarbon components, 8 heat exchangers, absorber/stripper (30 trays), debutanizer (20 trays), C3/C4 splitter (20 trays) and deisobutanizer (33 trays).
Real-time Optimization with rSQPSunoco Hydrocracker Fractionation Plant
(Bailey et al, 1993)
![Page 109: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/109.jpg)
109
Model consists of 2836 equality constraints and only ten independent variables. It is also reasonably sparse and contains 24123 nonzero Jacobian elements.
P= ziCiG
i ∈ G∑ + ziCi
E
i ∈ E∑ + ziCi
Pm
m=1
NP
∑ −U
Cases Considered:1. Normal Base Case Operation2. Simulate fouling by reducing the heat exchange coefficients for the debutanizer3. Simulate fouling by reducing the heat exchange coefficients for splitter
feed/bottoms exchangers4. Increase price for propane5. Increase base price for gasoline together with an increase inthe octane credit
Optimization Case Study Characteristics
![Page 110: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/110.jpg)
110
![Page 111: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/111.jpg)
111
Many degrees of freedom=> full space IPOPT
• work in full space of all variables• second derivatives useful for objective and constraints• use specialized large-scale Newton solver
+ W=∇ xxL(x,λ) and A=∇ c(x)sparse, often structured+ fast if many degrees of freedom present+ no variable partitioning required
- second derivatives strongly desired- W is indefinite, requires complex stabilization- requires specialized large-scale linear algebra
∇−=
Σ+
+ )(
)(
0 k
k
Tk
kk
xc
xd
A
AW ϕλ
![Page 112: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/112.jpg)
112
*$6�67$7,216
)LQDO�3URGXFW�WDQNV
6XSSO\�WDQNV
,QWHUPHGLDWH�WDQNV
*DVROLQH�%OHQGLQJ�+HUH�
*DVROLQH�%OHQGLQJ*DVROLQH�%OHQGLQJ2,/�7$1.63LSHOLQH
V
),1$/�352'8&7�758&.6
![Page 113: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/113.jpg)
113
Blending Problem & Model Formulation
⇒
⇒
Final Product tanks (k) Intermediate tanks (j) Supply tanks (i)
ijtf , jktf ,
jtv ,
itq ,
iq
jtq ,.. ktq ,..
kv ktf ,..
f & v ------ flowrates and tank volumesq ------ tank qualities
Model Formulation in AMPL
max ,
min
max ,
min
0,,,,
,,,1,1,,,,
0,,
,,1,, s.t.
)t
,( , max
jvjtvjv
kqktqkq
jjkt
fjt
qkt
fkt
q
jtv
jtq
jtv
jtq
iijt
fit
qk
jktf
jtq
jjkt
fktf
jtv
jtv
iijt
fk
jktf
i ic
ktf
k kc itf
≤≤
≤≤
=−
=++
+−
=−
=+
+−
∑−∑
∑
∑∑
∑
∑∑
∑
![Page 114: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/114.jpg)
114
F1
F2
F3
P 1
B 1
B 2
F 1
F 2
F 3
P 2
P 1
B 1
B 2
B 3
Haverly, C. 1978 (HM) Audet & Hansen 1998 (AHM)
Small MultiSmall Multi--day Blending Models day Blending Models
Single Qualities Single Qualities
![Page 115: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/115.jpg)
115
Honeywell Blending Model Honeywell Blending Model –– Multiple DaysMultiple Days48 Qualities48 Qualities
![Page 116: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/116.jpg)
116
Summary of Results – Dolan-Moré plot
Performance profile (iteration count)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 10 100 1000 10000 100000 1000000 10000000
τ
φ
IPOPT
LOQO
KNITRO
SNOPT
MINOS
LANCELOT
![Page 117: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/117.jpg)
117
Comparison of NLP Solvers: Data Reconciliation
0.01
0.1
1
10
100
0 200 400 600
Degrees of Freedom
CP
U T
ime
(s,
norm
.) LANCELOT
MINOS
SNOPT
KNITRO
LOQO
IPOPT
0
200
400
600
800
1000
0 200 400 600Degrees of Freedom
Itera
tions
LANCELOT
MINOS
SNOPT
KNITRO
LOQO
IPOPT
![Page 118: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/118.jpg)
118
At nominal conditions, p0
Min f(x, p0)s.t. c(x, p0) = 0
a(p0) � x � b(p0)
How is the optimum affected at other conditions, p � p0?• Model parameters, prices, costs• Variability in external conditions• Model structure
• How sensitive is the optimum to parameteric uncertainties?• Can this be analyzed easily?
Sensitivity Analysis for Nonlinear Programming
![Page 119: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/119.jpg)
119
Take KKT Conditions∇ L(x*, p, λ, v) = 0
c(x*, p0) = 0ETx* - bnd( p0)=0
and differentiate and expand about p0. ∇ pxL(x*, p, λ, v)T + ∇ xxL(x*, p, λ, v)T ∇ px*T + ∇ xh(x*, p, λ, v)T ∇ pλT + Ε ∇ pv
T = 0∇ pc(x*, p0)
T + ∇ xc(x*, p0)T ∇ px*T = 0
ET (∇ px*T - ∇ pbndT)=0Notes:• A key assumption is that under strict complementarity, the active set will not change for small perturbations of p.• If an element of x* is at a bound then ∇ pxi*
T = ∇ pbndT
• Second derivatives are required to calculate sensitivities, ∇ px*T
• Is there a cheaper way to calculate ∇ px*T?
Calculation of NLP Sensitivity
![Page 120: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/120.jpg)
120
Decomposition for NLP SensitivityDecomposition for NLP Sensitivity
∇−∇
∇−=
∇∇
∇
−++=
Tp
T
Tp
Txp
Tp
Tp
Tp
T
T
TTT
bndE
pxc
vpxL
v
x
E
A
EAW
vpbndxExcxfvpxL
)*,(
),,*,(*
00
00
)))(*((*)(*)(),,*,(Let
λλ
λλ
•Partition variables into basic, nonbasic and superbasic
∇ pxT= Z ∇ pxST + Y∇ pxB
T + T∇ pxNT
•Set∇ pxNT = ∇ pbndN
T, nonbasic variables to rhs,
•Substitute for remaining variables
•Perform range and null space decomposition
•Solve only for ∇ pxST and∇ pxB
T
![Page 121: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/121.jpg)
121
Decomposition for NLP SensitivityDecomposition for NLP Sensitivity
∇+∇∇+∇∇+∇
−=
∇∇∇
TNp
TTp
TNp
Txp
T
TNp
Txp
T
Tp
TSp
TBp
T
TT
TTT
xTApxc
xTWvpxLZ
xTWvpxLY
x
x
YA
WZZWYZ
AYWYYWYY
)*,(
) ),,*,((
) ),,*,((
00
0 λλ
λ
• Solve only for ∇ pxBT from bottom row and∇ pxS
T from middle row
• If second derivatives are not available, ZTWZ, ZTWY ∇ pxBT and
ZTWT ∇ pxNT can be constructed by directional finite differencing
• If assumption of strict complementarity is violated, sensitivity can be
calculated using a QP subproblem.
![Page 122: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/122.jpg)
122
x1
x2
z 1
z 2
Saddle Point
x*
At solution x*: Evaluate eigenvalues of ZT∇ xxL*Z
Strict local minimum if all positive.
• Nonstrict local minimum: If nonnegative, find eigenvectors for zero eigenvalues, Î regions of nonunique solutions
• Saddle point: If any are negative, move along directions of corresponding eigenvectors and restart optimization.
Second Order Tests
Reduced Hessian needs to be positive definite
![Page 123: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/123.jpg)
123
Sensitivity for Flash Recycle Optimization(2 decisions, 7 tear variables)
S1 S2
S3
S7
S4S5
S6
P
Ratio
Max S3(A) *S3(B) - S3(A) - S3(C) + S3(D) - (S3(E))2 2 3 1/2
Mixer Flash
•Second order sufficiency test:•Dimension of reduced Hessian = 1•Positive eigenvalue•Sensitivity to simultaneous change in feed rate and upper bound on purge ratio•Only 2-3 flowsheet perturbations required for second order information
![Page 124: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/124.jpg)
124
Reactor
Hi P
Flash
Lo P
Flash
17
17.5
18
18.5
19
19.5
20
Ob
ject
ive
Fu
nct
ion
0.0
01
0.0
1
0.1
Relative perturbation change
Sensitivities vs. Re-optimized Pts
Actual
QP2
QP1
Sensitivity
Ammonia Process Optimization(9 decisions, 8 tear variables)
•Second order sufficiency test:•Dimension of reduced Hessian = 4•Eigenvalues = [2.8E-4, 8.3E-10, 1.8E-4, 7.7E-5]•Sensitivity to simultaneous change in feed rate and upper bound on reactor conversion•Only 5-6 extra perturbations for second derivatives
![Page 125: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/125.jpg)
125
Multiperiod OptimizationCoordination
Case 1 Case 2 Case 3 Case 4 Case N
1. Design plant to deal with different operating scenarios (over time or with uncertainty)
2. Can solve overall problem simultaneously• large and expensive• polynomial increase with number of cases• must be made efficient through specialized decomposition
3. Solve also each case independently as an optimization problem (inner problem with fixed design)• overall coordination step (outer optimization problem for design)• require sensitivity from each inner optimization case with design
variables as external parameters
![Page 126: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/126.jpg)
126
Multiperiod Flowsheet Example
Ti
1
A
i C
Ti
2F
i
1F
i2
TwTi
w
W i
i C T
0A
A
0
2 1
1
V
Parameters Period 1 Period 2 Period 3 Period 4E (kJ/mol) 555.6 583.3 611.1 527.8k0 (1/h) 10 11 12 9F (kmol/h) 45.4 40.8 24.1 32.6Time (h) 1000 4000 2000 1000
![Page 127: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/127.jpg)
127
d
hi, gi
Min f0(d) + Σi fi(d, xi)s.t. hi(xi, d) = 0, i = 1,… N
gi(xi, d) � 0, i = 1,… Nr(d) � 0
Variables:x: state (z) and control (u) variables in each operating period d: design variables (e. g. equipment parameters) usedδi: substitute for d in each period and add δi = d
Multiperiod Design Model
Min f0(d) + Σi fi(d, xi)s.t. hi(xi, δi) = 0, i = 1,… N
gi(xi, δi) +si = 0, i = 1,… N0 � si, d –δi=0, i = 1,… Nr(d) � 0
![Page 128: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/128.jpg)
128
Multiperiod Decomposition StrategySQP Subproblem
•Block diagonal bordered KKT matrix
(arrowhead structure)
•Solve each block sequentially (range/null dec.)
to form small QP in space of d variables
•Reassemble all other steps from QP solution
![Page 129: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/129.jpg)
129
minimize φ = ∇ d f 0T + { ∇ p f i
T (ZBi+ YBi
) + (ZAi+ YAi
)T ∇ p2Li
i =1
N
∑ (ZBi+ YB i
) }
∆d
+ 1
2∆dT ∇ d
2L0 + { (ZBi+ YBi
)T ∇ p2Li
i =1
N
∑ (ZBi+ YB i
) }
∆d
subject to r + ∇ dr ∆d ≤ 0
Substituting back into the original QP subproblem leads to a QP only in terms of ∆d.
Multiperiod Decomposition Strategy
From decomposition of KKT block in each period, obtain the following directions that are parametric in∆d:
Once ∆d is obtained, directions are obtained from the above equations.
![Page 130: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/130.jpg)
130
� pi steps are parametric in ∆d and their components are created independently
� Decomposition linear in number of periods and trivially parallelizable
� Choosing the active inequality constraints can be done through:
-Active set strategy (e.g., bound addition)
-Interior point strategy using barrier terms in objective
• Easy to implement in decomposition
Starting Point
Original QP Decoupling
Null - Range Decomposition
QP at d
Calculate Step
Calculate Reduced Hessian
Line search
Optimality Conditions
NO
YES
Optimal Solution
Active Set Update
Bound AdditionMINOR ITERATION LOOP
Multiperiod Decomposition Strategy
![Page 131: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/131.jpg)
131
Ti
1
A
i C
Ti
2F
i
1F
i2
TwTi
w
W i
i C T
0A
A
0
2 1
1
V
Multiperiod Flowsheet 1(13+2) variables and (31+4) constraints (1 period)
262 variables and 624 constraints (20 periods)
3020100
0
100
200
300
400
SQP (T)MINOS(T)MPD/SQP(T)
Periods
CPU time (s)
![Page 132: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/132.jpg)
132
C1
C2
H1
H2
T1
563 K
393 K
T2
T3
T4
T8
T7
T6
T5
350 K
Qc
300 K
A1
A3
A2
i i
i
i
i
i
i
i i
4A
320 K
Multiperiod Example 2 – Heat Exchanger Network(12+3) variables and (31+6) constraints (1 period)
243 variables and 626 constraints (20 periods)
3020100
0
10
20
30
40
50
SQP (T)MINOS (T)MPD/SQP (T)
Periods
CPU time (s)
![Page 133: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/133.jpg)
133
-Unconstrained Newton and Quasi Newton Methods-KKT Conditions and Specialized Methods-Reduced Gradient Methods (GRG2, MINOS)-Successive Quadratic Programming (SQP)-Reduced Hessian SQP-Interior Point NLP (IPOPT)
Process Optimization Applications-Modular Flowsheet Optimization-Equation Oriented Models and Optimization-Realtime Process Optimization-Blending with many degrees of freedom
Further Applications-Sensitivity Analysis for NLP Solutions-Multiperiod Optimization Problems
Summary and Conclusions
![Page 134: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/134.jpg)
Optimization of Differential-Algebraic Equation Systems
L. T. BieglerChemical Engineering Department
Carnegie Mellon UniversityPittsburgh, PA
![Page 135: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/135.jpg)
2
I IntroductionProcess Examples
II ParametricOptimization- Gradient Methods
• Perturbation• Direct - Sensitivity Equations• Adjoint Equations
III Optimal ControlProblems- Optimality Conditions- Model Algorithms
• Sequential Methods• Multiple Shooting• Indirect Methods
IV SimultaneousSolutionStrategies- Formulation and Properties- Process Case Studies- Software Demonstration
DAE Optimization Outline
![Page 136: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/136.jpg)
3
tf, final timeu, control variablesp, time independent parameters
t, timez, differential variablesy, algebraic variables
Dynamic Optimization Dynamic Optimization ProblemProblem
( )ftp,u(t),y(t),z(t), min Φ
( )pttutytzfdt
tdz,),(),(),(
)( =
( ) 0,),(),(),( =pttutytzg
ul
ul
ul
ul
o
ppp
utuu
ytyy
ztzz
zz
dd
dd
dd
dd
)(
)(
)(
)0(
s.t.
![Page 137: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/137.jpg)
4
DAE Models in Process Engineering
Differential Equations•Conservation Laws (Mass, Energy, Momentum)
Algebraic Equations•Constitutive Equations, Equilibrium (physical properties, hydraulics, rate laws)•Semi-explicit form•Assume to be index one (i.e., algebraic variables can be solved uniquely by algebraic equations)•If not, DAE can be reformulated to index one (see Ascher and Petzold)
Characteristics•Large-scale models – not easily scaled•Sparse but no regular structure•Direct linear solvers widely used•Coarse-grained decomposition of linear algebra
![Page 138: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/138.jpg)
5
Catalytic Cracking of Gasoil (Tjoa, 1991)
number of states and ODEs: 2number of parameters:3no control profilesconstraints: pL � p � pU
Objective Function: Ordinary Least Squares
(p1, p2, p3)0 = (6, 4, 1)(p1, p2, p3)* = (11.95, 7.99, 2.02)(p1, p2, p3)true = (12, 8, 2)
1.00.80.60.40.20.00.0
0.2
0.4
0.6
0.8
1.0
YA_data
YQ_data
YA_estimate
YQ_estimate
t
Yi
Parameter Estimation
0)0( ,1)0(
)(
, ,
22
1
231
321
==−−=
+−=→→→
qa
qpapq
appa
SASQQA ppp
�
�
![Page 139: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/139.jpg)
6
Batch Distillation Multi-product Operating Policies
5XQ�EHWZHHQ�GLVWLOODWLRQ�EDWFKHV7UHDW�DV�ERXQGDU\�YDOXH�RSWLPL]DWLRQ�SUREOHP
:KHQ�WR�VZLWFK�IURP�$�WR�RIIFXW WR�%"+RZ�PXFK�RIIFXW WR�UHF\FOH"5HIOX["%RLOXS 5DWH"2SHUDWLQJ�7LPH"
$ %
![Page 140: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/140.jpg)
7
Nonlinear Model Predictive Control (NMPC)
Process
NMPC Controller
d : disturbancesz : differential statesy : algebraic states
u : manipulatedvariables
ysp : set points
( )( )dpuyzG
dpuyzFz
,,,,0
,,,,
==′
NMPC Estimation and Control
sConstraintOther
sConstraint Bound
)()),(),(),((0
)),(),(),(()(..
||))||||)(||min
init
1sp
ztztttytzG
tttytzFtzts
yty uy Q
kk
Q
==
=′
−+−∑ ∑ −
u
u
u(tu(tu
NMPC Subproblem
Why NMPC?
� Track a profile
� Severe nonlinear dynamics (e.g, sign changes in gains)
� Operate process over wide range (e.g., startup and shutdown)
Model Updater( )( )dpuyzG
dpuyzFz
,,,,0
,,,,
==′
![Page 141: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/141.jpg)
8
Optimization of dynamic batch process operation resulting from reactor and distillation column
DAE models:z’ = f(z, y, u, p)g(z, y, u, p) = 0
number of states and DAEs: nz + nyparameters for equipment design (reactor, column)nu control profiles for optimal operation
Constraints: uL � u(t) � uU zL � z(t) � zU
yL � y(t) � yU pL � p � pU
Objective Function: amortized economic function at end of cycle time tf
zi,I0 zi,II
0zi,III0 zi,IV
0
zi,IVf
zi,If zi,II
f zi,IIIf
Bi
A+B→C
C+B→P+E
P+C→G
5
10
15
20
25
580
590
600
610
620
630
640
0 0.5 1 1.5 2 2.50 0.25 0.5 0.75 1 1.25Tim e (h r.)
Dyn a mic
C o nsta nt
Dyn a mic
C o nsta nt
optimal reactor temperature policy optimal column reflux ratio
Batch Process Optimization
![Page 142: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/142.jpg)
9
FexitC H2 4
Texit ≤ 1180K
C2H CH
6 32→ • CH CH CH CH
3 2 6 4 2 5•+ → + •
C2H CH H
5 2 4•→ + •H CH H CH•+ → + •
2 6 2 2 52C
2H CH
5 4 10•→C
2H CH CH CH
5 2 4 3 6 3•+ → + •
H CH CH•+ → •2 4 2 5
0
1
2
3
4
5
6
0 2 4 6 8 10
Length m
Flo
w r
ate
mol
/s
0
500
1000
1500
2000
2500
Hea
t flu
x kJ
/m2s
C2H4 C2H6 log(H2)+12 q
Reactor Design ExamplePlug Flow Reactor Optimization
The cracking furnace is an important example in the olefin production industry, where various hydrocarbon feedstocks react. Consider a simplified model for ethane cracking (Chen et al., 1996). The objective is to find an optimal profile for the heat flux along the reactor in order to maximize the production of ethylene.
Max s.t. DAE
The reaction system includes six molecules, three free radicals, and seven reactions. The model also includes the heat balance and the pressure drop equation. This gives a total of eleven differential equations.
Concentration and Heat Addition Profile
![Page 143: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/143.jpg)
10
Dynamic Optimization Approaches
DAE Optimization Problem
Sequential Approach
Vassiliadis(1994)Discretize controls
Variational Approach
Pontryagin(1962)
Inefficient for constrained problems
Apply a NLP solver
Efficient for constrained problems
![Page 144: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/144.jpg)
11
Sequential Approaches - Parameter Optimization
Consider a simpler problem without control profiles:
e.g., equipment design with DAE models - reactors, absorbers, heat exchangers
Min Φ (z(tf))
z’ = f(z, p), z (0) = z0
g(z(tf)) � 0, h(z(tf)) = 0
By treating the ODE model as a "black-box" a sequential algorithm can be constructed that can be treated as a nonlinear program.
Task: How are gradients calculated for optimizer?
NLPSolver
ODEModel
GradientCalculation
P
φ,g,h
z (t)
![Page 145: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/145.jpg)
12
Gradient Calculation
Perturbation
Sensitivity Equations
Adjoint Equations
Perturbation
Calculate approximate gradient by solving ODE model (np + 1) times
Let ψ = Φ, g and h (at t = tf)
dψ/dpi = {ψ (pi + ̈ pi) - ψ (pi)}/ ¨pi
Very simple to set up
Leads to poor performance of optimizer and poor detection of optimum unless roundoff error (O(1/¨pi) and truncation error (O(¨pi)) are small.
Work is proportional to np (expensive)
![Page 146: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/146.jpg)
13
Direct Sensitivity
From ODE model:
(nz x np sensitivity equations)
• z and si , i = 1,…np, an be integrated forward simultaneously.
• for implicit ODE solvers, si(t) can be carried forward in time after converging on z
• linear sensitivity equations exploited in ODESSA, DASSAC, DASPK, DSL48s and a number of other DAE solvers
Sensitivity equations are efficient for problems with many more constraints than parameters (1 + ng + nh > np)
{ }
iii
T
iii
ii
p
zss
z
f
p
fs
dt
ds
ip
tzts
pzztpzfzp
∂∂=
∂∂+
∂∂==′
=∂
∂=
==′∂∂
)0()0( ,)(
...np 1, )(
)( define
)()0(),,,( 0
![Page 147: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/147.jpg)
14
Example: Sensitivity Equations
0)0(,0
22
1)0(,0
22
2,1,/)()(,/)()(
)0(,5
2,1,
1,1,22,112,
2,21,11,
2,1,
1,1,22,12,
2,21,11,
,,
21
1212
22
211
==
+++=′
+=′
==
++=′
+=′
=∂∂=∂∂===
+=′+=′
bb
bbbbb
bbb
aa
baaaa
aaa
bjjbajja
a
b
ss
psszszzs
szszs
ss
psszszs
szszs
jptztsptzts
pzz
pzzzz
zzz
![Page 148: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/148.jpg)
15
Adjoint Sensitivity
Adjoint or Dual approach to sensitivity
Adjoin model to objective function orconstraint
(ψ = Φ,g or h)
(λ(t)) serve as multipliers on ODE’s)
Now, integrate by parts
and find dψ/dp subject to feasibility of ODE’s
Now, set all terms notin dp to zero.
∫ −′−=ft
Tf dttpzfzt
0
)),,(()( λψψ
∫ +′+−+=ft
TTf
Tf
Tf dttpzFztztpzt
0
0 )),,()()()()0()( λλλλψψ
00
∫
∂∂+
∂∂+′+
∂
∂+
−
∂∂
=ft TTT
fff
f dtdpp
ftz
z
fdp
p
pztzt
tz
tzd
0
0 )()0()(
)()()(
))((λδλλλδλ
ψψ
![Page 149: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/149.jpg)
16
Adjoint System
Integrate model equations forward
Integrate adjoint equations backward and evaluate integral and sensitivities.
Notes:
nz (ng + nh + 1) adjoint equations must be solved backward (one for each objective and constraint function)
for implicit ODE solvers, profiles (and even matrices) can be stored and carried backward after solving forward for z as in DASPK/Adjoint (Li and Petzold)
more efficient on problems where: np > 1 + ng + nh
∫
∂∂+
∂∂=
∂∂
=∂∂−=′
ft
f
ff
dttp
f
p
pz
dp
d
tz
tztt
z
f
0
0 )()0()(
)(
))(()( ),(
λλψ
ψλλλ
![Page 150: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/150.jpg)
17
Example: Adjoint Equations
dttztdp
td
dp
td
tz
ttzz
tz
ttpzz
dttp
f
p
pz
dp
d
tz
tztt
z
f
pzzzzztpzf
pzz
pzzzz
zzz
f
f
t
b
f
a
f
f
ff
f
ffb
t
f
ff
bT
a
b
)()()(
)0()(
)(
)()( ,2
)(
)()( ),(2
:becomesthen
)()0()(
)(
))(()( ),(
)()(),,( Form
)0(,5
1
0
2
1
2212212
1122111
0
0
121222
211
21
1211
22
211
∫
∫
=
=
∂∂
=−−=′
∂∂
=+−−=′
∂∂+
∂∂=
∂∂
=∂∂−=′
+++=
==
+=′+=′
λψ
λψ
ψλλλλ
ψλλλλ
λλψ
ψλλλ
λλλ
![Page 151: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/151.jpg)
18
A + 3B --> C + 3D
L
T s
T R
T P
3:1 B/A 383 K
TP = specified product temperatureTR = reactor inlet, reference temperatureL = reactor lengthTs = steam sink temperatureq(t) = reactor conversion profileT(t) = normalized reactor temperature profile
Cases considered:• Hot Spot - no state variable constraints• Hot Spot with T(t) � 1.45
Example: Hot Spot Reactor
Roo
P
Pproducto
Rfeed
RS
L
RSTLTT
C/T C, T(L) T
, T(L)) (THC) -,(T+
Tdt
dqTTtT
dt
dT
qtTtqdt
dqts
dtTTtTLMinSRP
101120
0110
1)0( ,3/2)/)((5.1
0)0( )],(/2020exp[))(1(3.0 ..
)/)(( 0
,,,
+==
=∆
=+−−=
=−−=
−−=Φ ∫
![Page 152: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/152.jpg)
19
1.51.00.50.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Normalized Length
Con
vers
ion,
q
1.51.00.50.0
1.0
1.1
1.2
1.3
1.4
1.5
1.6
Normalized Length
Nor
mal
ized
Tem
pera
ture
Method: SQP (perturbation derivatives)
L(norm) TR(K) TS(K) TP(K)Initial: 1.0 462.23 425.26 250Optimal: 1.25 500 470.1 188.413 SQP iterations / 2.67 CPU min. (µVax II)
Constrained Temperature Case: could not be solved with sequential method
Hot Spot Reactor: Unconstrained Case
![Page 153: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/153.jpg)
20
Variable Final Time(Miele, 1980)
Define t = pn+1 τ, 0 ≤ τ ≤ 1, pn+1 = tf
Let dz/dt = (1/ pn+1) dz/dτ = f(z, p) ⇒ dz/dτ = (pn+1) f(z, p)
Converting Path Constraints to Final Time
Define measure of infeasibility as a new variable, znz+1(t) (Sargent & Sullivan, 1977):
Tricks to generalize classes of problems
)degenerate is constraint (however, )( Enforce
0)0( , ))(),((,0max()(
))(),((,0max()(
1
12
1
0
21
ε≤
==
=
+
++
+
∑
∑∫
fnz
nzj
jnz
j
t
jfnz
tz
ztutzgtzor
dttutzgtzf
�
![Page 154: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/154.jpg)
21
Profile Optimization - (Optimal Control)
Optimal Feed Strategy (Schedule) in Batch Reactor
Optimal Startup and Shutdown Policy
Optimal Control of Transients and Upsets
Sequential Approach: Approximate control profile as through parameters (piecewise constant, linear, polynomial, etc.)
Apply NLP to discretization as with parametric optimization
Obtain gradients through adjoints (Hasdorff; Sargent and Sullivan; Goh and Teo) or sensitivity equations (Vassiliadis, Pantelides and Sargent; Gill, Petzold et al.)
Variational (Indirect) approach: Apply optimality conditions and solve as boundary value problem
![Page 155: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/155.jpg)
22
Optimality Conditions(Bound constraints on u(t))
Min φ(z(tf))s.t. dz/dt = f(z, u), z (0) = z0
g (z(tf)) � 0h (z(tf)) = 0a � u(t) � b
Form Lagrange function - adjoin objective function and constraints:
Derivation of Variational Conditions Indirect Approach
dtbtutuauzfz
tztzvtzhtzgt
dtbtutuauzfz
vtzhtzgt
Tb
t Ta
TT
ffTTT
fT
ff
Tb
t Ta
T
Tf
Tff
f
f
))(())((),(
)()()0()0())(())(()(
:partsby Integrate
))(())(()),((
))(())(()(
0
0
−+−+++
−+++=
−+−+−−
++=
∫
∫
ααλλ
λλµφφ
ααλ
µφφ
�
�
![Page 156: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/156.jpg)
23
λ ft( )=∂φ∂z
+∂g
∂zµ +
∂h
∂zγ
ft=t
∂f
∂uλ =
∂H
∂u= 0
∂ H
∂u= α a − α b
α aT (a − u(t))
α bT (u(t) − b)
ua ≤ u(t) ≤ ub
α a ≥ 0,α b ≥ 0
∂H
∂u= −α b ≤ 0
∂H
∂u= α a ≥ 0
At optimum, δφ ≥0. Since u is the control variable, let all other terms vanish.⇒ δ z(tf):
δz(0): λ(0) = 0 (if z(0) is not specified)δz(t):
Define Hamiltonian, H = λTf(z,u)For u notat bound:
For u atbounds:
Upper bound, u(t) = b, Lower bound, u(t) = a,
Derivation of Variational Conditions
λλz
f
z
H
∂∂−=
∂∂−=�
0 )(),(
)(),(
)0()0()(
0≥
−+
∂∂+
∂∂++
+
−
∂∂+
∂∂+
∂∂=
∫ dttuu
uzftz
z
uzf
ztzvz
h
z
g
z
ftT
ab
T
Tf
T
δααλδλλ
δλδλµφδφ
�
![Page 157: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/157.jpg)
24
Car ProblemTravel a fixed distance (rest-to-rest) in minimum time.
0)(’,0)0(’
)(,0)0(
)(
" ..
==
==≤≤
=
f
f
f
txx
Ltxx
btua
uxts
tMin
0)(,0)0(
)(,0)0(
)(
1’
’
’ ..
22
11
3
2
21
3
==
==≤≤
===
f
f
f
txx
Ltxx
btua
x
ux
xxts
)(tMin x
s
f
f
f
f
f
tt
auctt
buctctttcc
u
H
tt
ttcct
ct
uxH
==
=>==>+=
−+==∂∂
====>=
−+===>−=
===>=
++=
at occurs )0(Crossover
,0,
,0,0)(
1)( ,1)(0
)()(
)(0 :Adjoints
:n Hamiltonia
2
2
21122
333
12212
111
3221
λ
λ
λλλ
λλλλλ
λλλ
�
�
�
![Page 158: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/158.jpg)
25
t f
u(t)
b
a
t s
1 / 2 bt2,t < ts
1 / 2 bts2 - a ts - tf( )2( ), t ≥ ts
bt, t < ts
bts + a t - ts( ), t ≥ ts
2L
b 1- b / a( )
1/2
(1− b / a)2L
b 1 - b / a( )
1/2
Optimal Profile
From state equations:
x1(t) =
x2 (t) =
Apply boundary conditions at t = tf:x1(tf) = 1/2 (b ts
2 - a (ts - tf)2) = L
x2(tf) = bts + a (tf - ts) = 0⇒ ts =
tf =
•Problem is linear in u(t). Frequently these problems have "bang-bang" character.•For nonlinear and larger problems, the variational conditions can be solved numerically as boundary value problems.
Car ProblemAnalytic Variational Solution
![Page 159: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/159.jpg)
26
A B
C
u
u /22
u(T(t))
Example: Batch reactor - temperature profile
Maximize yield of B after one hour’s operation by manipulating a transformed temperature, u(t).
⇒ Minimize -zB(1.0)s.t.
z’A = -(u+u2/2) zA
z’B = u zA
zA(0) = 1zB(0) = 00 � u(t) � 5
Adjoint Equations:H = -λA(u+u2/2) zA + λB u zA
∂H/∂u = λA (1+u) zA + λB zAλ’A = λA(u+u2/2) - λB u, λA(1.0) = 0λ’B = 0, λB(1.0) = -1
Cases Considered1. NLP Approach- piecewise constant and linear profiles.2. Control Vector Iteration
![Page 160: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/160.jpg)
27
Batch Reactor Optimal Temperature Program Piecewise Constant
Op
tima
l Pro
file
, u(t
)
0. 0.2 0.4 0.6 0.8 1.0
2
4
6
Time, h
ResultsPiecewise Constant Approximation with Variable Time ElementsOptimum B/A: 0.57105
![Page 161: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/161.jpg)
28
Op
tima
l Pro
file
, u(t
)
0. 0.2 0.4 0.6 0.8 1.0
2
4
6
Time, h
Results:Piecewise Linear Approximation with Variable Time ElementsOptimum B/A: 0.5726Equivalent # of ODE solutions: 32
Batch Reactor Optimal Temperature Program Piecewise Linear
![Page 162: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/162.jpg)
29
Op
tima
l Pro
file
, u(t
)
0. 0.2 0.4 0.6 0.8 1.0
2
4
6
Time, h
Results:Control Vector Iteration with Conjugate GradientsOptimum (B/A): 0.5732Equivalent # of ODE solutions: 58
Batch Reactor Optimal Temperature Program
Indirect Approach
![Page 163: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/163.jpg)
30
Dynamic Optimization - Sequential Strategies
Small NLP problem, O(np+nu) (large-scale NLP solver not required) • Use NPSOL, NLPQL, etc. • Second derivatives difficult to get
Repeated solution of DAE model and sensitivity/adjoint equations, scales with nz and np
• Dominant computational cost• May fail at intermediate points
Sequential optimization is not recommended for unstable systems. State variables blow up at intermediate iterations for control variables and parameters.
Discretize control profiles to parameters (at what level?)
Path constraints are difficult to handle exactly for NLP approach
![Page 164: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/164.jpg)
31
Instabilities in DAE ModelsThis example cannot be solved with sequential methods (Bock, 1983):
dy1/dt = y2
dy2/dt = τ2 y1 + (π2 − τ2) sin (π t)
The characteristic solution to these equations is given by:
y1(t) = sin (π t) + c1 exp(-τ t) + c2 exp(τ t)
y2 (t) = πcos (π t) - c1 τ exp(-τ t) + c2 τ exp(τ t)
Both c1 and c2 can be set to zero by either of the following equivalent conditions:
IVP y1(0) = 0, y2 (0) = π
BVP y1(0) = 0, y1(1) = 0
![Page 165: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/165.jpg)
32
IVP SolutionIf we now add roundoff errors e1 and e2 to the IVP and BVP conditions, we see significant differences in the sensitivities of the solutions.
For the IVP case, the sensitivity to the analytic solution profile is seen by large changes in the profiles y1(t) and y2(t) given by:
y1(t) = sin (π t) + (e1 - e2/τ) exp(-τ t)/2
+(e1 + e2/τ) exp(τ t)/2
y2 (t) = πcos (π t) - (τ e1 - e2) exp(-τ t)/2
+ (τ e1 + e2) exp(τ t)/2
Therefore, even if e1 and e2 are at the level of machine precision (< 10-13), a large value of τ and t will lead to unbounded solution profiles.
![Page 166: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/166.jpg)
33
BVP Solution
On the other hand, for the boundary value problem, the errors affect the analytic solution profiles in the following way:
y1(t) = sin (π t) + [e1 exp(τ)- e2] exp(-τ t)/[exp(τ) - exp(-τ)]
+ [e1 exp(-τ) - e2] exp(τ t)/[exp(τ) - exp(-τ)]
y2(t) = πcos (π t) – τ [e1 exp(τ)- e2] exp(-τ t)/[exp(τ) - exp(-τ)]
+ τ [e1 exp(-τ) - e2] exp(τ t)/[exp(τ) - exp(-τ)]
Errors in these profiles never exceed t (e1 + e2), and as a result a solution to the BVP is readily obtained.
![Page 167: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/167.jpg)
34
BVP and IVP Profiles
e1, e2 = 10-9
Linear BVP solves easily
IVP blows up before midpoint
![Page 168: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/168.jpg)
35
Dynamic Optimization Dynamic Optimization ApproachesApproaches
DAE Optimization Problem
Multiple Shooting
Sequential Approach
Vassiliadis(1994)
Can not handle instabilities properlySmall NLP
Handles instabilities Larger NLP
Discretize some state variables
Discretize controls
Variational Approach
Pontryagin(1962)
Inefficient for constrained problems
Apply a NLP solver
Efficient for constrained problems
![Page 169: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/169.jpg)
36
Multiple Shooting for Dynamic OptimizationDivide time domain into separate regions
Integrate DAEs state equations over each region
Evaluate sensitivities in each region as in sequential approach wrt uij, p and zj
Impose matching constraints in NLP for state variables over each region
Variables in NLP are due to control profiles as well as initial conditions in each region
![Page 170: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/170.jpg)
37
Multiple ShootingNonlinear Programming Problem
uL
x
xxx
xc
xfn
≤≤
=
ℜ∈
0)(s.t
)(min
( ))(),( min,,
ffpu
tytzji
ψ
( ) z)z(tpuyzfdt
dzjjji ==
,,,, ,
( ) 0,ji, =pz,y,ug
ul
uiji
li
ukkjij
lk
ukkjij
lk
jjjij
ppp
uuu
ytpuzyy
ztpuzzz
ztpuzz
≤≤
≤≤
≤≤
≤≤
=− ++
,
,
,
11,
),,,(
),,,(
0),,,(s.t.
(0)0 zzo = Solved Implicitly
![Page 171: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/171.jpg)
38
BVP Problem Decomposition
Consider: Jacobian of Constraint Matrix for NLP
• bound unstable modes with boundary conditions (dichotomy)
• can be done implicitly by determining stable pivot sequences in multiple shooting constraints approach
• well-conditioned problem implies dichotomy in BVP problem (deHoog and Mattheij)
Bock Problem (with t = 50)
• Sequential approach blows up (starting within 10-9 of optimum)
• Multiple Shooting optimization requires 4 SQP iterations
B1 A1
A2
A3
A4
AN
B2
B3
B4
BN
IC
FC
![Page 172: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/172.jpg)
39
Dynamic Optimization – Multiple Shooting Strategies
Larger NLP problem O(np+nu+NE nz) • Use SNOPT, MINOS, etc.• Second derivatives difficult to get
Repeated solution of DAE model and sensitivity/adjoint equations, scales with nz and np
• Dominant computational cost• May fail at intermediate points
Multiple shooting can deal with unstable systems with sufficient time elements.
Discretize control profiles to parameters (at what level?)
Path constraints are difficult to handle exactly for NLP approach
Block elements for each element are dense!
Extensive developments and applications by Bock and coworkers using MUSCOD code
![Page 173: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/173.jpg)
40
Dynamic Optimization Dynamic Optimization ApproachesApproaches
DAE Optimization Problem
Simultaneous Approach
Sequential Approach
Vassiliadis(1994)
Can not handle instabilities properlySmall NLP
Handles instabilities Large NLP
Discretize all state variables
Discretize controls
Variational Approach
Pontryagin(1962)
Inefficient for constrained problems
Apply a NLP solver
Efficient for constrained problems
![Page 174: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/174.jpg)
41
Nonlinear DynamicOptimization Problem
Collocation onfinite Elements
Continuous variablesContinuous variables
Nonlinear ProgrammingProblem (NLP)
Discretized variablesDiscretized variables
Nonlinear Programming Formulation
![Page 175: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/175.jpg)
42
Discretization of Differential Equations Orthogonal Collocation
Given:dz/dt = f(z, u, p), z(0)=given
Approximate z and u by Lagrange interpolation polynomials (order K+1 and K, respectively) with interpolation points, tk
kkNjk
jK
kjj
k
K
kkkK
kkNjk
jK
kjj
k
K
kkkK
ututt
ttttutu
ztztt
ttttztz
===>−−
∏==
===>−−
∏==
≠==
+
≠==
+
∑
∑
)()(
)()(,)()(
)()(
)()(,)()(
11
100
1
""
""
Substitute zN+1 and uN into ODE and apply equations at tk.
Kkuzftztr kk
K
jkjjk ,...1 ,0),()()(
0
==−= ∑=
"�
![Page 176: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/176.jpg)
43
Collocation Example
kkNjk
jK
kjj
k
K
kkkK ztz
tt
ttttztz ===>
−−
∏== +
≠==
+ ∑ )()(
)()(,)()( 1
001 ""
2
210
22
221
22
2222211200
12
121
12
1122111100
0
2
22
11
02
0
210
76303053371
706073840319029100
2334786
23
2346412098572
23
0
0023
4641203924 ,464102196252
391644836 ,4483621958
613 ,61
7886802113200
t. t - . z(t)
). (. ), z. (. , z z
) z - z z z.(-
z - z) (t z) (t z) (tz
) z - z z. z.(
z - z) (t z) (t z) (tz
z
) , z( z - zSolve z’
.t - . (t) t. - t. (t)
t. - . (t) t. t. -(t)
/ - t/(t) )/ - t (t(t)
. , t. , t t
=
===+=+
+=++
+=+
+=++
===>=+=
==
=+=
=+=
===
"�"�"�
"�"�"�
"�"
"�"
"�"
![Page 177: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/177.jpg)
44
z(t)
z N+1(t)
Sta
te P
rofil
e
t ft 1 t 2 t 3
r(t)
t 1 t 2 t 3
Min φ(z(tf))s.t. z’ = f(z, u, p), z(0)=z0
g(z(t), u(t), p) � 0h(z(t), u(t), p) = 0
to Nonlinear Program
How accurate is approximation
Converted Optimal Control Problem
Using Collocation
0)1(
,...1
0
0
z(0) ,0),()(
0
00
=−
=
=≤
==−
∑
∑
=
=
f
K
jjj
kk
kk
kk
K
jkjj
f
zz
Kk
),uh(z
),ug(z
zuzftz
)(z Min
"
"�
φ
![Page 178: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/178.jpg)
45
Results of Optimal Temperature Program Batch Reactor (Revisited)
Results- NLP with Orthogonal CollocationOptimum B/A - 0.5728# of ODE Solutions - 0.7(Equivalent)
![Page 179: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/179.jpg)
46
to tf
u u u u
Collocation points
• ••• •
•• •
•••
•
Polynomials
u uu u
•
Finite element, i
ti
Mesh points
hi
u u u u
∑=
=K
qiqq(t) zz(t)
0
"
u uu
uelement i
q = 1q = 1
q = 2 q = 2
uuuuContinuous Differential variables
Discontinuous Algebraic and Control variables
u
u
u u
Collocation on Finite ElementsCollocation on Finite Elements
∑=
=K
qiqq(t) yy(t)
1
" ∑=
=K
qiqq(t) uu(t)
1
"
τd
dz
hdt
dz
i
1=
),( uzfhd
dzi=
τ
NE 1,.. i 1,..K,k ,0),,())(()(0
===−= ∑=
K
jikikikjijik puzfhztr τ"�
]1,0[,1
1’’ ∈+= ∑
−
=
ττ ji
i
iiij hht
![Page 180: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/180.jpg)
47
Nonlinear Programming ProblemNonlinear Programming Problem
uL
x
xxx
xc
xfn
≤≤
=
ℜ∈
0)(s.t
)(min( )fzψ min
( ) 0,, ,,, =p,uyzg kikiki
ul
ujiji
lji
uji
lji
ul
ppp
uuu
yyy
zzz
≤≤
≤≤
≤≤
≤≤
,,,
,ji,,
ji, ji,ji,
s.t. ∑=
=−K
jikikikjij puzfhz
0
0),,())(( τ"�
)0( ,0))1((
,..2 ,0))1((
100
,
00,1
zzzz
NEizz
K
jfjjNE
K
jijji
==−
==−
∑
∑
=
=−
"
"
Finite elements,hi, can also be variable to determine break points for u(t).
Add hu � hi � 0, Σ hi=tf
Can add constraints g(h, z, u) � ε for approximation error
![Page 181: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/181.jpg)
48
A + 3B --> C + 3D
L
T s
T R
T P
3:1 B/A 383 K
TP = specified product temperatureTR = reactor inlet, reference temperatureL = reactor lengthTs = steam sink temperatureq(t) = reactor conversion profileT(t) = normalized reactor temperature profile
Cases considered:• Hot Spot - no state variable constraints• Hot Spot with T(t) � 1.45
Hot Spot Reactor Revisited
Roo
P
Pproducto
Rfeed
RS
L
RSTLTT
C/T C, T(L) T
, T(L)) (THC) -,(T+
Tdt
dqTTtT
dt
dT
qtTtqdt
dqts
dtTTtTLMinSRP
101120
0110
1)0( ,3/2)/)((5.1
0)0( )],(/2020exp[))(1(3.0 ..
)/)(( 0
,,,
+==
=∆
=+−−=
=−−=
−−=Φ ∫
![Page 182: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/182.jpg)
49
1.21.00.80.60.40.20.00
1
2
integrated profile
collocation
Normalized Length
Con
vers
ion
1.21.00.80.60.40.20.01.0
1.2
1.4
1.6
1.8
integrated profile
collocation
Normalized Length
Tem
pera
ture
Base Case SimulationMethod: OCFE at initial point with 6 equally spaced elements
L(norm) TR(K) TS(K) TP(K)Base Case: 1.0 462.23 425.26 250
![Page 183: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/183.jpg)
50
1.51.00.50.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Normalized Length
Con
vers
ion,
q
1.51.00.50.01.0
1.1
1.2
1.3
1.4
1.5
1.6
Normalized Length
Nor
mal
ized
Tem
pera
ture
Unconstrained CaseMethod: OCFE combined formulation with rSQP
identical to integrated profiles at optimum L(norm) TR(K) TS(K) TP(K)
Initial: 1.0 462.23 425.26 250Optimal: 1.25 500 470.1 188.4
123 CPU s. (µVax II)φ∗ = -171.5
![Page 184: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/184.jpg)
51
1.51.00.50.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Normalized Length
Con
vers
ion
1.51.00.50.01.0
1.1
1.2
1.3
1.4
1.5
Normalized Length
Tem
pera
ture
Temperature Constrained CaseT(t) � 1.45
Method: OCFE combined formulation with rSQP, identical to integrated profiles at optimum
L(norm) TR(K) TS(K) TP(K)Initial: 1.0 462.23 425.26 250Optimal: 1.25 500 450.5 232.1
57 CPU s. (µVax II), φ∗ = -148.5
![Page 185: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/185.jpg)
52
Theoretical Properties of Simultaneous Method
A. Stability and Accuracy of Orthogonal Collocation
• Equivalent to performing a fully implicit Runge-Kutta integration of the DAE models at Gaussian (Radau) points
• 2K order (2K-1) method which uses K collocation points• Algebraically stable (i.e., possesses A, B, AN and BN stability)
B. Analysis of the Optimality Conditions
• An equivalence has been established between the Kuhn-Tucker conditions of NLP and the variational necessary conditions
• Rates of convergence have been established for the NLP method
![Page 186: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/186.jpg)
53
Case Studies• Reactor - Based Flowsheets• Fed-Batch Penicillin Fermenter• Temperature Profiles for Batch Reactors• Parameter Estimation of Batch Data• Synthesis of Reactor Networks• Batch Crystallization Temperature Profiles• Grade Transition for LDPE Process• Ramping for Continuous Columns• Reflux Profiles for Batch Distillation and Column Design• Source Detection for Municipal Water Networks• Air Traffic Conflict Resolution• Satellite Trajectories in Astronautics• Batch Process Integration• Optimization of Simulated Moving Beds
Simultaneous DAE Optimization
![Page 187: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/187.jpg)
54
Production of High Impact Polystyrene (HIPS)Startup and Transition Policies (Flores et al., 2005a)
Catalyst
Monomer, Transfer/Term. agents
Coolant
Polymer
![Page 188: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/188.jpg)
55
Upper Steady−State
Bifurcation Parameter
System State
Lower Steady−State
Medium Steady−State
Phase Diagram of Steady States
Transitions considered among all steady state pairs
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
300
350
400
450
500
550
600
N1
N2
N3
A1
A4
A2
A3
A5
Cooling water flowrate (L/s)
Te
mp
era
ture
(K
)
1: Qi = 0.00152: Qi = 0.00253: Qi = 0.0040
3
2
1
0 0.02 0.04 0.06 0.08 0.1 0.12
300
350
400
450
500
550
600
650
Initiator flowrate (L/s)
Te
mp
era
ture
(K
)
1: Qcw = 102: Qcw = 1.03: Qcw = 0.1
32 1
3
2
1
N2
B1 B2 B3
C1C2
![Page 189: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/189.jpg)
56
0 0.5 1 1.5 20
0.5
1x 10
−3
Time [h]
Initi
ator
Con
c. [m
ol/l]
0 0.5 1 1.5 24
6
8
10
Time [h]
Mon
omer
Con
c. [m
ol/l]
0 0.5 1 1.5 2300
350
400
Time [h]
Rea
ctor
Tem
p. [
K]
0 0.5 1 1.5 2290
300
310
320
Time [h]
Jack
et T
emp.
[K
]
0 20 40 60 80 1000
0.5
1
1.5x 10
−3
Time [h]
Initi
ator
Flo
w. [
l/sec
]
0 0.5 1 1.5 20
0.5
1
Time [h]Coo
ling
wat
er F
low
. [l/s
ec]
0 0.5 1 1.5 20
1
2
3
Time [h]
Fee
drat
e F
low
. [l/s
ec]
• 926 variables• 476 constraints• 36 iters. / 0.95 CPU s (P4)
Startup to Unstable Steady State
![Page 190: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/190.jpg)
57
HIPS Process Plant (Flores et al., 2005b)
•Many grade transitions considered with stable/unstable pairs
•1-6 CPU min (P4) with IPOPT
•Study shows benefit for sequence of grade changes to achieve wide range of grade transitions.
![Page 191: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/191.jpg)
58
Batch Distillation – Optimization Case Study - 1
D(t), x d
V(t)
R(t)
xb
[ ]
[ ]
1+R
V-=
dt
dS
x-xS
V=
dt
dx
x-yH
V=
dt
dx
i,0i,i,0
i,i,cond
di,
d
dN
*DXJH�HIIHFW�RI�FROXPQ�KROGXSV�2YHUDOO�SURILW�PD[LPL]DWLRQ0DNH�7UD\�&RXQW�&RQWLQXRXV
![Page 192: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/192.jpg)
59
2SWLPL]DWLRQ�&DVH�6WXG\�� �0RGHOLQJ�$VVXPSWLRQV,GHDO�7KHUPRG\QDPLFV1R�/LTXLG�7UD\�+ROGXS1R�9DSRU�+ROGXS)RXU�FRPSRQHQW�PL[WXUH��α �����������������6KRUWFXW�VWHDG\�VWDWH�WUD\�PRGHO��)HQVNH�8QGHUZRRG�*LOOLODQG�
&DQ�EH�VXEVWLWXWHG�E\�PRUH�GHWDLOHG�VWHDG\�VWDWH�PRGHOV�)UHGHQVOXQG DQG�*DOLQGH]���������$OND\D�������
2SWLPL]DWLRQ�3UREOHPV�&RQVLGHUHG(IIHFW�RI�&ROXPQ�+ROGXS��+FRQG�7RWDO�3URILW�0D[LPL]DWLRQ
![Page 193: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/193.jpg)
60
0.0 0.2 0.4 0.6 0.8 1.00
10
20
30
With HoldupNo Holdup
Time ( hours )
Re
flu
x R
atio
0.0 0.2 0.4 0.6 0.8 1.00.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
With HoldupNo Holdup
Time ( hours )
Dis
tilla
te C
om
po
sitio
n
0D[LPXP�'LVWLOODWH�3UREOHP�
&RPSDULVRQ�RI�RSWLPDO�UHIOX[�SURILOHV�ZLWK�DQG�ZLWKRXW�KROGXS��+FRQG��
&RPSDULVRQ�RI�GLVWLOODWH�SURILOHV�ZLWK�DQG�ZLWKRXW�KROGXS��+FRQG��DW�������RYHUDOO�SXULW\�
![Page 194: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/194.jpg)
61
0 1 2 3 40
10
20
30
N = 20N = 33.7
Time ( hours )
Re
flu
x R
atio
%DWFK�'LVWLOODWLRQ�3URILW�0D[LPL]DWLRQMax {Net Sales(D, S0)/(tf +Tset) – TAC(N, V)}
N = 20 trays, Tsetup= 1 hourxd = 0.98, xfeed= 0.50, α = 2Cprod/Cfeed= 4.1V = 120 moles/hr, S0 = 100 moles.
![Page 195: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/195.jpg)
62
%DWFK�'LVWLOODWLRQ� 2SWLPL]DWLRQ�&DVH�6WXG\�� �
D(t), x d
V(t)
R(t)
xb
Ideal Equilibrium Equationsyi,p = Ki,p xi,p
Binary Column (55/45, Cyclohexane, Toluene)S0 = 200, V = 120, Hp = 1, N = 10, ~8000 variables, < 2 CPU hrs. (Vaxstation 3200)
dx1,N+1
dt=
VH N+1
y1,N - x1,N+1[ ]dx1,p
dt=
V
Hp
y1, p-1- y1,p +R
R+1x1,p+1- x1,p
[ ], p = 1,...,N
dx1,0
dt=
V
Sx1,0 - y1,0 + R
R+1x1,1 - x1,0
[ ]
dDdt
=V
R +1
xi,p1
C∑ = 1.0 yi,p
1
C∑ =1.0
S0x i,00 = S0 - Hp
p=1
N+1∑
x i,0 + Hp
p=1
N+1∑ x i,p
![Page 196: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/196.jpg)
63
2SWLPL]DWLRQ�&DVH�6WXG\�� �
&DVHV�&RQVLGHUHG&RQVWDQW�&RPSRVLWLRQ�RYHU�7LPH6SHFLILHG�&RPSRVLWLRQ�DW�)LQDO�7LPH%HVW�&RQVWDQW�5HIOX[�3ROLF\3LHFHZLVH�&RQVWDQW�5HIOX[�3ROLF\
0RGHOLQJ�$VVXPSWLRQV,GHDO�7KHUPRG\QDPLFV&RQVWDQW�7UD\�+ROGXS1R�9DSRU�+ROGXS%LQDU\�0L[WXUH�����WROXHQH����F\FORKH[DQH���KRXU�RSHUDWLRQ7RWDO�5HIOX[�,QLWLDO�&RQGLWLRQ
![Page 197: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/197.jpg)
64
1.00.80.60.40.20.00
1
2
3
4
5
6
0.000
0.001
0.002
0.003
0.004
0.005
0.006
Reflux
Purity
Time (hrs.)
Ref
lux
Pol
icy
Top
Pla
te P
urity
, (1-
x)
1.00.80.60.40.20.00
2
4
6
8
0.000
0.002
0.004
0.006
0.008
0.010
Reflux
Purity
Time (hrs.)
Ref
lux
Pol
icy
Dis
tilla
te P
urit
y, (
1-x)
5HIOX[�2SWLPL]DWLRQ�&DVHV
2YHUDOO�'LVWLOODWH�3XULW\∫[G�W��9��5����GW���'�WI���� �����
' �WI�� ������6KRUWFXW�&RPSDULVRQ
' �WI�� ������
&RQVWDQW�3XULW\�RYHU�7LPH[��W��� �����' �WI�� ������
![Page 198: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/198.jpg)
65
5HIOX[�2SWLPL]DWLRQ�&DVHV
3LHFHZLVH�&RQVWDQW�5HIOX[�RYHU�7LPH∫[G�W��9��5����GW���'�WI���� �����
' �WI�� ������
&RQVWDQW�5HIOX[�RYHU�7LPH[G�W��9��5����GW���'�WI���� �����' �WI�� �����
1.00.80.60.40.20.00.20
0.22
0.24
0.26
0.28
0.30
0.000
0.001
0.002
0.003
0.004
Reflux
Purity
Time (hrs.)
Ref
lux
Pol
icy
Dis
tilla
te P
urit
y, (
1-x)
1.00.80.60.40.20.00.000
0.002
0.004
0.006
0.008
0.010
0
1
2
3
4
5
6
Purity
Reflux
Time (hrs.)
Ref
lux
Pol
icy
Dis
tilla
te P
urit
y, (
1-x)
![Page 199: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/199.jpg)
66
Batch Reactive Distillation – Case Study 3
max Ddtt f
0
1=∫max Ddtt f
0
1=∫
s.t. DAE
xDEster ≥ 0 4600.xD
Ester ≥ 0 4600.
Reversible reaction between acetic acid and ethanolReversible reaction between acetic acid and ethanol
CH3COOH + CH3CH2OH l CH3COOCH2CH3 + H2O
t = 0, x = 0.25for all components
Wajde & Reklaitis (1995)
D(t), x d
V(t)
R(t)
xb
![Page 200: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/200.jpg)
67
2SWLPL]DWLRQ�&DVH�6WXG\�� �
&DVHV�&RQVLGHUHG6SHFLILHG�&RPSRVLWLRQ�DW�)LQDO�7LPH2SWLPXP�5HIOX[�3ROLF\9DULRXV�7UD\V�&RQVLGHUHG��������������KRXU�RSHUDWLRQ
0RGHOLQJ�$VVXPSWLRQV,GHDO�7KHUPRG\QDPLFV&RQVWDQW�7UD\�+ROGXS1R�9DSRU�+ROGXS7HUWLDU\�0L[WXUH��(W2+��+2$F��(7$F��+�2�&ROG�6WDUW�,QLWLDO�&RQGLWLRQ
![Page 201: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/201.jpg)
68
Batch Reactive Distillation
Distillate Composition
0.25
0.35
0.45
0.0 0.2 0.4 0.6 0.8 1.0time (hr)
Eth
yl A
ceta
te
8 trays 15 trays 25 trays
Optimal Reflux Profiles
0
20
40
60
80
100
0.0 0.2 0.4 0.6 0.8 1.0
time (hr)
Ref
lux
Rat
io
8 trays 15 trays 25 trays
0.0 0.2 0.4 0.6 0.8 1.0
0.00
0.10
0.20
0.30
0.40
0.50
M. f
ract
ion
time(hr)
Condenser Composition (8 trays)
Ester Eth. Wat. A. A.
�������YDULDEOHV������'$(V���GHJUHHV�RI�IUHHGRP���ILQLWH�HOHPHQWV�����,3237�LWHUDWLRQV�����&38�PLQXWHV
![Page 202: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/202.jpg)
69
CPU Decomposition Time
1
6
11
16
21
90 130 170 210 250
DAEs
CP
U(s
)/ite
r
DiscretizedVariables
Iterations CPU (s)
3048
1788
245.732
DAEsTrays
168
8 98
15
56.414
4678 1083.24525825
207.5
37.2
659.3
Global Elemental
Batch Reactive Distillation
![Page 203: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/203.jpg)
70
Nonlinear Model Predictive Control (NMPC)
Process
NMPC Controller
d : disturbancesz : differential statesy : algebraic states
u : manipulatedvariables
ysp : set points
( )( )dpuyzG
dpuyzFz
,,,,0
,,,,
==′
NMPC Estimation and Control
sConstraintOther
sConstraint Bound
)()),(),(),((0
)),(),(),(()(..
||))||||)(||min
init
1sp
ztztttytzG
tttytzFtzts
yty uy Q
kk
Q
==
=′
−+−∑ ∑ −
u
u
u(tu(tu
NMPC Subproblem
Why NMPC?
� Track a profile
� Severe nonlinear dynamics (e.g, sign changes in gains)
� Operate process over wide range (e.g., startup and shutdown)
Model Updater( )( )dpuyzG
dpuyzFz
,,,,0
,,,,
==′
![Page 204: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/204.jpg)
71
Dynamic optimization in a MATLAB Framework
Dynamic Optimization Problem
Process Model
Inequality Constraints
Initial Conditions
Constraints at Final Time
Objective Function
( ) 0puyxxf =′ t,,,,,
( ) 0=t,,,, puyxg
0),,,,,( ≤′ tpuyxxh
0xx =)( 0t
( ) 0,,,)(,)(,)(,)( 0 =′ fffff ttttt pxuyxxϕ
( )ffft
tttf
,,),(t,)(,)( P 0f,,(t),
min0
xpuyxxpu
NLP Optimization Problem
Process Model
Inequality Constraints
Constraints at Final Time
Objective Function
( ) 0xpuyxf 0 =,,,ˆ,ˆ,ˆˆft
( ) 0puyxg =ft,,ˆ,ˆ,ˆˆ
0),,ˆ,ˆ,ˆ(ˆ ≤ftpuyxh
( ) 0,,,, =fNNN tttt
puyxϕ
( )fNNN tttt
,,,, Pmin puyx
Full Discretization of State and
Control Variables
Discretization Method
No. of Time Elements
Collocation Order
Saturator-System
copy ofDesign
WärmeschaltplanNr. -F Ref T - Erlangen, 13.Oct.1999
SIEMENS AGF Ref T In Bearbeitung
P..Druck..bar
M..Massenstrom..kg/s
PHI..Luft-Feuchte..%
H..Enthalpie..kJ/kg
T..Temperatur..°C
bar kJ/kgkg/s °C (X)
JOBKENNUNG : C:\Krawal-modular\IGCC\IGCC_Puertol lano_komplett.gek
Alle Drücke sind absolut
Dynamic Process
![Page 205: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/205.jpg)
72
Tennessee Eastman Process
Unstable Reactor
11 Controls; Product, Purge streams
Model extended with energy balances
![Page 206: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/206.jpg)
73
Tennessee Eastman Challenge Process
Method of Full Discretization of State and Control Variables
Large-scale Sparse block-diagonal NLP
11Difference (control variables)
141Number of algebraic equations
152Number of algebraic variables
30Number of differential equations
DAE Model
14700Number of nonzeros in Hessian
49230Number of nonzeros in Jacobian
540Number of upper bounds
780Number of lower bounds
10260Number of constraints
109200
Number of variablesof which are fixed
NLP Optimization problem
![Page 207: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/207.jpg)
74
Setpoint change studies
Setpoint changes for the base case [Downs & Vogel]
+2%Make a step change so that the composition of component B in the gas purge changes from 13.82 to 15.82%
StepPurge gas composition of component B change
-60 kPaMake a step change so that the reactor operating pressure changes from 2805 to 2745 kPa
StepReactor operating pressure change
-15%Make a step change to the variable(s) used to set the process production rate so that the product flow leaving the stripper column base changes from 14,228 to 12,094 kg h-1
StepProduction rate change
MagnitudeTypeProcess variable
![Page 208: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/208.jpg)
75
Case Study:Change Reactor pressure by 60 kPa
Control profiles
All profiles return to their base case values
Same production rate
Same product quality
Same control profile
Lower pressure – leads to larger gas phase (reactor) volume
Less compressor load
![Page 209: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/209.jpg)
76
TE Case Study – Results I
Shift in TE process
Same production rate
More volume for reaction
Same reactor temperature
Initially less cooling water flow (more evaporation)
![Page 210: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/210.jpg)
77
Case Study- Results II
Shift in TE process
Shift in reactor effluent to more condensables
Increase cooling water flow
Increase stripper steam to ensure same purity
Less compressor work
![Page 211: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/211.jpg)
78
Case Study: Change Reactor Pressure by 60 kPa
Optimization with IPOPT
1000 Optimization Cycles
5-7 CPU seconds
11-14 Iterations
Optimization with SNOPT
Often failed due to poor conditioning
Could not be solved within sampling times
> 100 Iterations
![Page 212: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/212.jpg)
79
+ Directly handles interactions, multiple conditions+ Trade-offs unambiguous, quantitative
- Larger problems to solve - Need to consider a diverse process models
Research Questions
How should diverse models be integrated?Is further algorithmic development needed?
Optimization as a Framework for IntegrationOptimization as a Framework for Integration
![Page 213: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/213.jpg)
80
:KDW�DUH�WKH�,QWHUDFWLRQV�EHWZHHQ�'HVLJQ�DQG�'\QDPLFV�DQG�3ODQQLQJ"
:KDW�DUH�WKH�GLIIHUHQFHV�EHWZHHQ�6HTXHQWLDO�DQG�6LPXOWDQHRXV�6WUDWHJLHV"
(VSHFLDOO\�,PSRUWDQW�LQ�%DWFK�6\VWHPV
Batch Integration Case Study
Production PlanningStage 1
Stage 2
A
A B
B C
C
Plant Design
TR
Time Time
Dynamic Processing
![Page 214: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/214.jpg)
81
GLVFUHWL]H �'$(V���VWDWH�DQG�FRQWURO�SURILOHVODUJH�VFDOH�RSWLPL]DWLRQ�SUREOHPKDQGOHV�SURILOH�FRQVWUDLQWV�GLUHFWO\LQFRUSRUDWHV�HTXLSPHQW�YDULDEOHV�GLUHFWO\�'$(�PRGHO�VROYHG�RQO\�RQFHFRQYHUJHV�IRU�XQVWDEOH�V\VWHPV
6LPXOWDQHRXV�'\QDPLF�2SWLPL]DWLRQ
Best transient Best constant
Higher conversion in same time
T
Time
Con
v.
Time
Fewer product batches
TSame conversion in reduced time
Time
Con
v.
Time
Shorter processing times
Dynamic Processing
Production PlanningStage 1
Stage 2
A
A B
B C
C
Shorter Planning Horizon
![Page 215: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/215.jpg)
82
Scheduling Formulation
VHTXHQFLQJ�RI�WDVNV��SURGXFWV�HTXLSPHQWH[SHQVLYH�GLVFUHWH�FRPELQDWRULDO�RSWLPL]DWLRQFRQVLGHU�LGHDO�WUDQVIHU�SROLFLHV��8,6�DQG�=:�FORVHG�IRUP�UHODWLRQV��%LUHZDU DQG�*URVVPDQQ�������
A B Nstage I
stage 2
stage I
stage 2
A B N
=HUR�:DLW��=:��,PPHGLDWH�WUDQVIHU�UHTXLUHG6ODFN�WLPHV�GHSHQGHQW�RQ�SDLU/RQJHU�SURGXFWLRQ�F\FOH�UHTXLUHG
8QOLPLWHG�,QW��6WRUDJH�8,6�6KRUW�SURGXFWLRQ�F\FOH&\FOH�WLPH�LQGHSHQGHQW�RI�VHTXHQFH
![Page 216: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/216.jpg)
83
Case Study ExampleCase Study Example
z i ,I0 zi ,II
0z i ,III
0 zi ,IV0
z i ,IVf
zi ,If z i ,II
f z i ,IIIf
Bi
A + B → C
C + B → P + E
P + C → G
4 stages, 3 products of different purity Dynamic reactor - temperature profileDynamic column - reflux profile
Process Optimization Cases
SQ - Sequential Design - Scheduling - DynamicsSM - Simultaneous Design and Scheduling
Dynamics with endpoints fixed . SM* - Simultaneous Design, Scheduling and Dynamics
![Page 217: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/217.jpg)
84
5
10
15
20
25
580
590
600
610
620
630
640
0 0.5 1 1.5 2 2.50 0.25 0.5 0.75 1 1.25Tim e (h r.)
Dyn a mic
C o nsta nt
Dyn a mic
C o nsta nt
&RPSDULVRQ�RI�'\QDPLF�YV��%HVW�&RQVWDQW�3URILOHV5��� EHVW�FRQVWDQW�WHPSHUDWXUH�SURILOH5��� RSWLPDO�WHPSHUDWXUH�SROLF\&��� EHVW�FRQVWDQW�UHIOX[�UDWLR&��� RSWLPDO�UHIOX[�UDWLR
6FHQDULRV�LQ�&DVH�6WXG\
![Page 218: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/218.jpg)
85
Results for Simultaneous Cases
R0/R1 best constant / optimal temperature
C0/C1best constant / optimal reflux ratio
0
5 0 0 0
1 0 0 0 0
1 5 0 0 0
2 0 0 0 0
Pro
fit (
$)
R 0 C 0 R 1 C 0 R 0 C 1 R 1 C 1
C a s e s
S Im u lt a n e o u s w it h F R E E s t a t e s
S im u lt a n e o u s w it h F I X E D s t a t e sS e q u e n t ia l
I
I II I I
I V
I
I II I I
I V
II II I I
I V
- ZW schedule becomes tighter
- less dependent on product sequences
64
60
60
![Page 219: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/219.jpg)
86
SummarySequential Approaches- Parameter Optimization
• Gradients by: Direct and Adjoint Sensitivity Equations- Optimal Control (Profile Optimization)
• Variational Methods• NLP-Based Methods
- Require Repeated Solution of Model- State Constraints are Difficult to Handle
Simultaneous Approach- Discretize ODE's using orthogonal collocation on finite elements (solve larger optimization problem)- Accurate approximation of states, location of control discontinuities through element placement.- Straightforward addition of state constraints.- Deals with unstable systems
Simultaneous Strategies are Effective- Directly enforce constraints- Solve model only once- Avoid difficulties at intermediate points
Large-Scale Extensions- Exploit structure of DAE discretization through decomposition- Large problems solved efficiently with IPOPT
![Page 220: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/220.jpg)
87
ReferencesBryson, A.E. and Y.C. Ho, Applied Optimal Control, Ginn/Blaisdell, (1968).Himmelblau, D.M., T.F. Edgar and L. Lasdon, Optimization of Chemical Processes, McGraw-Hill, (2001).Ray. W.H., Advanced Process Control, McGraw-Hill, (1981).
Software• Dynamic Optimization CodesACM – Aspen Custom ModelerDynoPC - simultaneous optimization code (CMU)COOPT - sequential optimization code (Petzold)gOPT - sequential code integrated into gProms (PSE)MUSCOD - multiple shooting optimization (Bock)NOVA - SQP and collocation code (DOT Products)• Sensitivity Codes for DAEsDASOLV - staggered direct method (PSE)DASPK 3.0 - various methods (Petzold)SDASAC - staggered direct method (sparse)DDASAC - staggered direct method (dense)
DAE Optimization Resources
![Page 221: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/221.jpg)
88
Interface of DynoPC
Outputs/Graphic & TextsModel in FORTRAN
F2c/C++
ADOL_C
Preprocessor
DDASSL
COLDAE
Data/Config.
Simulator/Discretizer
F/DF
Starting Point Linearization
Optimal Solution
FILTER
Redued SQP
F/DFLine Search
Calc. of independent variable move
Interior PointFor QP
Decomposition
DynoPC – Windows Implementation
![Page 222: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/222.jpg)
89
A B C1 2
da
atk
E
RTa= − − ⋅1
1exp( )
db
atk
E
RTa k
E
RTb= − ⋅ − − ⋅1
12
2exp( ) exp( )
a + b + c = 1
..
)(
ts
tbMax f
T
Example: Batch Reactor Temperature
![Page 223: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/223.jpg)
90
Example: Car Problem
Min tfs.t. z1’ = z2
z2’ = uz2 � zmax
-2 � u � 1
subroutine model(nz,ny,nu,np,t,z,dmz,y,u,p,f)double precision t, z(nz),dmz(nz), y(ny),u(nu),p(np)
double precision f(nz+ny)
f(1) = p(1)*z(2) - dmz(1)f(2) = p(1)*u(1) - dmz(2)
returnend403020100
-3
-2
-1
0
1
2
0
2
4
6
8
10
12
Acceleration
Velocity
Time
Acc
eler
atio
n, u
(t)
Vel
ocit
y
![Page 224: Nonlinear Programming: Concepts, Algorithms and Applications](https://reader031.vdocuments.us/reader031/viewer/2022020118/586a14401a28ab063d8b7920/html5/thumbnails/224.jpg)
91
Example: Crystallizer Temperature
Steam
Cooling water
TT TC
Control variable = Tjacket = f(t)?
Maximize crystal sizeat final time
0.0E+00
5.0E-04
1.0E-03
1.5E-03
2.0E-03
2.5E-03
3.0E-03
3.5E-03
4.0E-03
4.5E-03
0 5 10 15 20
Time (hr)
Cry
stal
siz
e
0
10
20
30
40
50
60
T ja
cket
(C)
Crystal Size T jacket
SUBROUTINE model(nz,ny,nu,np,x,z,dmz,y,u,p,f)implicit double precision (a-h,o-z)double precision f(nz+ny),z(nz),dmz(nz),Y(ny),yp(4),u(1)double precision kgr, ln0, ls0, kc, ke, kex, lau, deltT, alphadimension a(0:3), b(0:3)data alpha/1.d-4/,a/-66.4309d0, 2.8604d0, -.022579d0, 6.7117d-5/,+ b/16.08852d0, -2.708263d0, .0670694d0, -3.5685d-4/, kgr/ 4.18d-3/,+ en / 1.1d0/, ln0/ 5.d-5/, Bn / 3.85d2/, em / 5.72/, ws0/ 2.d0/,+ Ls0/ 5.d-4 /, Kc / 35.d0 /, Kex/ 65.d0/, are/ 5.8d0 /,+ amt/ 60.d0 /, V0 / 1500.d0/, cw0/ 80.d0/,cw1/ 45.d0/,v1 /200.d0/,+ tm1/ 55.d0/,x6r/0.d0/, tem/ 0.15d0/,clau/ 1580.d0/,lau/1.35d0/,+ cp/ 0.4d0 /,cbata/ 1.2d0/, calfa/ .2d0 /, cwt/ 10.d0/
ke = kex*area x7i = cw0*lau/(100.d0-cw0)v = (1.d0 - cw0/100.d0)*v0 w = lau*v0 yp(1) = (deltT + dsqrt(deltT**2 + alpha**2))*0.5d0yp(2) = (a(0) + a(1)*yp(4) + a(2)*yp(4)**2 + a(3)*yp(4)**3)yp(3) = (b(0) + b(1)*yp(4) + b(2)*yp(4)**2 + b(3)*yp(4)**3)deltT = yp(2) - z(8) yp(4) = 100.d0*z(7)/(lau+z(7))
f(1) = Kgr*z(1)**0.5*yp(1)**en - dmz(1)f(2) = Bn*yp(1)**em*1.d-6 - dmz(2)f(3) = ((z(2)*dmz(1) + dmz(2) * Ln0)*1.d+6*1.d-4) -dmz(3)f(4) = (2.d0*cbata*z(3)*1.d+4*dmz(1)+dmz(2)*Ln0**2*1.d+6)-dmz(4)f(5) = (3.d0*calfa*z(4)*dmz(1)+dmz(2)*Ln0**3*1.d+6) - dmz(5)f(6) = (3.d0*Ws0/(Ls0**3)*z(1)**2*dmz(1)+clau*V*dmz(5))-dmz(6)f(7) = -dmz(6)/V - dmz(7)f(8) = (Kc*dmz(6) - Ke*(z(8) - u(1)))/(w*cp) - dmz(8)f(9) = y(1)+YP(3)- u(1) returnend