lecture 17-18 primal methods - solmaz s. kiasolmaz.eng.uci.edu/teaching/mae206/lecture17-18.pdf ·...
TRANSCRIPT
![Page 1: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/1.jpg)
Lecture 17-18Primal methods
Solmaz KiaMechanical and Aerospace Eng. Dept.,
University of California Irvine
Consult: Section 2.3 of Ref[1] and Sections 12.1,12.2 and 12.4 of Ref[2]
![Page 2: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/2.jpg)
Primal Methods for constraint 4timizationWe consider the problem . . ?>7ô
. Iminf(x), LI f’k c
g(x)<O ‘ ‘,— )
I ?,q4ajItQ 1h(x)=O t,‘> P/i
/Y
—:---U::T—; d140By a primal method we mean a search method that works bysearching through the feasible region.First Order Necessary Condition for Optimality: x is a local minimizer then
Vf(x*)TL\x ; 0, for I\x e V(x*)
. Set of first order feasible variations at x
V(x) ={d e R f Vhj,(x)Td = 0, Vg(x)Td 0, j e A(x*)}
. Active inequality constraints at x
A(x) = {j E {1, . . . , r} gj (x) = O}
![Page 3: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/3.jpg)
Primal Methods for constraint optimization
We consider the problem
mm f(x)g(x)Oh(x)=O
By a primal method we mean a search method that works bysearching through the feasible region.
![Page 4: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/4.jpg)
; I •; i
I S
\C’ “ ‘1
Advantages “
. If the process is terminated before reaching the solution, the. . . 4-current point is feasible. V — ‘j
. It can often be guaranteed that if the sequence of pointsconverges, then the limit is a local constrained minimum. 7
• Most of the primal methods do not rely on special structure,such as convexity.
Disadvantages• Needs a feasible starting point.• It can be computationally hard to remain in the feasible region.
![Page 5: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/5.jpg)
Feasible Direction MethodsThe idea of feasible direction methods is the same as with
) L
where dk 5 a feasible direction at xk, and ük 0.
is chosen to minimize f with the restriction that the point xk+1,and the line segment joining xk and xk+1 be feasible.
\>
unconstrained problems:,7)
Xk+1 Xk+kdkS
OPT: x =argmin f(x)xR
x E X (X is the set of constraints)for X = RU (problem becomes unconstrained) /
D RTh is a feasibledirection at x e X for OPTif(x+ocd) Xfor
&
LXEE [O,]
![Page 6: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/6.jpg)
c)
A Feasible Direction Method: Simplified Zoutendijk method4Consider the a problem with linear constraints:
Given a feasible point Xk, let A(Xk) be the set of indices of active constraints,aj1xk b for 1 e A(Xk).
• The other constraints assure that Xk + dk will be feasible for sufficientlysmall k > 0.
minimize f(x)
u[xb, 1=1,••• ,rS t. fr- 4
The direction vector dk is then chosen as the solution of
5r\ )Lc 52.l minimize Vf(xk)Td
ci’
v
s.t.
i e A(xk),ad0,
Idj1=
LL-e
2Th ‘
;
j4CIL)
j=1
I
bounded solution
.-1_ irJ_/(
.,‘\Ot,’ k
c, LII’
• The last equation (which can be converted to linear constraints) ensures a
• The objective function makes d as close to Vf(Xk) as possible.
![Page 7: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/7.jpg)
Feasible Direction Methods
There are two major shortcomings of feasible direction methods inthis form.. For general problems there may not exist any feasibledirections (see figure). ?‘ (‘‘
. They are also vulnerable to ,,jamming”, or ,,zigzagging”, whichwe have also seen with unconstrained problems, but here itmight converge to a point that is not even a constrained localminimum (the algorithmic map is not closed).
Feasibleset
![Page 8: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/8.jpg)
S
c)
IV
:c-
I
S
![Page 9: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/9.jpg)
r V
E r
1
2—.
0
V)
N
Ct
r
—r
f0
ci
Tp
11 0
2UJ
bj4J
9 p
![Page 10: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/10.jpg)
(N
IL I-1-11
I’
(
i-cD
•
-
ii‘s--‘
4
\\-
1
11 S \co
+I, ±
q
+çC
>
F
c1
4-
3
00
![Page 11: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/11.jpg)
1:___
r—cD-
%‘/
&c:s
_%$_J
9%1)
EDp
0
ac0U’
c
b
c_.
‘I
Ii
c)
![Page 12: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/12.jpg)
/
y-
U
U
03
U
![Page 13: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/13.jpg)
: $
±__k
r
k-
%rt-
r.
;:
‘I
4C:
:Lc
0
;,0
:S.
9S
454
r>-‘
..—
9
0
—
S
-11•’-r’O’
p-.
Icc;
![Page 14: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/14.jpg)
‘:,
‘f\
S
r—i
-i--L_J
LJ
0
/
LE
00I-
7II
1ic)c0
0
N
*
![Page 15: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/15.jpg)
cc *
I(C
a—T
‘%—
____
21
Th
--
r
![Page 16: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/16.jpg)
ito
U4:_k c,Lt&(’L
![Page 17: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/17.jpg)
Va
ft
‘I
‘
‘cicLQt
(‘1
0f’
ULI
(—_‘1Th
r
c±l
6
c’c
ra—’
6).
.,
ry__
4s
‘,
c
,;
.
Lc’
!
3_
.
;:3__r
>-,
:h,
s,
r’:r
,,c__
r’i
q:
f‘
%;:
-2—
•
_t-
.‘
3:(].
t?
.c;’.
tV
,‘\
.‘
0’?
\%—.,
.r?
(Cb
::ri
6•.
r’0
‘)—
r4
Lrc’
\‘—‘
S
S
rCI
\r
“
_‘_
-#
‘
r::
a•‘
c__%,
-rs
F-
11•T
?,
LJ
:
![Page 18: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/18.jpg)
F
,
0 1’
T_
zN
(t\
z
0
cc
4-
1 LJ
t
oL ft
\:<c2 I
(%)
: 1 ,‘
cr T
r t.
<0) 3—
-r;
![Page 19: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/19.jpg)
Th t
1
p A U )
Ac
jc (c
rOr
.4-r
(IL
mUI
.÷‘N
5
Cri
r
0 ci
x
1-
V
. (t.
‘
.
(cjJ
‘2
R,
O4-
%-c
![Page 20: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/20.jpg)
J
jj
%)
aI0
n
L‘3
c;4DJ
IcL‘
(Dr
1:I
cJ1
0
Hø—Th:
4
![Page 21: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/21.jpg)
fl4
04
.4
4.c4I’gLi
0
(Th-Jii
-‘
-Th
L
0Th
%
.—0c‘—h
%%J0
oJ
%%4-4
I;:
J‘
.%.-
a’
‘c-%
_)
ksi—
t)o
D,
‘S\_AT—
rL?(
4.-1’a
(Th
![Page 22: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/22.jpg)
C,--
ct
o0
II>
-
‘1
V_
)-
II
“I
![Page 23: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/23.jpg)
V)ì‘\J
_•%
A
U
TD
-Th
i
sJ
1—
L
=‘45
I
r
4
0
C
jp
o
![Page 24: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/24.jpg)
Xk1ke ri hci
L
k().
(
‘:c1 r(L)
-2-
![Page 25: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/25.jpg)
i) 4
c)
Ii 9r
-I’ o5 >
cc q
4
Os
‘
E
9 4
\1 4
\c
‘\I2\s \
c :‘s$\-°
2 L—
4”
Is
E
9 ‘\
11
![Page 26: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/26.jpg)
\
N\‘
it
I1Hz
ii
S
0‘I
—1
>)
4’
F1
> b
11
&\
\\
ii
![Page 27: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/27.jpg)
D
c aI%-
\\‘
‘F[
0
‘Ici.
0r
rCii
Uft
‘%__I
LA
k
__
4\
Csr
‘
Cl)%.
![Page 28: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/28.jpg)
PRACTICAL AUGMENTED LAGRANGIAN METHODS:BOUND-CONSTRAINED FORMULATION
minimize f(x)
h(x)=O, l<x<u
LA (x,;tk) = f(x)+rn , rn
Akh(x)+kh(x)2
Bounded Gradient Lagrangian method
; Ik ) subject to
l<x<u
Ak+l Ak +kh(Xk)
![Page 29: Lecture 17-18 Primal methods - Solmaz S. Kiasolmaz.eng.uci.edu/Teaching/MAE206/Lecture17-18.pdf · Idj1= LL-e 2Th ‘; j4CIL) j=1 I bounded solution.-1_ ir J_/(.,‘\ Ot,’ k c,](https://reader034.vdocuments.us/reader034/viewer/2022042303/5ecea1db4981711f50325a37/html5/thumbnails/29.jpg)
Ak+1 Ak;
k+1 1OO;1 iaOA
11k+1Wk+1 l/&k+1;
(Bound-Constrained Lagrongian Method). I here is the projection operator tor boxed I. , . . . . * . 0 inequality (check your notes on gradient
Choose an initial point x0 and initial multipliers A ; projection method for further details)Choose convergence tolerances i4, and w;Set Co = 10, w = 1/ , and i = if ‘;fork = 0, 1, 2, .
ar0mate solution Xk subproble mm £A(x,Ak;) subject to I x U
IIxk (xkV1CA(xk,A’;Ck),1. <wk;
An efficient technique forsolving the nonlinear programwith bound constraints(forfixed .t andA) is the(nonlinear) gradient projectionmethod (see your notes fromlectures on primal methods)
ifllh(xk)1I 1IkNN
( * test for cwergence )if IIh(xk ) Ii s ‘7* Xk ‘ (Xk VItA (Xk , Ak ; k )‘ 1, u) II <
stop with apprimte solution xk;end(if)(* update multipliers, tighten tokq.ices )Ak+l Ak + kh(xk);
Remember that gradient projection method stopswhen the point generated from Xk by the gradientdescent algorithm gets projected back on Xk
Check the stopping condition of the gradientprojection method in your notes for more details.
else
L.)k+1Io9
Ilk-i-i = TIk/L..’k+i,WkI1 Wk/k+1
(* increase penalty parameter, tighten tolerances
If this condition holds, the penalty parameter is not changed forthe next iteration because the current value of Pk is producing anacceptable level of constraint violation. The Lagrange multiplierestimates are updated according to the update formula and thetolerances Wk and flk are tightened in advance of the nextiteration. If, on the other hand, this condition does nothold, then we increase the penalty parameter to ensure that thenext subproblem will place more emphasis on decreasing theconstraint violations. The Lagrange multiplier estimates are notupdated in this case; the focus is on improving feasibility.
end (if)end (for)
The constants 100, 0.1, 1 appearing hereare to some extent arbitrary;
—-—-———-—-—-—-———————-— other values can be used withoutcompromising theoretical convergenceproperties.