continuous optimization techinique
TRANSCRIPT
-
8/18/2019 continuous optimization techinique
1/32
EPE-821
1
Dr. Muhammad Naeem
Dr. Ashfaq Ahmed
Continuous Optimization Techniques
-
8/18/2019 continuous optimization techinique
2/32
2
Outline
Review of Math
Basics of Optimization
Power systems basics
Review of Matab
!"ampes of Optimization in Power systems
#raphica Optimization
Optimization $ypes %onstraint and &nconstraint
Optimization Probem $ypes 'inear Non'inear etc
'inear Optimization and Power (ystems Appications
Non 'inear Optimization and Power (ystems Appications
)nte*er and Mi"ed inte*er pro*rammin* and Power (ystems Appications
%ompe"ity Anaysis
+uiz ne"t wee,
-
8/18/2019 continuous optimization techinique
3/32
Text%acuus and Anaytic #eometry By $homas and -inney th !dition
)ntroduction of Optimum Desi*n By /asbir (. Arora
(ome fi*ure from 0eb
Applied Numerical Methods ith MAT!A"# $or En%ineers and
&cientists 'rd Edition( &te)en C* Chapra
http122www.ece.mcmaster.ca23"wu2part4.pdf
Noninear Optimization with -inancia Appications by Michae Barthoomew5
Bi**s
'
-
8/18/2019 continuous optimization techinique
4/32
+
Optimialit, Condition(uppose that - 6"7 is a continuousy differentiabe function of the scaar
variabe "8 and that8 when " 9 ":8
2
20 0 (1)
dF d F and
dx dx= ≥
Above two conditions are ,nown as optimaity conditions
%onditions 6;7 impy that -6":7 is the smaest vaue of - in some re*ion
near ": . )t may aso be true that - 6": 7 < - 6"7 for a " but condition 6;7
does not *uarantee this.
Definition )f conditions 6;7 hod at " 9 ": and if -6":7 = -6"7 for a "
then ": is said to be the *oba minimum.
-
8/18/2019 continuous optimization techinique
5/32
Necessar, and &ucient Conditions
.
Example1 )f a number is divisibe by 46ca this >78 then it is divisibe by ?
6ca this %7. > impies % but % is not stron* enou*h to impy > for e"ampe
@ is divisibe by ? but not by 4. $herefore % is necessary for > but notsufficient for >.
Example: )n !ucidean *eometry8 a trian*e has equa sides 6ca this >7 if
and ony if the trian*e has equa an*es 6ca this %7. $his means
> if %1 > is necessary for % since % impies >
> ony if %1 % is necessary for > since > impies %
> if and ony if %1 > and % impy each other and are both necessary and
sufficient conditions for each other.
-
8/18/2019 continuous optimization techinique
6/32
Necessar, and &ucient Conditions
/
Example1 )f a number is divisibe by 46ca this >78 then it is divisibe by ?
6ca this %7. > impies % but % is not stron* enou*h to impy > for e"ampe
@ is divisibe by ? but not by 4. $herefore % is necessary for > but notsufficient for >.
Example: )n !ucidean *eometry8 a trian*e has equa sides 6ca this >7 if
and ony if the trian*e has equa an*es 6ca this %7. $his means
> if %1 > is necessary for % since % impies >
> ony if %1 % is necessary for > since > impies %
> if and ony if %1 > and % impy each other and are both necessary and
sufficient conditions for each other.
)n the conte"t of smooth functions 6Differentiabe functions7 the condition f 6"7 is a necessary condition for a reative ma"imum or minimum. 6$his is
not sufficient because zero sope may aso at infection point.
)n the conte"t of smooth functions 6Differentiabe functions7 the condition
f6"7 is a sufficient condition if f6"7 9 C.
-
8/18/2019 continuous optimization techinique
7/320
irect search and %radient methods
Both are iterative methods
direct search techniques are based on simpe comparison of function
vaues at tria points.
*radient methods. $hese use derivatives of the obective function and can
be viewed as iterative a*orithms for sovin* the noninear equation
#radient methods tend to conver*e faster than direct search methods.
$hey aso have the advanta*e that they permit an obvious conver*ence
test 5 namey stoppin* the iterations when the *radient is near zero.
0
dF
dx =
-
8/18/2019 continuous optimization techinique
8/328
Example
(oution1
-ind the minimum and ma"imum of for3 23 x x− [ ]1, 4 x ∈ −
-
8/18/2019 continuous optimization techinique
9/32
Example-ind the minimum and ma"imum of for
3 23 x x−
(oution1
3 2
2
3
3 6 0 0 2
6 6
(0) 0
(2) 0
F x x
F x x at x and x
F x
F Local Maximum
F Local Minimum
= −
′ = − = ⇒ = =
′′ = −
′′ ≤
′′ ≥
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15
-10
-5
0
5
10
15
20
25x3-3*x
2
Orig
1st Derv.
2nd Derv.
[ ]1, 4 x ∈ −
-
8/18/2019 continuous optimization techinique
10/3213
"isection Method to 4ind the 5oot
Ony suitabe for functions that has ony one minimum2ma"imum in the ran*e Ea8 bF
#iven a brac,eted root8 the method repeatedy haves the interva whie
continuin* to brac,et the root and it wi conver*e on the soution.
-
8/18/2019 continuous optimization techinique
11/3211
"isection Method to 4ind the 5oot1.Choose x
l and x
u as two guesses for the root such
that f ( xl ) f( x
u) < 0, or in other words, f(x) changes
sign between xl and xu.2.Estimate the root , x
m of the equation f (x) 0 as the
mid!"oint between xl and x
u as
3. #f f ( xl ) f ( x
m ) $ 0, then the root %ies between x
l and
xm& then x
l = x
l ; x
u = x
m.
E%se #f f( xl ) f( x
m) ' 0, then the root %ies between x
m
and u& then x
l = x
m; x
u = x
u.
E%se #f f (%) f(
m) 0& then the root is x
m.to" the
a%gorithm if this is true.
4. *et new estimate
+. bso%ute -e%atie ""roimate Error
6. Chec/ if error is %ess than "re!s"ecified to%eranceor if maimum number of iterations is reached
2
l um
x x x
+=
f()
u
m
2l u
m x x x +=100×
−=∈
new
m
old
m
new
a x
x xm
rootof estimatecurrent=new
m x
rootof estimate "reious=old m x
-
8/18/2019 continuous optimization techinique
12/3212
"isection Method to 4ind the 5oot
-
8/18/2019 continuous optimization techinique
13/32
1'
"isection Method( ) 423 10(3316+0 -.+ x.- x x f =
Choose the 6rac7et
( )
( ) 4
4
10662.211.0
103.30.011.0
00.0
−
−
−=
==
=
f
f x
x
u
-
8/18/2019 continuous optimization techinique
14/32
1+
"isection Method
0++.02
11.00
11.0,0
=+
=
==
m
u
x
x x
( )( )
( ) +
4
4
106++.60++.0
10662.211.0
103.30
−
−
−
=
−==
f
f
f
11.0
0++.0
=
=
u x
x
teration 91
-
8/18/2019 continuous optimization techinique
15/32
1.
"isection Methodteration 92
33.33
02+.02
11.00++.0
11.0,0++.0
=∈
=+
=
==
a
m
u
x
x x
( )
( )
( )02+.0,0++.0
10(62216.102+.0
10(662.211.0
10(6++.60++.0
4
4
+
==−=
−=
=
−
−
−
u x x
f
f
f
-
8/18/2019 continuous optimization techinique
16/32
1/
"isection Methodteration 9'
06+.02
02+.00++.0
02+.0,0++.0
=+
=
==
m
u
x
x x
( )
( )
( ) +
4
+
10+632.+06+.0
1062216.102+.0
106++.60++.0
20
−
−
−
−=
−=
=
=∈
f
f
f
a
-
8/18/2019 continuous optimization techinique
17/32
10
"isection MethodCon)er%ence
$abe ;1 Root of f6"79C as function of number of iterations for bisection method.
-
8/18/2019 continuous optimization techinique
18/32
18
"isection Method Advantages:
Aways conver*ent
$he root brac,et *ets haved with each iteration 5 *uaranteed. Disadvantages:
(ow conver*ence
)f one of the initia *uesses is cose to the root8 the conver*ence is sower
)f a function f6"7 is such that it ust touches the "5a"is it wi beunabe to find the ower and upper *uesses.
( ) 2 x x f =
5;C 5G C G ;C
C
?C
4C
@C
HC
;CC
y ( x )
x
y = x?
-
8/18/2019 continuous optimization techinique
19/32
1
"isection Method
4unction chan%es si%n 6ut root does not exist
( ) x
x f 1=
5C. 5C.@ 5C.I C.C C.I C.@ C.
5;CC
5HC
5@C
54C
5?C
C
?C
4C
@C
HC
;CC
y ( x )
x
y = 1/x
-
8/18/2019 continuous optimization techinique
20/32
23
"isection method $or optimization-ind the minimum of for . Aso write matab code for
verification.
3 23 x x− [ ]0,3 x ∈
%an we appy the bisection method to
find the minimum to this probemJ
-
8/18/2019 continuous optimization techinique
21/32
21
"isection method $or optimization-ind the minimum of for . . Aso write matab code for
verification.
3 23 x x− [ ]1.+,3 x ∈
%an we appy the bisection method tofind the root to this probemJ
No. 0hyJ. >ow to use root findin* a*orithms for optimization
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15
-10
-5
0
5
10
15
20
25x3-3*x
2
Orig
1st Derv.
2nd Derv.
-
8/18/2019 continuous optimization techinique
22/32
i:erence 6eteen roots and optima
-
8/18/2019 continuous optimization techinique
23/32
2'
"isection method $or optimization-ind the minimum of for . Aso write matab code for
verification.
3 23 x x− [ ]1.+,3 x ∈
%an we appy the bisection method tofind the root to this probemJ
No. 0hyJ. >ow to use root findin* a*orithms for optimization
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15
-10
-5
0
5
10
15
20
25x3-3*x2
Orig
1st Derv.
2nd Derv.
$o *et the minimum we need to
appy bisection method on the
derivative of x
-x!
otherwisewe wi *et the wron* answer.
Root findin* a*orithms find the
point where the function is zero.
0e want to see where
derivative is zero.
-
8/18/2019 continuous optimization techinique
24/32
2+
&ecant method to ;nd rootEquation o$ a !ine $rom 2 Points
,
2 1
2 1
5 ,
1 2 1 1 2 1
2 11 1
2 1
F"om t#e definition of $lo%ewe can dete"mine t#e $lo%e in t#i$ ca$e a$
y ym
x x
&lu''in' in t#i$ fo" m in $lo%e %oint fo"mula
y y y y x x x x
y y y y x x
x x
#i$ i$ called a$
−=
−−
− −=− −
− ⇒ − = − ÷− .two %oint fo"mula−
-
8/18/2019 continuous optimization techinique
25/32
2.
&ecant method to ;nd rooteri)ation o$ the method
e then use this new a%ue of x as x! and re"eat the "rocess
using x1 and x! instead of x0 and x1 . e continue this
"rocess, so%ing for x , x , etc., unti% we reach a
sufficient%7 high %ee% of "recision (a sufficient%7 sma%%
difference between xn and xn-1 ).
-
8/18/2019 continuous optimization techinique
26/32
2/
&ecant method to ;nd root
$he first two iterations of the secant method. $he red curve
shows the function f and the bue ines are the secants. -or this
particuar case8 the secant method wi not conver*e.
-
8/18/2019 continuous optimization techinique
27/32
20
&ecant method to ;nd root
nitialize x1(
x2( ε
-
8/18/2019 continuous optimization techinique
28/32
28
&ecant method $or Optimization
nitialize x1(
x2( ε
-
8/18/2019 continuous optimization techinique
29/32
2
&ecant method $or Optimization-ind the minimum of for . Aso write matab code for
verification.
3 23 x x− [ ]1.+,3 x ∈
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15
-10
-5
0
5
10
15
20
25x3-3*x
2
Orig1st Derv.
2nd Derv.4 < x' - 'Dx2 F $unctiond4 < 'Dx2 - /Dx F eri)ati)e
2 is 1.00000 and d8 is !1.00000
2 is 1.2+1 and d8 is !0.41326+
2 is 2.00264 and d8 is 0.042
2 is 1.6+ and d8 is !0.00122 is 1. and d8 is !0.00000
2 is 2.000000 and d8 is 0.000000
-
8/18/2019 continuous optimization techinique
30/32
'3
&ecant method $or Optimization-ind the minimum of for . Aso write matab code for
verification.
3 23 x x− [ ]1.+,3 x ∈
4 < x' - 'Dx2 F $unctiond4 < 'Dx2 - /Dx F eri)ati)e
2 is 1.00000 and d8 is !1.00000
2 is 1.2+1 and d8 is !0.41326+
2 is 2.00264 and d8 is 0.042
2 is 1.6+ and d8 is !0.00122 is 1. and d8 is !0.00000
2 is 2.000000 and d8 is 0.000000
0 0.5 1 1.5 2 2.5 3-20
-15
-10
-5
0
5
10
15
20
Orig
1st Derv.
Iter = 1
Iter = 2
Iter = 3
Iter = 4
Iter = 5
-
8/18/2019 continuous optimization techinique
31/32
'1
Neton-5aphson or Neton method
-e"eat the "rocess unti% 9 f ( xi) 9 is
sufficient%7 sma%%
( ) ( )
1
0ii
i i
f x f x x x +
−′ = −
0hich can be rearran*e as
( )
( )1i
i ii
f x
x x f x+ = −
′
Root
-
8/18/2019 continuous optimization techinique
32/32
'2
Neton-5aphson or Neton method
-e"eat the "rocess unti% 9 f ( xi) 9 is
sufficient%7 sma%%
( ) ( )
1
0ii
i i
f x f x x x +
−′ = −
0hich can be rearran*e as
( )
( )1i
i ii
f x
x x f x+ = −
′
Root
Optimization( )
( )1
i
i i
i
f x x x
f x+
′= −
′′