op tim ization uw 06
TRANSCRIPT
-
7/30/2019 Op Tim Ization Uw 06
1/29
Introduction to Optimization
Anjela Govan
North Carolina State University
SAMSI NDHS Undergraduate workshop 2006
-
7/30/2019 Op Tim Ization Uw 06
2/29
What is Optimization?
Optimization is the mathematical disciplinewhich is concerned with finding the maxima
and minima of functions, possibly subject to
constraints.
-
7/30/2019 Op Tim Ization Uw 06
3/29
Where would we use optimization?
Architecture
Nutrition
Electrical circuits
Economics
Transportation
etc.
-
7/30/2019 Op Tim Ization Uw 06
4/29
What do we optimize?
A real function of n variables
with or without constrains
),,,(21 n
xxxf
-
7/30/2019 Op Tim Ization Uw 06
5/29
Unconstrained optimization
22 2),(min yxyxf
-
7/30/2019 Op Tim Ization Uw 06
6/29
Optimization with constraints
2
2),(min
1,52
2),(min
0
2),(min
22
22
22
or
or
yx
yxyxf
yx
yxyxf
x
yxyxf
-
7/30/2019 Op Tim Ization Uw 06
7/29
Lets Optimize
Suppose we want to find the minimum of the
function
-
7/30/2019 Op Tim Ization Uw 06
8/29
Review max-min forR2
What is special about a local max or a local
min of a function (x)?
at local max or local min (x)=0
(x) > 0 if local min
(x) < 0 if local max
-
7/30/2019 Op Tim Ization Uw 06
9/29
Review max-min forR3
-
7/30/2019 Op Tim Ization Uw 06
10/29
Review max-min forR3
Second Derivative Test
Local min, local max, saddle point
Gradient of vector (d dx d /dy d /dz)
direction of fastest increase of
Global min/max vs. local min/max
-
7/30/2019 Op Tim Ization Uw 06
11/29
Gradient Descent Method Examples
Minimize function
11,11
)(5.0),(22
yx
yxyxf
Minimize function
4,4)cos()cos(),(
yxyxyxf
-
7/30/2019 Op Tim Ization Uw 06
12/29
Gradient Descent Method Examples
Use function gd(alpha,x0) Does gd.m converge to a local min? Is there a
difference if > 0 vs. < 0?
How many iterations does it take to converge to alocal min? How do starting points x0 affectnumber of iterations?
Use function gd2(x0)
Does gd2.m converge to a local min? How do starting points x0 affect number of
iterations and the location of a local minimum?
-
7/30/2019 Op Tim Ization Uw 06
13/29
How good are the optimization methods?
Starting point
Convergence to global min/max.
Classes of nice optimization problems
Example: f(x,y) = 0.5(x2+y2), > 0
Every local min is global min.
-
7/30/2019 Op Tim Ization Uw 06
14/29
Other optimization methods
Non smooth, non differentiable surfaces
can not compute the gradient of
can not use Gradient Method
Nelder-Mead Method
Others
-
7/30/2019 Op Tim Ization Uw 06
15/29
Convex Hull
A set C is convex ifevery point on the line
segment connecting xand y is in C.
The convex hull for aset of points X is the
minimal convex setcontaining X.
-
7/30/2019 Op Tim Ization Uw 06
16/29
Simplex
A simplex orn-simplex isthe convex hull of a set of
(n+1) . A simplex is an n-dimensional analogue of a
triangle.
Example:
a 1-simplex is a line segment
a 2-simplex is a triangle a 3-simplex is a tetrahedron
a 4-simplex is a pentatope
-
7/30/2019 Op Tim Ization Uw 06
17/29
Nelder-Mead Method
n = number of variables, n+1 points
form simplex using these points; convex hull
move in direction away from the worst of
these points: reflect, expand, contract, shrink
Example:
2 variables 3 points simplex is triangle
3 variables 4 points simplex is tetrahedron
-
7/30/2019 Op Tim Ization Uw 06
18/29
Nelder-Mead Methodreflect, expand
-
7/30/2019 Op Tim Ization Uw 06
19/29
Nelder-Mead Method-reflect, contract
-
7/30/2019 Op Tim Ization Uw 06
20/29
A tour of Matlab: Snapshots from the minimization
After 0 steps
-
7/30/2019 Op Tim Ization Uw 06
21/29
A tour of Matlab: Snapshots from the minimization
After 1 steps
-
7/30/2019 Op Tim Ization Uw 06
22/29
A tour of Matlab: Snapshots from the minimization
After 2 steps
-
7/30/2019 Op Tim Ization Uw 06
23/29
A tour of Matlab: Snapshots from the minimization
After 3 steps
-
7/30/2019 Op Tim Ization Uw 06
24/29
A tour of Matlab: Snapshots from the minimization
After 7 steps
-
7/30/2019 Op Tim Ization Uw 06
25/29
A tour of Matlab: Snapshots from the minimization
After 12 steps
-
7/30/2019 Op Tim Ization Uw 06
26/29
A tour of Matlab: Snapshots from the minimization
After 30 steps (converged)
-
7/30/2019 Op Tim Ization Uw 06
27/29
fminsearch function
parameters: q =[C,K]
cost function:
Minimize cost function
[q,cost]=
fninsearch(@cost_beam, q0,[],time,y_tilde)
2N
1ii
)y_tilde]),[,(y(tcosti
KC
-
7/30/2019 Op Tim Ization Uw 06
28/29
Our optimization problem
In our problem
Our function:
cost function lives in R3
2 parameters C and K, n=2
Simplex is a triangle
2N
1ii
)y_tilde]),[,(y(tcosti
KC
-
7/30/2019 Op Tim Ization Uw 06
29/29
Done!