mathworks technology session at ge optimization with ... · mathworks technology session at ge...
Post on 15-Apr-2018
240 Views
Preview:
TRANSCRIPT
1 © 2012 The MathWorks, Inc.
MathWorks Technology Session at GE
Optimization with MATLAB Products
November 8, 2012
2
MathWorks at a Glance
Earth’s topography
on a Miller cylindrical
projection, created
with MATLAB and
Mapping Toolbox
● Headquarters:
Natick, Massachusetts US
● Other U.S. Locations:
California, Michigan,
Texas, Washington, DC
● Europe:
France, Germany, Italy,
Netherlands, Spain, Sweden,
Switzerland, United Kingdom
● Asia-Pacific:
Australia, China, India,
Japan, Korea
● Worldwide training
and consulting
● Distributors serving more
than 20 countries
3
Optimization: What is it, Why use it?
Optimization in MATLAB
Gradient-based Optimization
Global Optimization
Additional Resources
Agenda
-2
0
2
-3-2
-10
12
3
-6
-4
-2
0
2
4
6
8
x
Peaks
y
Local minima
Global
minima
4
Optimization Problems
Minimize Risk
Maximize Profits
Maximize Fuel Efficiency
5
Design Process
Initial
Design
Variables
System
Modify
Design
Variables
Optimal
Design Objectives
met?
No
Yes
6
Why use Optimization?
Manually (trial-and-error or iteratively)
Initial
Guess
7
Why use Optimization?
Automatically (using optimization techniques)
Initial
Guess
8
Why use Optimization?
Finding better (optimal) designs
Faster Design Evaluations
Useful for trade-off analysis
Non-intuitive designs may be found
Antenna Design Using Genetic Algorithm http://ic.arc.nasa.gov/projects/esg/research/antenna.htm
9
Demos
Constrained optimization
Least squares / curve fitting
Quadratic programming
Finding global minimum
Mixed integer optimization
MATLAB Tips, Tricks, and Best
Practices along the way
1 75 150 2250
0.05
0.1
0.15
0.2
0.25
Assets
Fra
ction o
f in
vestm
ent
225-asset problem
Standard
With group constraints
Group 1 Group 2 Group 3
11
Optimization Approaches
Gradient Based Global Methods
Faster/fewer function
evaluations Larger problems (higher
dimensions) Finds local
minima/maxima
Finds global
minima/maxima (most of
the time)
Better on
non-smooth
stochastic
discontinuous
undefined gradients
Toolbox Optimization
Toolbox
Global
Optimization
Toolbox
12
Optimization: What is it, Why use it?
Optimization in MATLAB
Gradient-based Optimization
Global Optimization
Additional Resources
Agenda
-2
0
2
-3-2
-10
12
3
-6
-4
-2
0
2
4
6
8
x
Peaks
y
Local minima
Global
minima
13
Anatomy of an Optimization Problem
General Form for MATLAB Solvers
Objective Function
uxl
eqbxeqA
bAx
)(min xfx
Typically a linear or nonlinear function
Linear constraints
• inequalities
• equalities
• bounds
Nonlinear constraints
• inequalities
• equalities
Decision variables
0)(
0)(
xeq
c
xc
Subject to Constraints
14
Constrained Optimization Demo
min𝑥
log 1 + 𝑥1 −4
3
2
+ 3 𝑥1 + 𝑥2 − 𝑥13 2
15
Choosing the best solver
Doc: Optimization Toolbox > User’s Guide > Setting up an
Optimization > Choosing a Solver
Choose the algorithm based on your objective and constraint types.
16
Curve Fitting Demo
Given some data:
Fit a curve of the form:
teccty 21
)(
t = [0 .3 .8 1.1 1.6 2.3];
y = [.82 .72 .63 .60 .55 .50];
0 0.5 1 1.5 2 2.50.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
t
y
17
How to solve?
Ecc
cey
eccty
t
t
2
1
21
1
)(
As a linear system of equations:
2
2min yEc
Ecy
c
Can’t solve this exactly
(6 eqns, 2 unknowns)
An optimization problem!
2
1
3.2
6.1
1.1
8.0
3.0
0
1
1
1
1
1
1
50.0
55.0
60.0
63.0
72.0
82.0
c
c
e
e
e
e
e
e
18
Quadratic Programming
General Form Accepted by MATLAB Quadratic Solver
Objective Function
uxl
bxA
bAx
eqeq
xfHxxx
''min2
1
Quadratic function
Linear constraints
• inequalities
• equalities
• bounds
Decision variables
NOTE: In R2011a, quadprog was dramatically sped up for convex problems, particularly
significant for large ones. Uses sparse linear algebra, key for large, sparse problems.
Hessian (real, symmetric)
Subject to Constraints
19
Quadratic Programming Demo
1 75 150 2250
0.05
0.1
0.15
0.2
0.25
Assets
Fra
ction o
f in
vestm
ent
225-asset problem
Standard
With group constraints
Group 1 Group 2 Group 3
Given:
Historical data and a correlation matrix for 225 assets/
Find:
The portfolio weights that minimize risk.
C
w
Cwwrisk T
2
1
Portfolio weights
Covariance matrix
20
Gradient based Optimization Summary
Nonlinear minimization
– fminunc: unconstrained
– fmincon: linear and nonlinear constraints
• Supply Gradient and Hessian functions to
speed up
Constrained curve fitting with least squares solvers
– lsqlin: linear least-squares subject to linear constraints
– lsqnonlin: nonlinear least-squares with bound constraints
For linear or quadratic problems with linear constraints
– linprog: linear problems
– quadprog: quadratic problems
21
Optimization: What is it, Why use it?
Optimization in MATLAB
Gradient-based Optimization
Global Optimization
Additional Resources
Agenda
-2
0
2
-3-2
-10
12
3
-6
-4
-2
0
2
4
6
8
x
Peaks
y
Local minima
Global
minima
22
Why use Global Optimization?
Gradient based solvers are ineffective when the
gradient is undefined, and only converge to local
minima.
What if the problem is non-smooth, stochastic, has
multiple local minima?
Need a solver that can explore the solution space and
not fall into local minima.
*** No solver is guaranteed to find the global minimum
23
Global Optimization Solvers Covered Today
Multi Start
Global Search
Simulated Annealing
Pattern Search
Genetic Algorithm
24
Example Global Optimization Problems
Why does fmincon have a hard time finding the
function minimum?
0 5 10
-10
-5
0
5
10
x
Starting at 10
0 5 10
-10
-5
0
5
10
x
Starting at 8
0 5 10
-10
-5
0
5
10
x
Starting at 6
x s
in(x
) +
x c
os(2
x)
0 5 10
-10
-5
0
5
10
x
Starting at 3
0 5 10
-10
-5
0
5
10
x
Starting at 1
0 5 10
-10
-5
0
5
10
x
Starting at 0x s
in(x
) +
x c
os(2
x)
25
Example Global Optimization Problems
Why didn’t fminunc find the maximum efficiency?
Revolutions Per Minute, RPM
Manifold
Pre
ssure
Ratio
Peak VE Value = 0.96144
Start
End
1000 2000 3000 4000 5000 60000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
26
Example Global Optimization Problems
Why didn’t nonlinear regression find a good fit?
0 20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
t
c
c=b1e-b
4t+b
2e-b
5t+b
3e-b
6t
27
Global Optimization
Goal:
Want to find the lowest/largest value of
the nonlinear function that has many local
minima/maxima
Problem:
Traditional solvers often return one of the
local minima (not the global)
Solution:
A solver that locates globally optimal
solutions Rastrigin’s Function
28
MultiStart Demo – Nonlinear Regression
lsqcurvefit solution MultiStart solution
29
MULTISTART
30
What is MultiStart?
Run a local solver from
each set of start points
Option to filter starting
points based on feasibility
Supports parallel
computing
31
MultiStart Demo – Peaks Function
32
GLOBAL SEARCH
33
What is GlobalSearch?
Multistart heuristic algorithm
Calls fmincon from
multiple start points to try
and find a global minimum
Filters/removes non-
promising start points
34
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview Schematic Problem
Peaks function
Three minima
Green, z = -0.065
Red, z= -3.05
Blue, z = -6.55
x
y
35
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 0 Run from specified x0
x
y
36
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
0 0
4
0
-2
x
y
37
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
0 0
4
0
-2
x
y
38
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
x
y
39
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
40
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
41
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
42
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
43
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
6
Current penalty
threshold value : 4
x
y
44
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
45
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
Current penalty
threshold value : 4
-3
x
y
46
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2 Expand basin of attraction if minimum already found
Current penalty threshold value : 2
-0.1
x
y Basins can overlap
47
GlobalSearch Demo – Peaks Function
48
SIMULATED ANNEALING
49
What is Simulated Annealing?
A probabilistic metaheuristic
approach based upon the
physical process of annealing
in metallurgy.
Controlled cooling of a metal
allows atoms to realign from
a random higher energy state
to an ordered crystalline
(globally) lower energy state
50
Simulated Annealing Overview – Iteration 1 Run from specified x0
x
y
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
0.9
51
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Possible New Points:
Standard Normal N(0,1) * Temperature
Temperature = 1
52
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
11.01
1/)9.03(
Taccepte
P
53
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
54
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
55
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 1
0.3
56
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
57
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N-1
3
x
y 0.9
Temperature = 0.1
0.3
-1.2
-3
58
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N Reannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
59
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N Reannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
27.01
1/))3(2(
Taccepte
P
60
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N Reannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
61
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
62
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
63
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration …
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
-6.5
64
Simulated Annealing – Peaks Function
65
PATTERN SEARCH
(DIRECT SEARCH)
66
What is Pattern Search?
An approach that uses a
pattern of search directions
around the existing points
Expands/contracts around
the current point when a
solution is not found
Does not rely on gradients:
works on smooth and
nonsmooth problems
67
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1 Run from specified x0
x
y
3
68
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1 Apply pattern vector, poll new points for improvement
x
y
3
Mesh size = 1
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
0_*_ xvectorpatternsizemeshPnew
0]0,1[*1 x1.6
0.4
4.6
2.8
First poll successful
Complete Poll (not default)
69
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 2
x
y
3
Mesh size = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
Complete Poll
70
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 3
x
y
3
Mesh size = 4
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
71
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 4
x
y
3
Mesh size = 4*0.5 = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3 -2.8
72
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration N Continue expansion/contraction until convergence…
x
y
3 1.6
0.4
4.6
2.8
-4
0.3 -2.8
-6.5
73
Pattern Search – Peaks Function
74
GENETIC ALGORITHM
75
What is a Genetic Algorithm?
Uses concepts from
evolutionary biology
Start with an initial generation
of candidate solutions that are
tested against the objective
function
Subsequent generations
evolve from the 1st through
selection, crossover and
mutation
76
How Evolution Works – Binary Case
Selection
– Retain the best performing bit strings from one generation to the next.
Favor these for reproduction
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
Crossover
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
– child = [ 1 0 0 0 0 1 1 0 1 0 ]
Mutation
– parent = [ 1 0 1 0 0 1 1 0 0 0 ]
– child = [ 0 1 0 1 0 1 0 0 0 1 ]
77
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1 Evaluate initial population
x
y
78
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1 Select a few good solutions for reproduction
x
y
79
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2 Generate new population and evaluate
x
y
80
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2
x
y
81
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
82
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
83
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration N Continue process until stopping criteria are met
x
y
Solution found
84
Genetic Algorithm – Peaks Function
85
Global Optimization Solvers
GlobalSearch, MultiStart
– Well suited for smooth objective and constraints
– Return the location of local and global minima
ga, simulannealbnd
– Many function evaluations to sample the search space
– Work on both smooth and nonsmooth problems
patternsearch
– Fewer function evaluations than ga, simulannealbnd
– Does not rely on gradient calculation like GlobalSearch and
MultiStart
– Works on both smooth and nonsmooth problems
86
Pattern Search Climbs Mount Washington
87
Genetic Algorithm – Integer Constraints
Mixed Integer Optimization
s.t. some x are integers
Examples
Only certain sizes of components
available
Can only purchase whole shares of stock
)(min xfx
88
Simulated Annealing – Eight Queens
89
Optimization and Parallel Computing
Many optimization algorithms involve some sort of parallelism – Gradient Estimation (fmincon)
– Evaluating each member of a Population (ga)
– Running a local solver from several start points (MultiStart)
Can use Parallel Computing Toolbox to distribute parallel tasks to
different workers
Demo: Parallel Computing with MultiStart
90
Speeding up with Parallel Computing
Global Optimization Algorithms that support Parallel Computing: – ga: Members of population evaluated in parallel at each iteration
– patternsearch: Poll points evaluated in parallel at each iteration
– MultiStart: Start points evaluated in parallel
Optimization Algorithms that support Parallel Computing: – fmincon: parallel evaluation of objective function for finite differences
– fminimax, fgoalattain: same as fmincon
Parallel Computing can also be used in the Objective Function – parfor
91
Parallel Computing Considerations
Need to consider structure of optimization problem when choosing
number of workers
Example – ga with default population size of 20
– Large speedup (~2x) from 1-2, 2-4 workers
– Smaller speedup (~1.25x) from 8-16 workers
1 worker 2 workers 4 workers 8 workers
Objective
Function
Evaluations
(1 generation)
92
Parallel computing required products
To use parallel computing To also use a cluster
Parallel Computing Toolbox MATLAB Distributed Computing Server
93
Scale Up to Clusters, Grids and Clouds
Desktop Computer
Parallel Computing Toolbox
Computer Cluster
MATLAB Distributed Computing Server
Scheduler
94
Summary of Global Optimization
Solvers designed to explore the solution space and find
global solutions
Problems can be stochastic, or nonsmooth
Solvers in Global Optimization Toolbox
– MultiStart
– GlobalSearch
– simulannealbnd
– patternsearch
– ga
Parallel Computing with ga, patternsearch, MultiStart
95
Optimization: What is it, Why use it?
Optimization in MATLAB
Gradient-based Optimization
Global Optimization
Additional Resources
Agenda
-2
0
2
-3-2
-10
12
3
-6
-4
-2
0
2
4
6
8
x
Peaks
y
Local minima
Global
minima
96
Learn More about Optimization with MATLAB
MATLAB Digest: Using Symbolic
Gradients for Optimization
Recorded webinar: Optimization
in MATLAB: An Introduction to
Quadratic Programming
Optimization Toolbox Web demo:
Finding an Optimal Path using MATLAB
and Optimization Toolbox
MATLAB Digest: Improving
Optimization Performance with
Parallel Computing
98 © 2012 The MathWorks, Inc.
Questions?
top related