optimization with scilab
TRANSCRIPT
![Page 1: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/1.jpg)
The free and open source software for numerical computation
Optimization with Scilab
June 29th 2011
Michaël BAUDIN & Vincent COUVERTScilab Consortium
![Page 2: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/2.jpg)
The free and open source software for numerical computationThe free software for numerical computation
Part 1 - What's new in Scilab 5 ?
Focus on the Nelder-Mead component
Part 2 - Optimization in Scilab: Matlab® compatibility
Part 3 - OMD2 project: Scilab Platform Development
Part 4 - Conclusion
What is missing in Scilab ?
Outline
![Page 3: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/3.jpg)
The free and open source software for numerical computationThe free software for numerical computation
1. Introduction
1.1 What's in Scilab ?
1.2 What's new in Scilab v5 ?
1.3 What's new on Atoms ?
Part 1 – What's new in Scilab 5 ?
2. The Nelder-Mead Component
2.1 Introduction
2.2 The algorithm
2.3 Test cases
2.4 Conclusions
![Page 4: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/4.jpg)
The free and open source software for numerical computation
Objective Bounds Equality Inequalities Size GradientNeeded
Solver
Linear yes linear linear medium - linpro
Quadradic yes linear linear medium - quapro
Quadratic yes linear linear large - qpsolve
Quadratic yes linear linear medium - qld
Non-Linear yes large yes optim
Non-Linear small no fminsearch
Non-Linear yes small no neldermead
Non-Linear yes small no optim_ga
Non-Linear small no optim_sa
N.Li.Lea.Sq. large optional lsqrsolve
N.Li.Lea.Sq. large optional leastsq
Min-Max yes medium yes optim/nd
Multi-Obj yes small no optim_moga
Semi-Def. lin. (spectral) large no semidef
L.M.I. lin. (spectral) lin. (spectral) large no lmisolve
1.1 What's in Scilab ?
![Page 5: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/5.jpg)
The free and open source software for numerical computation
● Genetic Algorithms: nonlinear objective, bounds, global optimization
● Simulated Annealing: nonlinear objective, global optimization
● The Nelder-Mead component: nonlinear objective, unconstrained,
derivative-free, local optimization
● fminsearch: Matlab® compatible
1.2 What's new in Scilab 5 ?
![Page 6: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/6.jpg)
The free and open source software for numerical computation
● Optimization Solvers:
– Quapro: linear or quadratic objective, linear constraints (full matrices),
– SciIpopt: an interface to Ipopt. Nonlinear objective, nonlinear constraints (beta version),
– Fmincon: nonlinear objective, nonlinear constraints (alpha version) – Matlab® compatible,
– Other modules: Cobyla, Particle Swarm Optimization, Optkelley, …
● Test Problems:
– Uncprb: 35 unconstrained optimization problems,– AMPL: load AMPL problems into Scilab,– And also: CUTEr.
1.3 What's new on ATOMS ?
![Page 7: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/7.jpg)
The free and open source software for numerical computationThe free software for numerical computation
Outline
1. Introduction
1.1 What's in Scilab ?
1.2 What's new in Scilab v5 ?
1.3 What's new on Atoms ?
2. The Nelder-Mead Component
2.1 Introduction
2.2 The algorithm
2.3 Test cases
2.4 Conclusions
![Page 8: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/8.jpg)
The free and open source software for numerical computation
John Ashworth Nelder (8 October 1924 – 7 August 2010)Source: http://www.guardian.co.uk/technology/2010/sep/23/john-nelder-obituary
2. The Nelder-Mead Component2.1 Introduction
![Page 9: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/9.jpg)
The free and open source software for numerical computation
● We are interested in solving the unconstrained continuous optimization problem:
Minimize f(x)
with unbounded, real, multidimensional, variable x.
● A direct search algorithm:
– Uses only function values (no gradient needed),– Does not approximate the gradient.
● « A simplex method for function minimization », John Nelder, Roger Mead, Computer Journal, vol. 7, no 4, 1965, p. 308-313
2. The Nelder-Mead Algorithm2.1 Introduction
![Page 10: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/10.jpg)
The free and open source software for numerical computation
2. The Nelder-Mead Algorithm2.1 Introduction
Virginia Torczon (1989) writes:
"Margaret Wright has stated that over fifty percent of the calls received by the support group for the NAG software library concerned the version of the Nelder-Mead simplex algorithm to be found in that library."
![Page 11: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/11.jpg)
The free and open source software for numerical computation
A simplex: a set of n+1 vertices, in n dimensions.
In 2 dimensions.
In 3 dimensions.
2. The Nelder-Mead Algorithm2.2 The algorithm
![Page 12: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/12.jpg)
The free and open source software for numerical computation
● Steps in the Nelder-Mead algorithm
● Inputs:
– the n+1 vertices v(1), v(2), ..., v(n+1) of a nondegenerate simplex in n dimensions,
– the associated function values f(1),...,f(n+1), – the coefficients ρ (reflection), χ (expansion), γ (contraction),
and σ (shrinkage).● Standard Nelder-Mead: ρ=1, χ=2, γ=1/2, and σ=1/2.
2. The Nelder-Mead Algorithm2.2 The algorithm
![Page 13: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/13.jpg)
The free and open source software for numerical computation
2. The Nelder-Mead Algorithm2.2 The algorithm
![Page 14: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/14.jpg)
The free and open source software for numerical computation
f x1, x2=x12x2
2− x1 x2
function [ y , index ] = quadratic ( x , index )
y = x(1)^2 + x(2)^2 - x(1) * x(2);
endfunction
nm = neldermead_new ();
nm = neldermead_configure(nm,"-numberofvariables",2);
nm = neldermead_configure(nm,"-function",quadratic);
nm = neldermead_configure(nm,"-x0",[2 2]');
nm = neldermead_search(nm);
xopt = neldermead_get(nm,"-xopt");
nm = neldermead_destroy(nm);
2. The Nelder-Mead Algorithm2.3 Test cases
![Page 15: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/15.jpg)
The free and open source software for numerical computation
2. The Nelder-Mead Algorithm2.3 Test cases
![Page 16: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/16.jpg)
The free and open source software for numerical computation
● Mc Kinnon, « Convergence of the neldermead simplex method to a nonstationary point ». SIAM J. on Optimization, 1998
● Failure by repeated inside contraction
2. The Nelder-Mead Algorithm2.3 Test cases
![Page 17: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/17.jpg)
The free and open source software for numerical computation
● C. T. Kelley. « Detection and remediation of stagnation in the neldermead algorithm using a sufficient decrease condition » SIAM J. on Optimization, 1999
● Restart the algorithm...
2. The Nelder-Mead Algorithm2.3 Test cases
![Page 18: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/18.jpg)
The free and open source software for numerical computation
● Some general facts:
– Memory requirement is O(n²)– Shrink steps are rare– Generally 1 or 2 function evaluations by iteration– Convergence is slow. Typical number of iterations is 100n,
where n is the number of dimensions– Hundreds of iterations are not rare– Convergence can be even slower when n > 10 (Han &
Neumann, 2006)– Restart the algorithm when in doubt for convergence (Kelley,
1999)– Convergence is guaranteed in 1 dimension (Lagarias et al.,
1999)
2. The Nelder-Mead Algorithm2.4 Conclusions
![Page 19: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/19.jpg)
The free and open source software for numerical computation
● We should not use this algorithm just because the gradient is not required:
– For example, if f is smooth, Quasi-Newton methods (optim) with numerical derivatives converge much faster.
● We may use this algorithm when:
– No other property of the problem can be used (e.g. non linear least squares can be solved by lsqrsolve),
– The objective function is nonsmooth or "noisy" (Kelley, 1999),– We do not need too much accuracy (Torzcon, 1989),– The number of parameters is moderate (Han & Neumann,
2006).
2. The Nelder-Mead Algorithm2.4 Conclusions
![Page 20: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/20.jpg)
The free and open source software for numerical computationThe free software for numerical computation
1. Introduction
2. Scilab Coverage
3. Overview
Part 3 – Optimization in Scilab: Matlab® compatibility
![Page 21: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/21.jpg)
The free and open source software for numerical computation
● Matlab® has many functions for optimization:
– Minimization,– Equation solving,– Datafitting and nonlinear least squares,– Global optimization.
● Scilab has often similar functions: let's see which ones.
Matlab is a registered trademark of The Mathworks, Inc.
1. Introduction
![Page 22: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/22.jpg)
The free and open source software for numerical computation
● For each Matlab® function, we search:
– Scilab function, if available,– Differences of features, differences of algorithms.
● (*) : Function will be reviewed at the end of the talk,
● For most functions, the match is not 100% identical,
– But some other functions can do it : which ones ?● We consider only Scilab Industrial Grade solvers:
– Scilab internal modules,– ATOMS modules,– Portables on all OS,– Well documented,– Tested.
1. Introduction
![Page 23: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/23.jpg)
The free and open source software for numerical computation
1. IntroductionMain differences
● Design:
– Matlab®: problem oriented (may be with several solvers),– Scilab: solver oriented (may be several solvers).
● Function arguments:
– Matlab® nearly always provides common options,– Scilab is less homogeneous.
● Management of the callbacks/extra-arguments:
– Matlab®: M-file or @
– Scilab: list
![Page 24: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/24.jpg)
The free and open source software for numerical computation
● Minimization:
– fminbnd Not 100% identical,
But optim can do it.
– fmincon ATOMS/fmincon (alpha version)– fminimax Not 100% identical,
but optim/''nd'' is designed for it.
– fminsearch fminsearch 90% identical in Scilab 5.3.2.
fminsearch 99% identical in 5.4.0
2. Scilab Coverage
![Page 25: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/25.jpg)
The free and open source software for numerical computation
– fminunc Not 100% identical,
but optim/''qn'' or optim/''gc'' are designed for it.
No sparsity pattern of Hessian in Scilab.
No PCG in optim: L-BFGS instead.
– linprog 100% for full matrices: karmarkar
ATOMS/quapro: linpro
No known solver for sparse matrices (*).
– quadprog 100% for full matrices: qpsolve, qp_solve
ATOMS/quapro: quapro
No known solver for sparse matrices.
2. Scilab Coverage
![Page 26: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/26.jpg)
The free and open source software for numerical computation
● Equation Solving:
– fsolve fsolve 100% for full matrices.
No known solver with sparse Jacobian (*).
– fzero No identical function.
But fsolve can do it.
● Least Squares (Curve Fitting):
– lsqcurvefit datafit
– lsqnonlin lsqrsolve (leastsq)
2. Scilab Coverage
![Page 27: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/27.jpg)
The free and open source software for numerical computation
● Global Optimization Toolbox:
● Genetic Algorithm Not 100% identical,
But optim_ga is built-in Scilab.
No linear equality and inequality in Scilab,
but bounds are managed.
● Simulated Annealing Not 100% identical,
But optim_sa is built-in Scilab
No bounds in Scilab SA, but user can
customize the neighbour function.
2. Scilab Coverage
![Page 28: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/28.jpg)
The free and open source software for numerical computation
Matlab Problem Scilab
bintprog Binary Integer Programming -
fgoalattain Multiobjective goal attainment -
fminbd Single-variable, on interval optim
fmincon Constrained, nonlinear, multivariable ATOMS/fmincon
fminimax Minimax, constrained optim/''nd''
fminsearch Unconstrained, multivariable, derivative-free fminsearch (100%)
fminunc Unconstrained, multivariable optim/''qn'',''gc''
fseminf Semi-infinitely constrained, multivariable, nonlinear
-
ktrlink Constrained or unconstrained, nonlinear, multivariable using Knitro
-
linprog Linear programming karmarkar, ATOMS/quapro
quadprog Quadratic programming qpsolve, ATOMS/quapro
3. Overview
![Page 29: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/29.jpg)
The free and open source software for numerical computation
Matlab Problem Scilab
fsolve Solve systems of nonlinear equations fsolve
fzero Root of continuous function of one variable -
lsqcurvefi t Nonlinear least squares curve fitting datafit
lsqlin Constrained linear least squares -
lsqnonlin Nonlinear least-squares (nonlinear data-fitting) lsqrsolve, leastsq
lsqnonneg Nonnegative least squares -
optimtool GUI to select solvers, options and run problems -
Global Search Solve GlobalSearch problems -
Multi Start Solve MultiStart problems -
Genetic Algorithm Genetic Algorithms optim_ga
Direct Search Pattern Search -
Simulated Annealing Simulated Annealing optim_sa
3. Overview
![Page 30: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/30.jpg)
The free and open source software for numerical computationThe free software for numerical computation
Part 4 -OMD2 project: Scilab Platform Development
1. Overview
2. Modules
2.1 Data Management
2.2 Modeling
2.3 Optimization
![Page 31: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/31.jpg)
The free and open source software for numerical computation
● OMD2 / CSDL projects collaboration
● Will be available on Scilab forge:
– http://forge.scilab.org/index.php/p/omd2/
– Private project up to first release.
● Scilab Optimization Platform:
– Batch mode (script edition, large scale execution),
– GUI mode (interactive edition, prototyping).
● Future Scilab external module available through ATOMS.
Overview
![Page 32: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/32.jpg)
The free and open source software for numerical computation
● Project management (Save & Load working data as HDF5 files)
● Wrappers:
– Scilab algorithms,– External tools,– Proactive.
● Mask complexity for users
● Modules:
– Data Management,– Modeling,– Optimization,– Visualization.
Main functionalities
![Page 33: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/33.jpg)
The free and open source software for numerical computation
● Factors / Parameters:
– Load existing Design Of Experiments (Isight .db files, …)– Generate Design Of Experiments:
● DoE generator wrappers (LHS, …),● DoE generator settings.
● Responses simulation using:
– External tool (openFOAM, Catia, CCM+, …),– Scilab function.
● 2-D visualization:
– Factor / Factor,– Response / Factor.
Data Management Module (1/2)
![Page 34: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/34.jpg)
The free and open source software for numerical computation
Data Management Module (2/2)
![Page 35: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/35.jpg)
The free and open source software for numerical computation
● Point selection:
– Learning points used for modeling,– Validation points used to validate model,– Bad points (simulation issue, …).
● Modeler:
– Selected among modeler wrappers (DACE, Lolimot, …),– Parameters configuration,– Multiple model management with best model user selection.
● Visualization:
– 2-D models,– Cross correlation,– Sensibility analysis.
Modeling module (1/2)
![Page 36: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/36.jpg)
The free and open source software for numerical computation
Modeling module (2/2)
![Page 37: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/37.jpg)
The free and open source software for numerical computation
● Responses coefficients values setting
● Optimizer:
– Selection among generic wrappers (optim, fmincon, genetic algorithms, …),
– Optimizer configuration,– Enable two chained optimizers.
● Visualization:
– Optimal point,– Paretos,– Robustness.
Optimization Module (1/2)
![Page 38: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/38.jpg)
The free and open source software for numerical computation
Optimization Module (2/2)
![Page 39: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/39.jpg)
The free and open source software for numerical computationThe free software for numerical computation
1. What is missing ?
2. Bibliography
Part 4 - Conclusion
![Page 40: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/40.jpg)
The free and open source software for numerical computation
● High Performance Optimization:
– Use BLAS/LAPACK within optim ?● Sparse Linear Programming:
– Update LIPSOL ?● Non Linear Programming:
– Improve fmincon ?● Non Linear Programming Test Cases:
– CUTEr requires a compiler on the test machine, – Connect the Hock-Schittkowski collection ?
Conclusion1. What is missing ?
![Page 41: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/41.jpg)
The free and open source software for numerical computation
● « Nelder-Mead User's Manual », Michaël Baudin, Consortium Scilab – DIGITEO, 2010
● « Optimization in Scilab », Baudin, Couvert, Steer, Consortium Scilab - DIGITEO – INRIA, 2010
● « Optimization with scilab, present and future », Michaël Baudin and Serge Steer, 2009 IEEE International Workshop on Open Source Software for Scientific Computation, pp.99-106, 18-20 Sept. 2009
● « Introduction to Optimization with Scilab », Michaël Baudin, Consortium Scilab – DIGITEO, 2010
● « Unconstrained Optimality Conditions with Scilab », Michaël Baudin, Consortium Scilab – DIGITEO, 2010
Conclusion2. Bibliography
![Page 42: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/42.jpg)
The free and open source software for numerical computation
Thanks for your attention
www.scilab.org
![Page 43: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/43.jpg)
The free and open source software for numerical computation
Some slides you won't see, unless you ask...
Extra-Slides
![Page 44: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/44.jpg)
The free and open source software for numerical computation
Some Historical References:
● Spendley, Hext, Himsworth (1962): fixed shape simplex algorithm
● Nelder, Mead (1965): variable shape algorithm
● Box (1965): simplex algo., with constraints
● O'Neill (1971): Fortran 77 implementation.
● Torczon (1989): Multi Directional Search.
● Mc Kinnon (1998): Counter examples of N-M.
● Lagarias, Reeds, Wright, Wright (1998): Proof of convergence in dimensions 1 and 2 for strictly convex functions.
● Han, Neumann (2006): More counter examples of N-M.
The Nelder-Mead Algorithm
![Page 45: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/45.jpg)
The free and open source software for numerical computation
In what softwares N-M can be found ?
● Matlab (fminsearch)
● NAG (E04CBF)
● Numerical Recipes (amoeba)
● IMSL (UMPOL)
● … and Scilab since v5.2.0 in 2009
● … and R after Sébastien Bihorel's port of Scilab's source code.
The Nelder-Mead Algorithm
![Page 46: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/46.jpg)
The free and open source software for numerical computation
1. Sort by function value. Order the vertices: f(1) ≤ · · · ≤ f(n) ≤ f(n+1)
2. Calculate centroid. B = (v(1)+...+v(n))/n
3. Reflection. Compute R = (1+ρ)B − ρv(n+1) and evaluate f(R).
4. Expansion. If f(R)<f(1), compute E=(1+ρχ)B − ρχv(n+1) and evaluate f(E). If f(E)<f(R), accept E, else accept R and goto 1.
5. Accept R. If f(1) ≤ f(R) < f(n), accept R and goto 1.
6. Outside Contraction. If f(n)≤f(R)<f(n+1), compute Co=(1+ργ)B − ργv(n+1) and evaluate f(Co). If f(Co)<f(R), then accept Co and goto 1 else, goto 8.
7. Inside Contraction. If f(n+1)≤f(R), compute Ci=(1-γ)B +γv(n+1) and evaluate f(Ci). If f(Ci)<f(n+1), then accept Ci and goto 1 else, goto 8.
8. Shrink. Compute the points v(i)=v(1)+σ(v(i)-v(1)) and evaluate f(i)=f(x(i)), for i=2,3,...,n+1. Goto 1.
The Nelder-Mead Algorithm
![Page 47: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/47.jpg)
The free and open source software for numerical computation
● Lagarias, Reeds, Wright, Wright (1998)
1. In dimension 1, the Nelder-Mead method converges to a minimizer, and convergence is eventually M-step linear, when the reflection parameter ρ = 1.
2. In dimension 2, the function values at all simplex vertices in the standard Nelder-Mead algorithm converge to the same value.
3. In dimension 2, the simplices in the standard Nelder-Mead algorithm have diameters converging to zero.
● Note that Result 3 does not implies that the simplices converge to a single point x*.
The Nelder-Mead Algorithm
![Page 48: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/48.jpg)
The free and open source software for numerical computation
● Minimization:
– bintprog Solve binary integer programming problems– fgoalattain Solve multiobjective goal attainment problems– fminbnd Find minimum of single-variable function on
fixed interval
– fmincon Find minimum of constrained nonlinear
multivariable function
– fminimax Solve minimax constraint problem– fminsearch Find minimum of unconstrained multivariable
function using derivative-free method
What's in Matlab® ?
![Page 49: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/49.jpg)
The free and open source software for numerical computation
– fminunc Find minimum of unconstrained multivariable function
– fseminf Find minimum of semi-infinitely constrained
multivariable nonlinear function
– ktrlink Find minimum of constrained or unconstrained
nonlinear multivariable function using KNITRO
third-party libraries
– linprog Solve linear programming problems– quadprog Quadratic programming
What's in Matlab® ?
![Page 50: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/50.jpg)
The free and open source software for numerical computation
● Equation Solving:
– fsolve Solve system of nonlinear equations– fzero Find root of continuous function of one variable
● Least Squares (Curve Fitting):
– lsqcurvefit Solve nonlinear curve-fitting (data-fitting) problems in least-squares sense
– lsqlin Solve constrained linear least-squares problems– lsqnonlin Solve nonlinear least-squares problems– lsqnonneg Solve nonnegative least-squares constraint
problem
What's in Matlab® ?
![Page 51: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/51.jpg)
The free and open source software for numerical computation
● Utilities:
– optimtool GUI to select solver, optimization options, and
run problems
– optimget Optimization options values– optimset Create or edit optimization options structure
What's in Matlab® ?
![Page 52: Optimization with Scilab](https://reader036.vdocuments.us/reader036/viewer/2022082206/586e18a41a28ab3f228b7aba/html5/thumbnails/52.jpg)
The free and open source software for numerical computation
● Global Optimization Toolbox:
– GlobalSearch Create and solve GlobalSearch problems
– MultiStart Create and solve MultiStart problems– Genetic Algorithm Use genetic algorithm and Optimization
Tool, and modify genetic algorithm options
– Direct Search Use direct search and Optimization Tool,
and modify pattern search options
– Simulated Annealing Use simulated annealing and Optimization Tool, and modify simulated annealing options
What's in Matlab® toolboxes ?