Download - CMA-ES with local meta-models
![Page 1: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/1.jpg)
Investigating the Local-Meta-Model CMA-ES forLarge Population Sizes
Zyed Bouzarkouna1,2 Anne Auger2 Didier Yu Ding1
1IFP (Institut Francais du Petrole)
2TAO Team, INRIA Saclay-Ile-de-France, LRI
April 07, 2010
![Page 2: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/2.jpg)
Statement of the Problem
Objective
To solve a real-world optimization problem formulated in ablack-box scenario with an objective function f : Rn 7→ R.
f may be:
multimodal non-smoothnoisy non-convexnon-separable computationally expensive. . .
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 2 of 15
![Page 3: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/3.jpg)
A Real-World Problem in Petroleum Engineering
History Matching
The act of adjusting a reservoir model until it closely reproducesthe past behavior of a production history.
A fluid flow simulation takes several minutes to several hours !!
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 3 of 15
![Page 4: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/4.jpg)
Statement of the Problem (Cont’d)
Difficulties
Evolutionary Algorithms (EAs) are usually able to cope withnoise, multiple optima . . .
Computational cost
build a model of f , based on true evaluations ;
use this model during the optimization to save evaluations.
⇒ How to decide whether:
the quality of the model is good enough to continueexploiting this model ?
ornew evaluations on the “true” objective function should be
performed ?
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 4 of 15
![Page 5: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/5.jpg)
Table of Contents
1 CMA-ES with Local-Meta-ModelsCovariance Matrix Adaptation-ESLocally Weighted RegressionApproximate Ranking Procedure
2 A New Variant of lmm-CMAA New Meta-Model Acceptance Criterionnlmm-CMA Performance
3 Conclusions
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 5 of 15
![Page 6: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/6.jpg)
Covariance Matrix Adaptation-ES
CMA-ES (Hansen & Ostermeier 2001)
Initialize distribution parameters m, σ and C, set population sizeλ ∈ N.while not terminate
Sample xi = m + σNi (0,C), for i = 1 . . . λ according to amultivariate normal distribution
Evaluate x1, . . . , xλ on f
Update distribution parameters(m, σ,C)← (m, σ,C, x1, . . . , xλ, f (x1), . . . , f (xλ))
where
m ∈ Rn: the mean of the multivariate normal distribution
σ ∈ R+: the step-size
C ∈ Rn×n: the covariance matrix.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 6 of 15
![Page 7: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/7.jpg)
Covariance Matrix Adaptation-ES (Cont’d)
Moving the mean
m =µ(=λ
2 )∑i=1
ωixi :λ.
where xi :λ is the i th ranked individual:
f (x1:λ) ≤ . . . f (xµ:λ) ≤ . . . f (xλ:λ) ,
ω1 ≥ . . . ≥ ωµ > 0,µP
i=1ωi = 1.
Other updates
Adapting the Covariance Matrix
Step-Size Control
⇒ Updates rely on the ranking of individuals according to f andnot on their exact values on f .
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 7 of 15
![Page 8: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/8.jpg)
Locally Weighted Regression
q ∈ Rn: A point to evaluate
⇒ f (q) : a full quadratic meta-model on q.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 8 of 15
![Page 9: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/9.jpg)
Locally Weighted Regression
A training set containing m points with their objective functionvalues (xj , yj = f (xj)) , j = 1..m
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 8 of 15
![Page 10: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/10.jpg)
Locally Weighted Regression
We select the k nearest neighbor data points to q according toMahalanobis distance with respect to the current covariance matrixC.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 8 of 15
![Page 11: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/11.jpg)
Locally Weighted Regression
h is the bandwidth defined by the distance of the kth nearestneighbor data point to q.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 8 of 15
![Page 12: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/12.jpg)
Locally Weighted Regression
Building the meta-model f on q
mink∑
j=1
[(f (xj , β)− yj
)2ωj
], w.r.t β ∈ R
n(n+3)2
+1.
f (q) = βT(q2
1 , · · · , q2n, · · · , q1q2, · · · , qn−1qn, q1, · · · , qn, 1
)T.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 8 of 15
![Page 13: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/13.jpg)
Approximate Ranking Procedure
Every generation g , CMA-ES has λ points to evaluate.
⇒ Which are the points that must be evaluated with:
the true objective function f ?
the meta-model f ?
Approximate ranking procedure (Kern et al. 2006)
1 approximate f and rank the µ best individuals
2 evaluate f on the ninit best individuals
3 for nic := 1 to“λ−ninit
nb
”do
4 approximate f and rank the µ best individuals
5 if (the exact ranking of the µ best individuals changes) then
6 evaluate f on the nb best unevaluated individuals
7 else
8 break
9 fi
10 od
11 adapt ninit depending on nic
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 9 of 15
![Page 14: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/14.jpg)
A New Meta-Model Acceptance Criterion
Requiring the preservation of the exact ranking of the µ bestindividuals is a too conservative criterion to measure the quality ofthe meta-model.
New acceptance criteria (nlmm-CMA)
The meta-model is accepted if it succeeds in keeping:
the best individual and the ensemble of the µ best individualsunchanged
or
the best individual unchanged, if more than one fourth of thepopulation is evaluated.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 10 of 15
![Page 15: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/15.jpg)
nlmm-CMA Performance
Success Performance (SP1):
SP1 = mean (number of function evaluations for successful runs)ratio of successful runs .
Speedup (algo) = SP1(algo)SP1(CMA−ES) .
0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 11 of 15
![Page 16: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/16.jpg)
nlmm-CMA Performance
4 nlmm-CMA � lmm-CMA
fSchwefel fSchwefel1/4 fNoisySphere
(2, 6) (4, 8) (8, 10) (16, 12)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 6) (4, 8) (5, 8) (8, 10)0
2
4
6
8
Sp
eed
up
(Dimension, Population Size)(2, 6) (4, 8) (8, 10) (16, 12)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
fRosenbrock fAckley fRastrigin
(2, 6) (4, 8) (5, 8) (8, 10)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 5) (5, 7) (10, 10)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 50) (5, 140)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
⇒ nlmm-CMA outperforms lmm-CMA, on the test functions investigated
with a speedup between 1.5 and 7.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 11 of 15
![Page 17: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/17.jpg)
nlmm-CMA Performance for Increasing PopulationSizes
4 nlmm-CMA � lmm-CMADimension n = 5
fSchwefel1/4 fRosenbrock fRastrigin
8 16 24 32 48 960
1
2
3
4
5
Population Size
Sp
eed
up
8 16 24 32 48 960
1
2
3
4
5
Population Size
Sp
eed
up
70 140 2800
1
2
3
4
5
Population Size
Sp
eed
up
⇒ nlmm-CMA maintains a significant speedup,between 2.5 and 4, when
increasing λ while the speedup of lmm-CMA drops to one.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 12 of 15
![Page 18: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/18.jpg)
Impact of the Recombination Type
nlmm-CMA
a default weighted recombination type
ωi = ln(µ+1)−ln(i)µ ln(µ+1)−ln(µ!) , for i = 1 . . . µ.
nlmm-CMAI
an intermediate recombination type
ωi = 1µ , for i = 1 . . . µ.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 13 of 15
![Page 19: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/19.jpg)
Impact of the Recombination Type (Cont’d)
4 nlmm-CMA � nlmm-CMAI (with equal RT)
fSchwefel fSchwefel1/4 fNoisySphere
(2, 6) (4, 8) (8, 10) (16, 12)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 6) (4, 8) (8, 10)0
2
4
6
8
Sp
eed
up
(Dimension, Population Size)(2, 6) (4, 8) (8, 10) (16, 12)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
fRosenbrock fAckley fRastrigin
(2, 6) (4, 8) (8, 10)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 5) (5, 7) (10, 10)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
(2, 50) (5, 140)0
2
4
6
8
(Dimension, Population Size)
Sp
eed
up
⇒ nlmm-CMA outperforms nlmm-CMAI .
⇒ The ranking obtained with the new acceptance criterion still has an amount
of information to guide CMA-ES.Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 14 of 15
![Page 20: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/20.jpg)
Summary
CMA-ES with meta-modelsThe speedup of lmm-CMA with respect to CMA-ES drops to one when the populationsize λ increases.
⇒ The meta-model acceptance criterion is too conservative.
New variant of CMA-ES with meta-modelsA new meta-model acceptance criterion: It must keep:
the best individual and the ensemble of the µ best individuals unchanged
the best individual unchanged, if more than one fourth of the population
is evaluated.
nlmm-CMA outperforms lmm-CMA on the test functions investigated with aspeedup in between 1.5 and 7.
nlmm-CMA maintains a significant speedup, between 2.5 and 4, when
increasing the population size on tested functions.
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 15 of 15
![Page 21: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/21.jpg)
Thank You For Your Attention
Zyed Bouzarkouna, Anne Auger, Didier Yu Ding page 16 of 15
![Page 22: CMA-ES with local meta-models](https://reader036.vdocuments.us/reader036/viewer/2022081404/5592f0121a28ab8b138b4641/html5/thumbnails/22.jpg)
Investigating the Local-Meta-Model CMA-ES forLarge Population Sizes
Zyed Bouzarkouna1,2 Anne Auger2 Didier Yu Ding1
1IFP (Institut Francais du Petrole)
2TAO Team, INRIA Saclay-Ile-de-France, LRI
April 07, 2010