Evolution StrategyHow Nature Solves Problems
Ingo Rechenberg
Shanghai Institute for Advanced Studies
CAS-MPG Partner Institute for Computational Biology / 2006-04-11
1 What Evolution Strategy does2 How Evolution Strategy works
1Protoplasm lump in the primordial ocean
What Evolution does
2From this the fish
developed
What Evolution does
3Life peeks out of the water and spreads over the country
What Evolution does
4Our ancestors climb
the treetops
What Evolution does
5Finally we admire
ourselves in the mirror
What Evolution does
History of the Evolution Strategy
Windtunnel
Flexible flow body
to adjust random mutations
Air flow
Gear
DARWIN in the windtunnel
The kink plate for the key experiment with the Evolution Strategy
Number of possible adjustments
515 = 345 025 251
80400
2
4
6
0120 160 200 240 280 320
M u ta t io n s
R e s u lt
Re
sis
tan
ce
The experimentum crucis – Drag minimization of the kink plate
Zigzag after DARWIN
Story in the Magazin
18 th November 1964
Start Result
Six manually adjustable shafts determine the form of a 90°pipe bend
Evolution of a 90° pipe bend
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
35
31
32
33
34
36
37
38
39
40
41
42
43
44
0
45
Evolution of a two phase flow nozzle(Hans-Paul Schwefel)
HistoryEvolution Strategy today
Evolution-Strategy
ES-]),/(,/[
Wright Haldane Fisher ' = Number of offspring populations'= Number of population generations
' = Number of parental populations
= Number of parental individuals
= Number of offspring individuals = Generations of isolation
' = Mixing number for populations
= Mixing number for individuals
Elementary Evolution-Strategic Algorithms
(1 + 1)-ES
DARWINs theory at thelevel of maximum abstraction
(1 , )-ES
Evolution Strategywith more than one offspring
= 6
( , )-ES
Evolution Strategy with more parents and more offspring
= 7 = 2
( , )-ES
Evolution Strategy with mixing of variables
= 8
= 2 = 2
ES]),(,[
12154
New founder populations
The NestedEvolution Strategy
will be an algebraic scheme
The notation
An artificial evolution experiment in the windtunnel
Evolution of a spread wing in the windtunnel
Multiwinglets at a glider designed with the Evolution Strategy
Phot
o: M
icha
el S
tach
e
Darwin was very uncertain whether his theory is correct.
To suppose that the eye, with all its inimitable contrivances for adjusting the focus to different distances, for admitting different amounts of light, and for the correction of spherical and chromatic abberation, could have been formed by natural selection, seems, I freely confess, absurd in the highest possible degree.
He stated in his book „The Origin of Species“:
Fdk
qk
Evolution of an eye lens
Computer simulated evolution of a covergent lens
Flexible glass body
Minimum2kq
Evolution-strategic development of a framework construction
y
x
Weight Minimum
y
x
Weight Minimum
y
x
Weight Minimum
y
x
Weight Minimum
Evolution-strategic optimization of a truss bridge with minimum weight
Arched bridge
Fishbelly bridge
Bridge
designs
Lu Pu Bridge
Dynamic optimization of a truss bridge
Melencolia, engraved in 1514 by Albrecht Dürer
Magic SquareChinese
2 0 0 6
Min)()()()()()()()(
2753
2951
2963
2852
2741
2987
2654
2321
1515151515151515
nnnnnnnnnnnnnnnnnnnnnnnnQ
Objective function for a 3 3-square ?
nn
14
7
2
5
8
3
6
9
nnn
nnn
y
x
The min/max distance problem
DD
min
maxMinimum
ES-Solutions of the min/max-
distance problem
7 Points 12 Points
24 Points 27 Points
9093,2325minmax / DD2minmax / DD
5826,421minmax / DD 8045,4minmax / DD
Maximum distance = 1
Minimum distance
Optimal swarm configuration of 48 individuals
Dmax
Dmin= 6.707
94
94
86
86
103
10377
77
Elements of the optimal structure
Structure of the 48 individual swarm
2 How Evolution Strategy works
Search for a document
(Search)Strategies are of no use in an disordered world
(Search)Strategies need a predictable order of the world
Strategy in military operation
A military strategy is of no use, if the enemy behaves randomly
General
An evolution strategy is of no use, if nature (opponent) behaves randomly
Evolution Strategist
Causality
Weak Causality
Strong Causality
A predictable world order is
Equal cause, equal effect
Similar cause, not similar effect
Similar cause, similar effect !
Billiards-Effect
Example for
weak causality
Strong Causality
Normal behaviour of the world
Weak and strong causality in a graphic view
Weak causality
Strong causality
Experimenter
Plumbing the depth
Search area
The search for the optimum
The search for the optimum
Plumbing the depth
Experimenter
Search area
1. Global deterministic search
3. Local deterministic search
2. Global stochastic search
4. Local stochastic search
4 strategies to localize an optimum
Z
1 m
m
1
1. Global deterministic searchSystematic scanning of the variable space
2)2( mG
nn mG )(
Z
1 m
m
1
2. Global stochastic searchTo find the target with 95% probability
2)2( 99.2 mG
nn mG 99.2)(
1. Global deterministic search
3. Local deterministic search
2. Global stochastic search
4. Local stochastic search
4 strategies to localize an optimum
distance moved uphillnumber of generations
Definition of the rate of progress
Z
x
y
Linearity radius
Progress
3. Local deterministic searchWalking following the steepest ascent
3)2(
grad
1)(
grad nn
Z
x
y
Linearity radius
4. Local stochastic searchRandom drifting along the steepest ascent
1. Offspring
2. OffspringParent
?)2(evo
?)(evon
Plus-offspring
Minus-offspring Center of gravity
Statistical mean of the progress
Determiation of the linear rate of
progress
ParentLinearity radius
2/s
s+
−
Because half of the offspring are failures
rr
rs rs 2
1 rn
ns )(
)(2
121
2 Dim. 3 Dim. n Dim.
s ss
Center of gravity
4
n1
2 n >> 1
rn
s 21
n >> 1
Gradient Strategy contra Evolution Strategy
For n >> 1
nn
2
1)(evon
n )(grad
1/ n
Evolution Strategy
1/n
Gradient Strategy
Local climbing of the Evolution Strategy
linear
Local climbing of the Evolution Strategy
nonlinear
)0(!2
1)0(!1
1)0()0(1 1
2
1ji
n
i
n
j jii
n
i ixxxx
fxxfff
x
TAYLOR series expansionin n dimensions (MACLAURIN series)
1 11
0 ji
n
i
n
jjii
n
ii xxbxaQQ
Transformation to the principle axes
2
110 k
n
ikk
n
kk xdxcQQ
nc
dn
cn
n
k
k 2
2,1
Tabel
1 0
2 0,5642
3 0,8463
4 1,0294
5 1,1630
6 1,2672
7 1,3522
8 1,4236
9 1,4850
10 1,5388
,1c
11 1,5864
12 1,6292
13 1,6680
14 1,7034
15 1,7359
16 1,7660
17 1,7939
18 1,8200
19 1,8445
20 1,8675
,1c
21 1,8892
22 1,9097
23 1,9292
24 1,9477
25 1,9653
26 1.9822
27 1,9983
28 2,0137
29 2,0285
30 2,0428
,1c
35 2,1066
40 2,1608
45 2,2077
50 2,2491
55 2,2860
60 2,3193
65 2,3496
70 2,3774
80 2,4268
90 2,4697
,1c
100 2,5076
200 2,7460
300 2,8778
400 2,9682
500 3,0367
600 3,0917
700 3,1375
800 3,1768
900 3,2111
1000 3,2414
,1c
of the progress coefficients
= zero
=
high
= medium
The complexity
nc
dn
cn
n
k
k 2
2,1
rn2
r
Evolution Window
-5 -3 -1 310
0,2
0,1
0,3
1 01 01 01 010
2,1
c
,1cn
2 Central law of progress
not so
but so
For n >> 1 the white catchment areas of the hills are neglectible small compared with the vaste black space between them
Parent
Evolution Window
-5 -3 -1 310
0,2
0,1
0,3
1 01 01 01 010
How to find the Evolution Window ?
Mutation
Duplicator
DNA
Has made the duplicator
Heredity of the mutabilityCrucial point of the Evolution Strategy
? ? ?
I am the fi rst
Assessment of the climbing style
Climbing alone Climbing in a group
N
Four mountaineers, four climbing styles
Fraidycat
Columbus
Amundsen
Hothead
In a compact notation
Nested Evolution Strategy
Four moutaineers, four climbing styles
On the way to an evolution-strategic algebra
1 +1( ) - ES ,+,
On the way to an evolution-strategic algebra
( ) - ES +,
On the way to an evolution-strategic algebra
/
Example = 2
( ) - ES +,/ 2
Only half of the parental information builds up an offspringMulti-Recombination
( ) - ES +,
On the way to an evolution-strategic algebra
Example:
(1+ 6)4 = (1+ 6) (1+ 6) (1+ 6) (1+ 6)
( ) - ES +,
On the way to an evolution-strategic algebra
+,[ ]
| Family Genus { Species [ Variety ( Individual ) ] } |
Biological equivalent to the strategy nesting
( ) - ES +,
Nested Evolution Strategy
+,[ ]
Adaptationof the objektive
variables xk
Adaptationof the
mutation size
to operate in the Evolution Window!
Reduction of the lateral component of the mutation step using intermediary variable mixing (multi-recombination)
Contour line
Parent
Best of offspring
Recombination of the best of offspring
Reduction of the lateral mutation step
Lateral component
Progress component
The wonder of sexual reproduction
nnc2
,,
nnc2
,,
with multi-recombination
without recombination
42,
max,c
times faster !
I thank you for your attention
www.bionik.tu-berlin.de