lecture 2: canonical genetic algorithms - purdue …sudhoff/ee630/lecture02.pdflecture 2: canonical...
TRANSCRIPT
![Page 1: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/1.jpg)
Lecture 2:Canonical Genetic Algorithms
Suggested reading: D. E. Goldberg, Genetic Algorithm in Search, Optimization, and Machine Learning, Addison Wesley Publishing Company, January 1989
![Page 2: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/2.jpg)
2
What Are Genetic Algorithms?Genetic algorithms are optimization algorithm inspired from natural selection and geneticsA candidate solution is referred to as an individualProcess
Parent individuals generate offspring individualsThe resultant offspring are evaluated for their fitnessThe fittest offspring individuals survive and become parents The process is repeated
![Page 3: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/3.jpg)
3
History of Genetic AlgorithmsIn 1960’s
Rechenberg: “evolution strategies”Optimization method for real-valued parameters
Fogel, Owens, and Walsh: “evolutionary programming”Real-valued parameters evolve using random mutation
In 1970’sJohn Holland and his colleagues at University of Michigan developed “genetic algorithms (GA)”Holland’s1975 book “Adaptation in Natural and Artificial Systems” is the beginning of the GAHolland introduced “schemas,” the framework of most theoretical analysis of GAs.
![Page 4: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/4.jpg)
4
In 1990’s John Koza: “genetic programming” used genetic algorithms to evolve programs for solving certain tasks
It is generally accepted to call these techniques as evolutionary computation
Strong interaction among the different evolutionary computation methods makes it hard to make strict distinction among GAs, evolution strategies, evolutionary programming and other evolutionary techniques
![Page 5: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/5.jpg)
5
Differences Between GAs and Traditional Methods
GAs operate on encodings of the parameters values, not necessarily the actual parameter valuesGAs operate on a population of solutions, not a single solutionGAs only use the fitness values based on the objective functions GAs use probabilistic computations, not deterministic computationsGAs are efficient in handling problems with a discrete or mixed search spaces
![Page 6: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/6.jpg)
The Canonical GA
![Page 7: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/7.jpg)
7
Canonical GA The canonical genetic algorithm refers to the GA proposed by John Holland in 1965
START
END
STOP?
Initialization
Fitness evaluation
Selection
Crossover
Mutation
![Page 8: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/8.jpg)
8
Gene RepresentationParameter values are encoded into binary strings of fixed and finite length
Gene: each bit of the binary stringChromosome: a binary stringIndividual: a set of one or multiple chromosomes, a prospective solution to the given problemPopulation: a group of individuals
Longer string lengthsImprove resolutionRequires more computation time
![Page 9: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/9.jpg)
9
Binary Representation
Suppose we wish to maximize
Where
Binary representation
We map to
Thus
)(xf
[ ]maxmin xxx ,; =ΩΩ∈
[ ]maxmin xx , [ ]12,0 −l
[ ]121 bbbbx llbinary L−∈
∑=
−
−
−+=
l
i
iil
minmaxmin b
xxxx
1
1212
![Page 10: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/10.jpg)
10
Example
Let l = 5, Ω = [-5, 20]Then
[ ][ ]
[ ] 3226.1012
)5(20)222(5 1 1 0 0 1
20 1 1 1 1 1
5 0 0 0 0 0
5014
binary
binary
binary
=−−−
+++−=⇒=
=⇒=
−=⇒=
xx
xx
xx
![Page 11: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/11.jpg)
11
Fitness EvaluationEach individual x is assigned with a fitness value f(x) as the measure of performanceIt is assumed that the fitness value is positive and the better the individual as a solution, the fitness value is more positiveThe objective function can be the fitness function itself if it is properly defined
![Page 12: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/12.jpg)
12
Example 1
Consider the problem
A fitness function
max g(x) = -x2 + 4x, x ∈ [1, 5]
f(x) =-g(x)+100= -x2 + 4x + 100
![Page 13: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/13.jpg)
13
Example 2
Consider the problem
A fitness function
min g(x) = x2, x ∈ [-10, 10]
f(x) = 1/(g(x) + 0.1) = 1/(x2 + 0.1)
![Page 14: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/14.jpg)
14
SelectionChooses individuals from the current population to constitute a mating pool for reproductionFitness proportional selection methods
Each individual x is selected and copied in the mating pool with the probability proportional to fitness (f(x) / Σf(x))
![Page 15: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/15.jpg)
15
Roulette Wheel Selection
Winner19764427
8533176
Individuals with fitness values
Assign a piece proportional to the
fitness value
Mating pool
![Page 16: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/16.jpg)
16
Crossover
Single-point crossover is assumedTwo parent individuals are selected from mating poolCrossover operation is executed with the probability pc
Crossover point is randomly chosen and the strings are swapped with respect the crossover point between the two parents
![Page 17: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/17.jpg)
17
Single Point Crossover
1 0 1 0 0 1 1 0
1 1 0 0 1 0 1 1
1 0 1
0 0 1 1 01 1 0
0 1 0 1 1Parent 1
Parent 2
Child 1
Child 2
Crossover point
![Page 18: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/18.jpg)
18
Mutation
Mutation operator is applied gene-wise, that is, each gene undergoes mutation with the probability pmWhen the mutation operation occurs to a gene, its gene value is flipped
0110
1 1 0 0 1 0 1 1 1 0 0 0 1 1 1 1
Mutation points
![Page 19: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/19.jpg)
19
Overview of Canonical GAInitial
population
Mating poolMutation Crossover
Selection
19764427
8533176
FitnessEvaluation
f (x)
![Page 20: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/20.jpg)
20
Summary of Canonical GA
Binary representationFixed string lengthFitness proportional selection operatorSingle-point crossover operatorGene-wise mutation operator
![Page 21: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/21.jpg)
A Manual Example Using Canonical GAs
![Page 22: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/22.jpg)
22
Problem DescriptionConsider the following maximization problem
max f(x) = x2
where x is an integer between 0 and 31
0 5 10 15 20 25 300
100
200
300
400
500
600
700
800
900
1000
x
f(x)
f(x) = x2 has its maximum value 961 at x = 31
![Page 23: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/23.jpg)
23
Before applying GA, a representation method must be definedUse unsigned binary integer of length 5
A five-bit unsigned binary integer can have values between 0(0000) and 31(11111)
Gene Representation
10110 1⋅24 + 0⋅23 + 1⋅22 + 1⋅21 + 0⋅20 = 22
![Page 24: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/24.jpg)
24
Canonical GA Execution Flow
START
END
STOP?
Initialization
Fitness evaluation
Selection
Crossover
Mutation
![Page 25: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/25.jpg)
25
Initialization
Initial populations are randomly generatedSuppose that the population size is 4An example initial population
191 0 0 1 1480 1 0 0 03
241 1 0 0 02130 1 1 0 11
x valueInitial population
Individual No.
![Page 26: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/26.jpg)
26
Fitness EvaluationEvaluate the fitness of initial populationThe objective function f(x) is used as the fitness functionEach individual is decoded to integer and the fitness function value is calculated
![Page 27: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/27.jpg)
27
Fitness Evaluation
Decoding
Results
01000 Decode x = 8 f(8) = 82 = 64Evaluation
0.31
0.06
0.49
0.14
fi / Σf
1.24
0.24
1.96
0.56
Expected number
361
64
576
169
f(x)
19
8
24
13
x value
1 0 0 1 14
0 1 0 0 03
1 1 0 0 02
0 1 1 0 11
Initial populationIndividual No.
![Page 28: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/28.jpg)
28
SelectionSelect individuals to the mating poolSelection probability is proportional to the fitness value of the individualRoulette wheel selection method is used
0.310.060.490.14
fi / Σf
1.240.241.960.56
Expected number
36164576169
f(x)
1982413
x value
1 0 0 1 140 1 0 0 031 1 0 0 020 1 1 0 11
Initial populationIndividual No.
![Page 29: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/29.jpg)
29
1234
SelectionRoulette wheel
Outcome of Roulette wheel is 1, 2, 2, and 4Resulting mating pool
1 0 0 1 141 1 0 0 031 1 0 0 020 1 1 0 11
Mating poolNo.
![Page 30: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/30.jpg)
30
CrossoverTwo individuals are randomly chosen from mating poolCrossover occurs with the probability of pc = 1Crossover point is chosen randomly
16272512
x
1 0 0 0 01 1 0 1 11 1 0 0 10 1 1 0 0
New population
2
4
Crossover point
2561 0 0 1 17291 1 0 0 06251 1 0 0 01440 1 1 0 1
f(x) Mating pool
![Page 31: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/31.jpg)
31
MutationApplied on a bit-by-bit basisEach gene mutated with probability of pm = 0.001
18272512
x
324729625144
f(x)
1 0 0 1 01 1 0 1 11 1 0 0 10 1 1 0 0
After Mutation
1 0 0 0 01 1 0 1 11 1 0 0 10 1 1 0 0
Before Mutation
![Page 32: Lecture 2: Canonical Genetic Algorithms - Purdue …sudhoff/ee630/Lecture02.pdfLecture 2: Canonical Genetic Algorithms Suggested reading: D. E. Goldberg, Genetic Algorithm in Search,](https://reader031.vdocuments.us/reader031/viewer/2022022008/5adad24e7f8b9aee348d4747/html5/thumbnails/32.jpg)
32
Fitness EvaluationFitness values of the new population are calculated
1756sum1170sum439avg293avg
324181 0 0 1 0361191 0 0 1 1
max
82413
x
0 1 0 0 01 1 0 0 00 1 1 0 1
Old population
max
272512
x
729
729625144
f(x)
1 1 0 1 11 1 0 0 10 1 1 0 0
New population
576
64576169
f(x)