genetic algorithms: an introduction · – evolution 4. genetics • gene: basic unit of heredity...
TRANSCRIPT
![Page 1: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/1.jpg)
GENETIC ALGORITHMS:AN INTRODUCTION
Date: Tuesday 17 February 2015
Course: Functional Programming and Intelligent Algorithms
Lecturer: Robin T. Bye
1
![Page 2: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/2.jpg)
Natural optimisation methods
2
![Page 3: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/3.jpg)
Nature-inspired algorithms• Genetic algorithm (GA)
• Genetic programming (GP)
• Simulated annealing (SA)
• Particle swarm optimisation (PSO)
• Ant colony optimisation (ACO)
• Evolutionary strategies (ES)
• Cultural algorithms (CA)
• +++
3
![Page 4: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/4.jpg)
Natural selection• Biological optimisation
• Observations from nature:
1. Extreme diversity of organisms
2. Large degree of complexity in organisms
3. Organism features with great usefulness
Survival of the fittest
• Natural selection have two components:
– Genetics
– Evolution
4
![Page 5: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/5.jpg)
Genetics
• Gene: Basic unit of heredity
– Determines characteristics or functions of an individual
• Chromosome: Set of genes
• DNA: Set of chromosomes
• Humans: 1 DNA with 23 pairs (=46) of chromosomes with about 40 000 genes
5
![Page 6: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/6.jpg)
6
![Page 7: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/7.jpg)
Evolution
• Darwin’s four premises:
1. Offspring inherits many characteristics from parents
2. Variations in characteristics between individuals can be passed on
3. Only a small percentage of offspring survives to adulthood
4. Survival is dependent on inherited characteristics
7
![Page 8: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/8.jpg)
8
![Page 9: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/9.jpg)
Evolution
• Static population –> No evolution
• Dynamic population requires
– Mutations (random changes in gene characteristics)
– Gene flow (new organisms join the population)
– Genetic drift (occurs due to chance)
– Natural selection (survival of the fittest)
9
![Page 10: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/10.jpg)
Simulation of natural evolution
• Evolutionary computing is based on:
– Creating an initial population
– Evaluate the fitness of each individual
– Generate new population through genetic operations
– Repeat until desired fitness reached
10
![Page 11: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/11.jpg)
11
Natural evolution simulated evolution
![Page 12: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/12.jpg)
Neo-Darwinian paradigm
• Mendel: Genetics
• Darwin, Wallace: Theory of evolution
• Weismann: Theory of natural selection
• Together Neo-Darwinism
• Central processes:
– Reproduction, mutation, competition, selection
12
![Page 13: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/13.jpg)
Evolutionary fitness
• A population’s ability to survive and reproduce over several generations
• Ability to anticipate environmental changes
• Is optimised in natural life
13
![Page 14: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/14.jpg)
The genetic algorithm (GA)
14
![Page 15: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/15.jpg)
Principles of the GA
• Optimisation and search technique based on principles of genetics and natural selection
• A population with many individuals evolves under specified selection rules
• Each generation improves fitness (minimises cost function) until satisfied
15
![Page 16: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/16.jpg)
History
• Developed by John Holland (1970s)
• Popularised by David Goldberg (1990s)
– Holland’s aim: Imitate nature’s evolution with computers
– Concerned with bit strings
– 1 bit = 1 artificial gene (either 0 or 1)
– Bit string = artificial chromosome
16
1 10 1 0 1 0 0 0 0 0 1 0 1 10
![Page 17: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/17.jpg)
Encoding and evaluation
– Encoding and evaluation mechanisms link GA to the problem to be solved
– Encoding:
• How to represent candidate solutions
• Binary, continuous-valued, other schemes
– Evaluation:
• How good is a candidate solution?
• Choice of cost function, performance of chromosome (candidate solution)
17
![Page 18: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/18.jpg)
Advantages of GA
– Can optimise with both discrete and continuous variables
– Does not need derivative information
– Simultaneously searches a wide sampling of the cost surface
– Handles large number of variables
– Well suited for parallel computing
– Can optimise in complex cost surfaces
18
![Page 19: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/19.jpg)
Advantages of GA
– Does not get stuck in local minima
– Can provide several optimal solution candidates, not just one solution
– Can encode variables so optimisation is done with encoded variables
– Works with• numerically generated data• experimental data
• analytical functions
– Can stop GA anytime with a (working) non-optimal solution that is often good enough
19
![Page 20: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/20.jpg)
Basic steps in a GA1. Represent the problem variable domain (encoding)
2. Define a cost function
3. Generate initial population
4. Calculate cost of each chromosome
5. Select parent chromosomes for mating based on cost
6. Create offspring chromosome based on genetic operators crossover and mutation
7. Place offspring in new population
8. Repeat Steps 5-7 until size of new population equals initial population
9. Replace initial population (parents) with new population (offspring)
10. Go to Step 4 and repeat steps until stopping criteria (reached global minimum, certain cost, max number of iterations, or other)
20
![Page 21: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/21.jpg)
Reproduction
• Select some chromosomes in population for reproduction, eg.,
• top half of a ranked list
• roulette wheel selection
• Use genetic operators for reproduction:
• Crossover
• Mutation
21
![Page 22: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/22.jpg)
Crossover
• Two parent chromosomes are each split up in two parts at crossover point
• Swap all genes to the right of crossover point to create offspring chromosomes
• Let crossover only happen with some probability
• No crossover clone parents
22
![Page 23: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/23.jpg)
23
X6i 1 00 0 01 0 X2i
0 01 0X2i 0 11 1 X5i
0X1i 0 11 1 X5i1 01 0
0 10 0
11 101 0
Crossoverat 3rd gene
Crossoverat 2nd gene
Cloning
Task: Make a table showing chromosomes before and after
![Page 24: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/24.jpg)
Mutation
• Change in a gene, e.g. flip a 1 to a 0
• Randomly select which gene
• Some small probability of mutation to occur
• Aids against getting trapped in local minimum
24
![Page 25: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/25.jpg)
25
0 11 1X5'i 01 0
X6'i 1 00
0 01 0X2'i 0 1
0 0
0 1 111X5i
1 1 1 X1"i1 1
X2"i0 1 0
0X1'i 1 1 1
0 1 0X2i
![Page 26: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/26.jpg)
Roulette wheel selection
26
• Rank chromosomes in list• Give a weight according to
fitness/ability to minimise cost
• High fitness, high weight
• Spin wheel and randomly draw chromosomes
• Fit chromosomes have higher probabiliy of getting picked
![Page 27: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/27.jpg)
27
100 0
16.5
36.743.149.5
75.2
X1: 16.5%X2: 20.2%X3: 6.4%X4: 6.4%X5: 25.3%X6: 24.8%
![Page 28: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/28.jpg)
GA cycle
28
1 01 0X1i
Generation i
0 01 0X2i
0 00 1X3i
1 11 0X4i
0 11 1X5i f = 56
1 00 1X6i f = 54
f = 36
f = 44
f = 14
f = 14
1 00 0X1i+1
Generation (i + 1)
0 01 1X2i+1
1 10 1X3i+1
0 01 0X4i+1
0 11 0X5i+1 f = 54
0 11 1X6i+1 f = 56
f = 56
f = 50
f = 44
f = 44
Crossover
X6i 1 00 0 01 0 X2i
0 01 0X2i 0 11 1 X5i
0X1i 0 11 1 X5i1 01 0
0 10 0
11 101 0
Mutation
0 11 1X5'i 01 0
X6'i 1 00
0 01 0X2'i 0 1
0 0
0 1 111X5i
1 1 1 X1"i1 1
X2"i0 1 0
0X1'i 1 1 1
0 1 0X2i
![Page 29: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/29.jpg)
Other selection methods
29
• Tournament selection: Picks a subset of the population at random, then selects the members with the best fitness.
• Truncation selection: Chromosomes whose cost is below a certain value (truncation point) are selected as parents for the next generation.
![Page 30: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/30.jpg)
Other selection methods
30
• Single-point crossover: A random-point in two chromosomes is selected. Offspring are formed by joining the genetic material to the right of the crossover point of one parent with the genetic material to the left of the crossover point of the other parent.
![Page 31: GENETIC ALGORITHMS: AN INTRODUCTION · – Evolution 4. Genetics • Gene: Basic unit of heredity – Determines characteristics or functions of an individual • Chromosome: Set](https://reader036.vdocuments.us/reader036/viewer/2022070903/5f690e38c0231720a535b022/html5/thumbnails/31.jpg)
Other selection methods
31
• Two-point crossover: Choose two points in the chromosome and exchange bits between those two points.
• Uniform crossover: Randomly assign the bit from one parent to one offspring and the bit from the other parent to the other offspring.