cap6938 neuroevolution and artificial embryogeny neuroevolution of augmenting topologies (neat)
DESCRIPTION
CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT). Dr. Kenneth Stanley February 6, 2006. TWEANN Problems Reminder. Competing conventions problem Topology matching problem Initial population topology randomization Defective starter genomes - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/1.jpg)
CAP6938Neuroevolution and Artificial Embryogeny
NeuroEvolution of Augmenting Topologies
(NEAT)Dr. Kenneth Stanley
February 6, 2006
![Page 2: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/2.jpg)
TWEANN Problems Reminder
• Competing conventions problem– Topology matching problem
• Initial population topology randomization – Defective starter genomes– Unnecessarily high-dimensional search space
• Loss of innovative structures– More complex can’t compete in the short run– Need to protect innovation
• NEAT directly addresses these challenges
![Page 3: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/3.jpg)
Solutions: NEAT
• Historical markings match up different structures• Speciation
– Keeps incompatible networks apart– Protects innovation
• Incremental growth from minimal structure, i.e. complexification– Avoids searching in unnecessarily high-d
space– Makes finding high-d solutions possible
![Page 4: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/4.jpg)
Genetic Encoding in NEAT
![Page 5: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/5.jpg)
Topological Innovation
![Page 6: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/6.jpg)
Link Weight Mutation• A random number is added or subtracted from
the current weight/parameter• The number can be chosen from uniform,
Gaussian (normal) or other distributions• Continuous parameters work best if capped• The probability of mutating a particular gene
may be low or high, and is separate from the magnitude added
• Probabilities and mutation magnitudes have a significant effect
![Page 7: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/7.jpg)
Link Weight Mutation in NEAT C++
randnum=randposneg()*randfloat()*power;if (mut_type==GAUSSIAN) {
randchoice=randfloat(); if (randchoice>gausspoint) ((*curgene)->lnk)->weight+=randnum; else if (randchoice>coldgausspoint) ((*curgene)->lnk)->weight=randnum; } else if (mut_type==COLDGAUSSIAN) ((*curgene)->lnk)->weight=randnum;
//Cap the weights at 3.0 if (((*curgene)->lnk)->weight > 3.0) ((*curgene)->lnk)->weight = 3.0; else if (((*curgene)->lnk)->weight < -3.0) ((*curgene)->lnk)->weight = -3.0;
![Page 8: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/8.jpg)
Topology Matching Problem
• Problem arises from adding new genes• Same gene may be in different positions
• Different genes may be in same positions
![Page 9: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/9.jpg)
Biological Motivation• New genes appeared over biological evolution as well• Nature has a solution to still know which is which
– Process of aligning and matching genes is called synapsis
– Uses homology to align genes:
“. . .Crossing over thus generates homologousrecombination; that is, it occurs between 2 regions ofDNA containing identical or nearly identical sequences.” (Watson et al. 1987)
![Page 10: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/10.jpg)
Artificial Synapsys: Tracking Genes through Historical Markings
The numbers tell exactly when in history particular topological featuresappeared, so now they can be matched up any time in the future. Inother words, they reveal gene homology.
![Page 11: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/11.jpg)
Matching up Genes
![Page 12: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/12.jpg)
Second Component: Speciation Protects Innovation
• Originally used for multimodal function optimization (Mahfood 1995)
• Organisms grouped by similarity (compatibility)• Fitness sharing (Goldberg 1987, Spears 1995):
Organisms in a species share the reward of their fitness peak
• To facilitate this, NEAT needs– A compatibility measure– Clustering based on compatibility, for fitness sharing
![Page 13: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/13.jpg)
Measuring Compatibility• Possible in NEAT through historical markings• 3 factors affect compatibility via historical
markings on connection genes: – Excess – Disjoint– Average Weight Distance W
• Compatibility distance WcNDc
NEc
321
![Page 14: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/14.jpg)
Clustering Into Species
![Page 15: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/15.jpg)
Dynamic Compatibility Thresholding
![Page 16: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/16.jpg)
Fitness Sharing: Assigning Offspring to Species
![Page 17: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/17.jpg)
Third Component: Complexification from Minimal Structure
• Addresses initialization problem• Search begins in minimal-topology space• Lower-dimensional structures easily optimized• Useful innovations eventually survive• So search transitions into good part of higher-dim. space• The ticket to high-dimensional space
![Page 18: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/18.jpg)
NEAT Performed Well on Double Pole Balancing Without Velocity
Inputs
![Page 19: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/19.jpg)
DPNV Solutions Are Compact
![Page 20: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/20.jpg)
Harder DPNV (0.3m short pole) solution
![Page 21: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/21.jpg)
Visualizing Speciation
![Page 22: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/22.jpg)
Next Class: More NEAT
• Implementation issues• Where NEAT can be changed• Areas for advancement• Issues in applying NEAT (e.g. sensors and
outputs)
Homework due 2/15/05: Working domain and phenotype code. Turn in summary, code, and examples demonstrating how it works.
Evolving a Roving Eye for Go by Kenneth O. Stanley and Risto Miikkulainen (2004) Neuroevolution of an Automobile Crash Warning System by Kenneth O. Stanley and Risto Miikkulainen (2005)
![Page 23: CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)](https://reader036.vdocuments.us/reader036/viewer/2022062501/568160e0550346895dd01079/html5/thumbnails/23.jpg)
Project Milestones (25% of grade)
• 2/6: Initial proposal and project description• 2/15: Domain and phenotype code and examples• 2/27: Genes and Genotype to Phenotype mapping • 3/8: Genetic operators all working• 3/27: Population level and main loop working• 4/10: Final project and presentation due (75% of grade)