1. Dr. Kenneth Stanley February 6, 2006
CAP6938 Neuroevolution and Artificial Embryogeny NeuroEvolution of Augmenting Topologies (NEAT)
Dr. Kenneth Stanley
February 6, 2006

2. TWEANN Problems Reminder
Competing conventions problem
Topology matching problem
Initial population topology randomization
Defective starter genomes
Unnecessarily high-dimensional search space
Loss of innovative structures
More complex can’t compete in the short run
Need to protect innovation
NEAT directly addresses these challenges

3. Solutions: NEAT Historical markings match up different structures
Speciation
Keeps incompatible networks apart
Protects innovation
Incremental growth from minimal structure, i.e. complexification
Avoids searching in unnecessarily high-d space
Makes finding high-d solutions possible

4. Genetic Encoding in NEAT

5. Topological Innovation

6. Link Weight Mutation
A random number is added or subtracted from the current weight/parameter
The number can be chosen from uniform, Gaussian (normal) or other distributions
Continuous parameters work best if capped
The probability of mutating a particular gene may be low or high, and is separate from the magnitude added
Probabilities and mutation magnitudes have a significant effect

7. Link Weight Mutation in NEAT C++
randnum=randposneg()*randfloat()*power;
if (mut_type==GAUSSIAN) {
randchoice=randfloat();
if (randchoice>gausspoint)
((*curgene)->lnk)->weight+=randnum;
else if (randchoice>coldgausspoint)
((*curgene)->lnk)->weight=randnum; }
else if (mut_type==COLDGAUSSIAN)
//Cap the weights at 3.0
if (((*curgene)->lnk)->weight > 3.0) ((*curgene)->lnk)->weight = 3.0;
else if (((*curgene)->lnk)->weight < -3.0) ((*curgene)->lnk)->weight = -3.0;

8. Topology Matching Problem
Problem arises from adding new genes
Same gene may be in different positions
Different genes may be in same positions

9. Biological Motivation
New genes appeared over biological evolution as well
Nature has a solution to still know which is which
Process of aligning and matching genes is called synapsis
Uses homology to align genes:
“. . .Crossing over thus generates homologous
recombination; that is, it occurs between 2 regions of
DNA containing identical or nearly identical sequences.” (Watson et al. 1987)

10. Artificial Synapsys: Tracking Genes through Historical Markings
The numbers tell exactly when in history particular topological features
appeared, so now they can be matched up any time in the future. In
other words, they reveal gene homology.

11. Matching up Genes

12. Second Component: Speciation Protects Innovation
Originally used for multimodal function optimization (Mahfood 1995)
Organisms grouped by similarity (compatibility)
Fitness sharing (Goldberg 1987, Spears 1995): Organisms in a species share the reward of their fitness peak
To facilitate this, NEAT needs
A compatibility measure
Clustering based on compatibility, for fitness sharing

13. Measuring Compatibility
Possible in NEAT through historical markings
3 factors affect compatibility via historical markings on connection genes:
Excess
Disjoint
Average Weight Distance W
Compatibility distance

14. Clustering Into Species

15. Dynamic Compatibility Thresholding

16. Fitness Sharing: Assigning Offspring to Species

17. Third Component: Complexification from Minimal Structure
Addresses initialization problem
Search begins in minimal-topology space
Lower-dimensional structures easily optimized
Useful innovations eventually survive
So search transitions into good part of higher-dim. space
The ticket to high-dimensional space

18. NEAT Performed Well on Double Pole Balancing Without Velocity Inputs

19. DPNV Solutions Are Compact

20. Harder DPNV (0.3m short pole) solution

21. Visualizing Speciation

22. Next Class: More NEAT Implementation issues Where NEAT can be changed
Areas for advancement
Issues in applying NEAT (e.g. sensors and outputs)
Evolving a Roving Eye for Go by Kenneth O. Stanley and Risto Miikkulainen (2004) Neuroevolution of an Automobile Crash Warning System by Kenneth O. Stanley and Risto Miikkulainen (2005)
Homework due 2/15/05: Working domain and phenotype code. Turn in summary, code, and examples demonstrating how it works.

23. Project Milestones (25% of grade)
2/6: Initial proposal and project description
2/15: Domain and phenotype code and examples
2/27: Genes and Genotype to Phenotype mapping
3/8: Genetic operators all working
3/27: Population level and main loop working
4/10: Final project and presentation due (75% of grade)