1. Guarding Against Premature Convergence while Accelerating Evolutionary Search July 9, 2010 Josh C. Bongard Morphology, Evolutionary and Cognition Laboratory Department of Computer Science College of Engineering and Mathematical Sciences University of Vermont josh.bongard@uvm.edu Gregory S. Hornby University Affiliated Research Center University of California Santa Cruz NASA Ames Research Center, Mail Stop 269-3 Moffett Field, CA USA

2. Computational Cost of Evolutionary Robotics In Evolutionary Robotics… Robots are simulated in a 3D physics engine To generate robust behavior, each robot must be evaluated multiple times in different environments.

3. Computational Cost of Evolutionary Robotics In Evolutionary Robotics… Robots are simulated in a 3D physics engine To generate robust behavior, each robot must be evaluated multiple times in different environments. Incremental shaping is often employed: Evolve the robot in a single environment. When successful, re-evaluate the successful robot (and its progeny) in the original and a new environment… Problem: If fitness drops to 0 in the new environment, evolution starts from scratch in which each robot must be evaluated twice…

4. Guarding Against Premature Convergence Solution: Employ a multi-deme evolutionary algorithm such as ALPS. Population divided into layers Genomes migrate up from lower to higher layers. Robots in higher layers are exposed to more environments than robots in lower layers. Periodically, seed the lowest layer with random genomes. Each genome has an age, and each layer has an age cap: If a genome is too old for its layer, its ability to compete with rising genomes is weakened. A genome g i from layer i may challenge a genome g i+1 in layer i+1: If fitness(g i ) > fitness (g i+1 ), or g i+1 is too old for its layer, g i dislodges g i+1 and takes its place. g i+1 now challenges a genome g i+2 in layer i+2… Otherwise, g i is discarded. i gigi i+1 g i+1

5. Early Stopping t=0 t=500 t=1000 t=0 t=20 Problem: A robot that performs poorly in the first environment is unlikely to perform well in the second environment, but will be evaluated regardless in environments 2, …, k and then fail to produce offspring. Solution: Early Stopping Rather than summing the fitness scores of a genome, subtract fitness scores from a theoretical maximum.

6. Early Stopping Problem: A robot that performs poorly in the first environment is unlikely to perform well in the second environment, but will be evaluated regardless in environments 2, …, k and then fail to produce offspring. Solution: Early Stopping Rather than summing the fitness scores of a genome, subtract fitness scores from a theoretical maximum. Objective 1: Sensor 1 Objective 2: Sensor 2 Objective 1: Sensor 1 Objective 2: Sensor 2

7. Early Stopping Problem: A robot that performs poorly in the first environment is unlikely to perform well in the second environment, but will be evaluated regardless in environments 2, …, k and then fail to produce offspring. Solution: Early Stopping Rather than summing the fitness scores of a genome, subtract fitness scores from a theoretical maximum. Objective 1: Sensor 1 Objective 2: Sensor 2 Objective 1: Sensor 1 Objective 2: Sensor 2

8. Early Stopping Success: A robot achieves a sufficiently high value for each of its proximity sensors Objective 1: Sensor 1 Objective 2: Sensor 2 Objective 1: Sensor 1 Objective 2: Sensor 2

9. Results: Five Sets of Experiments ALPS = 0, Early Stopping = 0(100 independent runs) ALPS = 0, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 0(100 independent runs) ALPS = 1, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 1, Compression = 1 (100 independent runs) 1 gigi 2 gigi

10. Results: Five Sets of Experiments ALPS = 0, Early Stopping = 0(100 independent runs) ALPS = 0, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 0(100 independent runs) ALPS = 1, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 1, Compression = 1 (100 independent runs) 1 gigi 2 gigi

11. Results: Five Sets of Experiments ALPS = 0, Early Stopping = 0(100 independent runs) ALPS = 0, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 0(100 independent runs) ALPS = 1, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 1, Compression = 1 (100 independent runs) 1 gigi 1 g i+1 1 gigi 2

12. Results: Five Sets of Experiments ALPS = 0, Early Stopping = 0(100 independent runs) ALPS = 0, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 0(100 independent runs) ALPS = 1, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 1, Compression = 1 (100 independent runs) 1 gigi 1 g i+1 1 gigi 2

13. Results: Five Sets of Experiments ALPS = 0, Early Stopping = 0(100 independent runs) ALPS = 0, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 0(100 independent runs) ALPS = 1, Early Stopping = 1(100 independent runs) ALPS = 1, Early Stopping = 1, Compression = 1 (100 independent runs) 2 g i+1 gigi 1 gigi

14. Results