1. Spacetime Constraints Revisited Joe Marks J. Thomas Ngo Using genetic algorithms to find solutions to spacetime constraint problems in 2D

2. Difficulties with SC problems Multimodality –exponential number of possible solutions Search space discontinuities –small changes in actuators leads to large changes in trajectory

3. Why Genetic Algorithms? Traditional SC algorithms –Initial trajectory needed to work on Initial trajectory is done by human hand - “coarse” Initial trajectory is locally optimized Genetic Algorithms –Discovers solutions “from scratch” –Cover global optimization problem –Can find “novel” solutions that were not imagined

4. Overview of GA in SC problems All trajectories are described as behaviors Behaviors generated by parametric algorithms (nothing smart here) Evolutionary computation chooses behavior parameters to find optimal solutions –can find multiple effective solutions Better solutions (genomes) bred further Worse solutions bred-out of gene pool

5. GA algorithm for 2D simulation (Ngo and Marks) Dynamics module –simulates a physics driven world for testing creatures Behavior module –generates behavior using parameterized algorithm Search module –uses GA to choose values for parameters –generates near-optimal solutions

6. Dynamics Module Most CPU intensive part of this algorithm Simulating simplified physics –Based on work by Hahn –rate of change of internal degrees of creature controlled, rather than calculating torques –friction and slippage simulated upon contact

7. Behavior Module Behaviors generated by stimulus-response (SR) control algorithm –causes instinctive reflexes to conditions conditions are stimulus functions over senses reflexes are responses to conditions –avoids traditional use of forces –behaviors selected in search module –no learning or planning algorithms involved

8. Stimulus functions Scalar functions based on sense variables –state of joint angles –force between body endpoints and floor –vertical velocity of center of mass –height from floor of center of mass Contain parameters that are determined by GA-based search module Stimulus functions exhibit sensitive regions –locus of points for which function is positive –important notion during mutation

9. Response Change in shape in reaction to a condition –Conducted on highest valued positive stimulus function –Change of creature’s actual shape to a target shape –Change kept smooth by damped motion equations Target shape may change during a response –Creature must respond to another stimulus Response is active for several time steps

10. Search Module Use of GA to pick near optimal behaviors Parallel processing of solutions –each processor handles one genome solution per generation do parallel Randomize genome end do for generation = 1 to number_of_generations do parallel Evaluate genome Select mate from another processor Cross genome with mate Mutate genome end do end for

11. Randomization Initial parameters chosen at random Favorable initial gene pool formed –hill climbing algorithm used to find good initial solutions to populate gene pool –each solution mutated, re-evaluated four times mutation skewed towards multi-step solutions –best of five solutions on each processor used final population is non-random, skewed

12. Evaluation One of the most important aspects of GA Net horizontal distance covered by center of mass in given time –Sometimes this encourages “cheating” –e.g., leap head-first rather than walking To encourage walking, modify criteria –e.g., net horizontal distance covered by midpoint between two feet - i.e., use your feet!

13. Mate Selection Only local mating permitted –maintains diversity, handles multimodality –mate chosen on a random walk of 10 steps if better solution than self found, mate is chosen otherwise, current solution stays single –produces large local colonies of good genes larger colonies use up more processing power Convergence - when one colony dominates

14. Crossover Linear crossover not seen as meaningful –certain parameters should migrate as groups More structured, specialized crossover used –two (of ten) SR pairs taken from self –six (of ten) SR pairs taken from mate –one SR pair created with stimuli and response taken from each parent, respectively –one SR pair created with numbers taken by random from parents

15. Mutation Mutation algorithm tailored for SR method One SR pair in genome is subject to creep –each parameter in pair is slightly changed One SR pair is randomized from scratch –but one corner of sensitive regions of new stimulus function coincides with original sense- space trajectory keeps new functions from either dominating or not having any effect.

16. Results Hardware, software used –4,096 processor Thinking Machines CM-2 –Code written in C* Creatures used –5 rod figures; 50 time steps; 100 generations Five-rod Fred Mr. Star-man Beryl Biped –importance of adapting criteria 30 to 60 minutes of computation used