1. Approaches to Artificial Evolution Verena Hamburger

2. Problems in engineering Simple designs are often not effective Highly sophisticated morphologies are hard to model Each motor action has various effects Much effort to make sensors highly precise measuring devices A lot of a priori and designer knowledge is needed

3. Benefits of evolutionary robotics (1) Overcoming the stiff human concepts => Truly new designs & innovative forms of sensory-motor-coordination Sensory-motor-systems acts as a whole in close coupling with the environment => An essential aspect of real cognition

4. Benefits of evolutionary robotics (2) Reduction of modelling process => Emphasis on behaviour analysis More, but simpler sensors => Non-linearities are exploited or amended => Sensors are combine to extract useful information

5. Open-ended evolution An evolutionary process that leads to the ongoing development of new traits that tend – to be retained for long evolutionary periods and – to constitute important building blocks for further evolutionary stages.

6. Requirements to open-ended evolution (1) Requirements according to Bianco and Nolfi: ● implicit and general selection criteria, but BII fitness functions may cause a “bootstrap problem”. ● favourable organisation of the evolving individuals, a good genotype-to-phenotype mapping is expressive, compact, autonomous & complete ● dynamically changing environmental conditions

7. Requirements to open-ended evolution (2) Requirements according to Maley: ● Endogenous implementation of ecological niches ● Unbounded diversity during growth phase ● Selection must be embodied ● System must exhibit continuing (“positive”) new adaptive activity

8. Requirements to open-ended evolution (3) Requirements according to Channon: ● Evolutionary emergence: The constant need to change one's model to keep up with a system’s behaviour. ● Natural selection: evolution as a result of the system dynamics is a prerequisite for evolutionary emergence (also Packard, Ray). ● Non-linear systems which do not obey the superposition principle (also Langton).

9. Stages of natural evolution 1. The creature's body plan changes 2. New sensors and actuators are explored 3. The environment changes (at least as the organisms perceives it) 4. The nervous system adapts.

10. State of the Art Evolution of complete agents are still quite restricted to: – single tasks – few basic shapes – limited variety of sensors, actuators and materials Some methods require a lot of human interference.

11. Modular evolution ● A simple body plan with two legs ● Robot drags itself along the ground ● Development of multi-jointed legs ●..... ● Quadruped with stable locomotion => Stages are induced by the designer (Muthuraman, MacLeod and Maxwell)

12. Evolving Neural Networks Support of various learning techniques Resistant to noise Biologically inspired Real values for in- and output Graceful degradation Low level Evtl. flexible neuron model Recurrent NNs (internal state, rich intrinsic dynamics)

13. TPE learning in PNN (Hebb, post-/pre- synaptic, covariance) More complex skills (also sequential tasks) Fewer generations Adapting to environments never seen during evolution (even different morphology) Controllers transferred to reality quickly (Floreano & Urzelai) Evolution of learning

14. LAE by (Punctuated) Anytime learning Algorithm receiving information from the real world to adapt off-board simulator or GA. Best solution is periodically sent to controller. Parker and others: Punctuated anytime learning with “cyclic GA” for hexapod gaits: Incrementally evolving individual leg controllers Adapting to a different environments Evolving a simulated team of legged robots.

15. Importance of morphology Locomotion for ten legged agents with different body plans (Bongard and Pfeifer) => Shape and mass determine fast/slow/no success => Build up parameter directory Closed loop controller for stable biped walking (Paul and Bongard) => The more control over the weight distribution, the more stable gaits

16. Importance of environment Fast locomotion for Sony Aibo: TPE on flat carpet did not generalise well to new surfaces TPE on uneven surface generalised well but: it was tricky to get the level of unevenness right (Hornby, Fujita, Takamura, Yamamoto, Hanagata)

17. Evolution with meta-model Partial order via classification of genotype Classifier is evolved online Additional regular fitness based evaluation => Reduction of evaluation (~50%) => Less deterioration of the robot => False classification may lead to loss of best individual (Jens Ziegler: GP for fast locomotion of Sony Aibo)

18. Variation of evolutionary parameters (1) The performance of GA is sensitive to initial conditions but: no quantitative methods are available => lower risk through parameter variation

19. Variation of evolutionary parameters (2) Classification, regression, robot control Comparing fitness expectation and variance after 200 generations Wilcoxon-Test (confidence niveau 0.95) => deterministic or adaptive variation with mutation rate ↑ - crossover rate ↓ (Jens Ziegler)

20. Architectures of variable complexity (1) Fixed architectures are unsuitable since complexity cannot be foreseen (fixation = limitation) Open-ended evolution gets interesting when problem-oriented evolution reaches equilibrium At equilibrium all individuals are similar => only mutation can bring forth new traits

21. Architectures of variable complexity (2) Mutation rate tied to whole genotype or single gene is both inappropriate => Mutation-lock of beneficial genes Mutation-locked genes evtl. still suboptimal => Mutation-lock immunises against major, but not against minor mutation

22. My personal idea We need a methodology for automatic modular (co-)evolution of learning in architectures of variable complexity – Mutation-lock – Parameter variation in individual stages – Evtl. meta-model for real world experiment in dynamically changing environment