1. Com3240 Adaptive Intelligence Epigenetic Robotics Amanda Sharkey Reinforcement Learning Eleni Vasilaki

2. Epigenetic Robotics Interdisciplinary: developmental psychology and robotics Overlaps with artificial life, cognitive science, adaptive behaviour. Focus on “prolonged developmental process through which varied and complex cognitive and perceptual structures emerge as a result of the interaction of an embodied system with a physical and social environment”. (Zlatev and Balkenius, 2001)

3. 2001-9 Epigenetic Robotics conferences 2003: Connection Science special issue 2006: Interaction Studies special issue

4. Robotics learning from psychology – Models of cognitive development – Biologically inspired – Just inspired, or can it generate testable predictions? Psychology learning from robotics? – Building robotic models requires further specification and implementation of underlying mechanisms (synthetic modelling)

5. “true intelligence in natural and (possibly) artificial systems presupposes three crucial properties: (a) the embodiment of the system; (b) its situatedness in a physical and social environment; (c) a prolonged epigenetic developmental process through which increasingly more complex cognitive structures emerge in the system as a result of interactions with the physical and social environment.” Zlatev and Balkenius (2001 proceedings)

6. Brain, body and environment: reciprocally coupled. Contrast to earlier emphasis on ‘mind as computer’, where body is output device for mind that manipulates symbols representing the world, in rule-like way.

7. Progress in epigenetic robotics? Successive proceedings. “Epigenetic Robotics is a new discipline at the frontier of developmental psychology, neural-, and engineering sciences whose goal is to model the development of cognition in natural and artificial systems” Metta and Berthouze, (2006)

8. The word “epigenetic”was chosen here in antithesis to “phylogenetic”, to distinguish this work from the field of evolutionary approaches and artificial life, and rather stress the aspect of postnatal development. Metta and Berthouze (2006) Phylogenetic development vs ontogenetic

9. – Previous efforts to create intelligent machines Direct programming Supervised machine learning Evolutionary adaptation Developmental robotics – Autonomous learning – Greater emphasis on learning own sensor and action models – Epigenetic robotics

10. Weng et al (2001) Autonomous mental development in robots and animals. Science, 291, 599-600 The autonomous development paradigm for constructing developmental robots is as follows: · Design a body according to the robot's ecological working conditions (e.g., on land or under water). · Design a developmental program. · At "birth," the robot starts to run the developmental program. · To develop its mind, humans mentally "raise" the developmental robot by interacting with it in real time.

11. Relevant concepts Previous courses are relevant – John Searle (1980): Strong AI: an appropriately programmed computer really is a mind, can be said to understand, and has other cognitive states. Weak AI: a computer is a valuable tool for study of mind – makes it possible to formulate and test hypotheses rigorously (Kurzweil (2005) confusingly also uses term strong AI to refer to a “machine with the full range of human intelligence”)

12. Turing – how to achieve intelligence? Child machine – Some innate ability – Education (punishment and reward)

13. Some background: Jean Piaget Developmental psychologist: constructivist Baby and child’s mental structures develop as a result of adaptation to the environment Adaptation to the environment – Assimilation: assimilating the environment to own mental structures E.g. babies sucking objects Transforming the world to fit own mental structures – Accommodation: altering mental structures to fit with environment Mental structures changed by interactions with the environment E.g. learning to turn head, and finding nipple to suck – Equilibration When external reality does not match internal logical mental structures, more sophisticated mental structures are developed.

14. Stages of development 1. Sensorimotor development (birth-2 years) “ (i) simple reflexes; (sucking, grasping) (ii) first habits and primary circular reactions; (e.g. thumb sucking) (iii) secondary circular reactions; (repeat actions to see effect, e.g. grasping cord to ring bell) (iv) coordination of secondary circular reactions; (v) tertiary circular reactions, novelty, and curiosity; and (vi) internalization of schemes."

15. 2. Preoperational stage (2-7 yrs) – No logical thought 3. Concrete operational stage (7-12) – Logical thought with practical aids 4. Formal operational stage (12 onwards) – Abstract logical thought – Each stage is reached as a result of experiences at previous stage

16. Epigenetic Robotics Is it easy to define? What are its aims? What techniques does it use? What is there of value in it? What are the main references/people in the area? Is it progressing?

17. Assignment: 2500-4000 word essay: What are the aims of Epigenetic Robotics, and how likely are they to be achieved? Or What is Epigenetic Robotics, and how likely is it to make a significant contribution to Artificial Intelligence? Or Epigenetic Robotics, Developmental Robotics, Swarm Robotics, Behaviour-based Robotics, Evolutionary Robotics: what are the differences between them, and how useful is it to distinguish between them? or How easy, and also how useful, is it to distinguish Epigenetic Robotics from the following: Developmental Robotics, Swarm Robotics, Behaviour-based Robotics, Evolutionary Robotics. Assignment due: Tuesday April 13 th (week 7 semester2)

18. Strengths – Sets new (ambitious) challenges and goals for AI – Broad church – many approaches – New inspiration from a rich source (child development) New concepts e.g. Ongoing emergence, affordances Weaknesses? Overambitious? Can autonomous learning be achieved in artificial systems? – Sometimes goal described without saying how to achieve it – Sometimes focus is on method/technique not clearly related to ER Broad church... Lack of focus – Achievements? Practical? Theoretical? – How much leeway for new approach?

19. Marking scheme: Content – Is it well researched? Does it show good understanding of Epigenetic Robotics and associated issues? (aims, examples, criticism/reviews) Argument and organisation – Well structured? Spelling, grammar, references – Poor writing obscures what you want to say.