1. Developmental Artificial Intelligence 1 er April 2014 Olivier.georgeon@liris.cnrs.fr http://www.oliviergeorgeon.com t oliviergeorgeon.com1/31

2. Outline Example – Demos with robots. Conclusion of the course. – Learning through experience. – Intermediary level of AI: semantic cognition. Exercise – Implement your self-programming agent (follow- up). oliviergeorgeon.com2/31

3. Robotics research oliviergeorgeon.com3/31 Bumper tactile sensor Panoramic camera Ground optic sensor http://liris.cnrs.fr/simon.gay/index.php?page=eirl&lang=en

4. Experimentation oliviergeorgeon.com4/31

5. Validation paradigm Behavioral analysis rather than performance measure Developmental Learning in AI Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems. Naively confusing perception and sensing (Crowley, 2014) Reinforcement Learning. POMDP, etc. “Constructivist Schema mechanisms” (Drescher, 1991). Theories Implementations Cybernetic control theory (Foerster, 1960) Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” “intrinsic motivation” (Oudeyer et al. 2007) Radical Interactionism, ECA (Georgeon et al., 2013) Symbolic Non-symbolic Learning by registering (Georgeon, 2014) Learning by experiencing perception-action inversion (Pfeifer & Scheier, 1994) Soar ACT-R CLARION Non-symbolic Machine Learning. Embodied AI. Kuipers et al., (1997) 5/28

6. The “environment” passes “symbols” to the agent as input. We encode the “semantics” of symbols in the agent. We implement a “reasoning engine”. (“symbolic” should not be mistaken with “discrete”) Symbolic AI 6/28

7. Validation paradigm Behavioral analysis rather than performance measure Developmental Learning in AI Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems. Naively confusing perception and sensing (Crowley, 2014) Reinforcement Learning. POMDP, etc. “Constructivist Schema mechanisms” (Drescher, 1991). Theories Implementations Cybernetic control theory (Foerster, 1960) Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” “intrinsic motivation” (Oudeyer et al. 2007) Radical Interactionism, ECA (Georgeon et al., 2013) Symbolic Non-symbolic Learning by registering (Georgeon, 2014) Learning by experiencing perception-action inversion (Pfeifer & Scheier, 1994) Soar ACT-R CLARION Non-symbolic Machine Learning. Embodied AI. Kuipers et al., (1997) 7/28

8. The “environment” passes “observations” to the agent as input. The relation state -> observation is “statistically” a surjection. We implement algorithms that assume that a given “state” induces a given “observation” (although partial and subject to noise). 1 2 3 4 4 2 Learning by registering 8/28

9. Validation paradigm Behavioral analysis rather than performance measure Developmental Learning in AI Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems. Naively confusing perception and sensing (Crowley, 2014) Reinforcement Learning. POMDP, etc. “Constructivist Schema mechanisms” (Drescher, 1991). Theories Implementations Cybernetic control theory (Foerster, 1960) Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” “intrinsic motivation” (Oudeyer et al. 2007) Radical Interactionism, ECA (Georgeon et al., 2013) Symbolic Non-symbolic Learning by registering (Georgeon, 2014) Learning by experiencing perception-action inversion (Pfeifer & Scheier, 1994) Soar ACT-R CLARION Non-symbolic Machine Learning. Embodied AI. Kuipers et al., (1997) 9/28

10. Apprentissage par l’observation (Georgeon, 2014) Apprentissage par l’expérience Positionnement dans le cadre de l’IA Newell & Simon (1972) goals drive problem solving; (1976) Physical Symbol Systems. En confondant naivement “input” et “perception” (Crowley, 2014) Reinforcement Learning. “intrinsic motivation” (Oudeyer et al. 2007) Symbolic Non-symbolic Apprentissage « désincarnée » Cognition située (Clancey 1992) 10/28 Neural networks. Machine learning. A* etc. Validation paradigm Behavioral analysis rather than performance measure Theories Implementations Horde (Sutton et al., 2011) SMC theory (O’Regan & Noë, 2001) Phenomenology (Dreyfus, 2007) “even more Heideggarian AI” Radical Interactionism, ECA (Georgeon et al., 2013) perception-action inversion (Pfeifer & Scheier, 1994)

11. Change blindness oliviergeorgeon.com http://nivea.psycho.univ-paris5.fr/ 11/23

12. Time The environment passes the “result” of an “experiment” initiated by the agent. This is counter intuitive ! We implement algorithms that learn to “master the laws of sensorimotor contingencies” (O’Regan & Noë, 2001). Learning by experiencing 12/28

13. Accept the counter-intuitiveness We have the impression that the sun revolves around the earth. – False impression! (Copernic, 1519) We have the impression to receive input data about the state of the world. – False impression! (Philosophy of knowledge since the enlightenments, at least). – How to translate this counter-intuitiveness into the algorithms? oliviergeorgeon.com13/23

14. The stakes: semantic cognition Stimumuls-response adaptation Reasoning and language Semantic cognition Reinforcement-learning, neural nets, traditional machine learning. Rule-based systems, Ontologies, traditional AI. Knowledge-grounding, sense-making, Self-programming. 14/9oliviergeorgeon.com

15. Conclusion Think in terms of interactions – Rather than separating perception et action. Think in terms of generated behaviors – Rather than in terms of learned data. Keep your critical thinking – Invent new approaches ! http://e-ernest.blogspot.fr/15/21

16. Invent new approaches “Hard problem of AI” Formalized problem Etc. http://e-ernest.blogspot.fr/16/21 A E A E

17. Exercice oliviergeorgeon.com Part 4. 17/31

18. Environnement 3 - modified Behave like Environment 0 up to cycle 5, then like environment 1 up to cycle 10, then like environment 0. Implementation – If (step 10) If (experiment = e1) then result = r1 If (experiment = e2) then result = r2 – Else If (experiment = e1) then result = r2 If (experiment = e2) then result = r1 – Step++ oliviergeorgeon.com18/23

19. Agent 3 in Environment 3 oliviergeorgeon.com 0. e1r1,-1,0 1. e1r1,-1,0 learn (e1r1e1r1),-2,1 activated (e1r1e1r1),-2,1 propose e1,-1 2. e2r2,1,0 learn (e1r1e2r2),0,1 3. e2r2,1,0 learn (e2r2e2r2),2,1 activated (e2r2e2r2),2,1 propose e2,1 4. e2r2,1,0 activated (e2r2e2r2),2,2 propose e2,2 5. e2r1,-1,0 learn (e2r2e2r1),0,1 6. e2r1,-1,0 learn (e2r1e2r1),-2,1 activated (e2r1e2r1),-2,1 propose e2,-1 7. e1r2,1,0 learn (e2r1e1r2),0,1 8. e1r2,1,0 learn (e1r2e1r2),2,1 activated (e1r2e1r2),2,1 propose e1,1 9. e1r2,1,0 activated (e1r2e1r2),2,2 propose e1,2 10. e1r1,-1,0 learn (e1r2e1r1),0,1 activated (e1r1e2r2),0,1 activated (e1r1e1r1),-2,1 propose e2,1 propose e1,-1 11. e2r2,1,0 activated (e2r2e2r1),0,1 activated (e2r2e2r2),2,2 propose e2,1 12. e2r2,1,0 activated (e2r2e2r1),0,1 activated (e2r2e2r2),2,3 propose e2,2 13. e2r2,1,0 Environnement 0Environnement 1Environnement 0 19/23

20. Time Activated i12 Propose … i12 i t-3 i t-2 i t-4 i t-1 i t = i11 i11 PRESENT FUTUREPAST Learn AGENT itit (i t-1,i t ) Activate i t-1 oliviergeorgeon.com20/31 Principle of Agent 3 (i11,i11) i11 e1 Choose Execute (i12,i12)

21. Environment 4 Returns result r2 only after twice the same experience. e1 -> r1, e1 -> r2, e1 -> r1, e1-> r1, … e1->r1, e2 - > r1, e2->r2, e2 -> r1, …, e2 -> r1, e1 -> r1, e1 -> r2, e2 -> r1, e2 -> r2, e1 -> r1, e1 -> r2, … If (experience t-2 !=experience t && experience t-1 ==experience t ) result = r2; else result = r1; oliviergeorgeon.com21/33

22. Rapport Agent 1 – Explanation of the code. – Traces in environments 0 and 1 with different motivations. – Explanation of the behaviors. Agent 2 – Explanation of the code. – Traces in environments 0 to 2 with different motivations. – Explanation of the behaviors. Agent 3 – Explanation of the code. – Traces in environments 0 to 4 with différent motivations. – Explanation of the behaviors. Conclusion – What would be the next step towards Agent 4 able to adapt to Environments 1 to 4? oliviergeorgeon.com22/23