Developmental Artificial Intelligence 27 March t oliviergeorgeon.com1/29

Outline Organization in time and space. – Formalism for Spatio-temporal coupling. – Cognitive architecture. – Demonstrations Exercise – Implement your self-programming agent (follow up). oliviergeorgeon.com2/29

Reminder of theoretical ideas The objective is to learn (discover, organize and exploit) regularities of interaction in time and space to satisfy innate criteria (survival, curiosity, etc.). To autonomously construct an ontology of reality. – From experience of interaction Detect and respond to increasingly sophisticated affordances – (self-programming). oliviergeorgeon.com3/29

Propose Trace-Based Reasoning Time Hierarchical abstraction Matching Select Try to Enact More abstraction oliviergeorgeon.com4/29

Example 2 oliviergeorgeon.com5/29

Examples of learned behaviors oliviergeorgeon.com6/29

Example 2 oliviergeorgeon.com7/29

Spatio-sequential regularity learning oliviergeorgeon.com8/29

Concept of affordance Property of an object or an environment that allows an individual to perform an action. “To afford” an action. « objects push us into doing » (Heinz Werner) – Affordances are properties of the coupling between the agent and the environment. We know the world in terms of possibilities of interaction. oliviergeorgeon.com9/29

Formalisme Agent Environment ObservationAction Traditionnal formalization  O O  A A Agent Environment ExpérienceRésultat Learning by experiencing  R R  E E oliviergeorgeon.com10/29 Agent Environment Intended Interaction Enacted Interaction  I I  I I Radical Interactionism: I = E ✕ R X: Interactions composites

How to learn the existence of possibly persistent entities in the environment? How to adapt to different categories of entities? Spatial Radical Interactionism Agent Environment Intended Interaction i  I Enacted Interaction e  I oliviergeorgeon.com11/29 Spatial position σ Spatial transformation τ τ represents the vestibular system. Can be implemented through an accelerometer. σ represents position information (eye convergence, interaural phase difference, etc.)

Spatial exemple 1 oliviergeorgeon.com Agent Environment Enacted interaction:…….. Spatial position σ = (1,0) Spatial transformation τ= (0,0) Intended interaction 12/29

Agent Environment Environment “known” at time t d e cd  X d i cd  X d e p1 i p1 i pj  Ie pj  I Decisional mechanism Recursivity problem How to maintain recursivity ? oliviergeorgeon.com Spatial position σ Spatial transformation τ 13/29

Spatial Exemple 1 oliviergeorgeon.com14/29

Enactive Cognitive Architecture -The agent programs itself through experience of interaction. -The architecture does not program itself. (“Kantian space”, e.g., Buzsaki 2013, Space, time, and memory) oliviergeorgeon.com15/29

Inspiration from neurosciences oliviergeorgeon.com16/29

More inspiration from neurosciences? oliviergeorgeon.com Cotterill R. (2001). Progress in Neurobiology. 17/29

ECA agent exemple 2 oliviergeorgeon.com18/29

From “drives” to “goals” Afforded Simulated oliviergeorgeon.com19/29

Exercise 3 oliviergeorgeon.com20/29

Exercise Two possible experiences E = {e 1,e 2 } Two possible results R = {r 1,r 2 } Four possible interactions E x R = {i 11, i 12, i 21, i 22 } Two environments – environment 0 : e 1 -> r 1, e 2 -> r 2 (i 12 et i 21 are never enacted) – Environment 1 : e 1 -> r 2, e 2 -> r 1 (i 11 et i 22 are never enacted) – Environment 2 : e 1 -> r 2, e 2 -> r 2 – Environment 3 Motivational systems: – motivation 0 : v(i 12 ) = v(i 22 ) = 1, v(i 11 ) = v(i 21 ) = -1 – motivation 1 : etc. Implement un agent that learn to enact positive interactions without knowing its motivatins and its environnement a priori. Write a rapport of behavioral analysis based on activity traces. oliviergeorgeon.com21/29

Implémentation public static Experience e1 = new experience(); Experience e2 = new experience(); public static Result r1 = new result(); Result r2 = new result(); public static Interaction i11 = new Interaction(e1,r1, 1); etc. Public static void main() Agent agent = new Agent3(); // Agent1(); Agent2(); Environnement env = new Env3(); // Env1(); // Env2(); for(int i=0 ; i < 10 ; i++) e = agent.chooseExperience(r); r = env.giveResult(e); System.out.println(e, r, value); Class Agent Class Agent3 public Experience chooseExperience(Result r) Class Environnement Class Env3 public Result giveResult(experience e) Class Experience Class Result Class Interaction(experience, result, value) public int getValue() oliviergeorgeon.com22/29

Analyse de traces d’activité. oliviergeorgeon.com e1,r2,-1 e2,r1,1 e1,r1,1 Motivation1, Environnement 0. Motivation1, Environnement 1. Motivation1, Environnement 2. e1,r2,1 learn e1r1-e1r2,0 e1,r1,-1 learn e1r2-e1r1,0 e1,r1,-1 learn e1r1-e1r1,-2 e2,r2,1 learn e1r1-e2r2,0 e2,r1,-1 learn e2r2-e2r1,0 e2,r1,-1 learn e2r1-e2r1,-2 e1,r2,1 learn e2r1-e1r2,0 e2,r2,1 learn e1r2-e2r2,2 e1,r2,1 learn e2r2-e1r2,2 e2,r2,1 23/29

Environnement 3 Behaves like Environnement0 during the first 10 cycles, then like environnement1. Implémentation – If (step < 10) If (experiment = e1) then result = r1 If (experiment = e2) then result = r2 – Else If (experiment = e1) then result = r2 If (experiment = e2) then result = r1 – Step++ oliviergeorgeon.com24/29

Temps Activated i11 Propose … i11 i t-3 i t-2 i t-4 i t-1 i t = i11 i11 PRESENT FUTURPAST learn AGENT itit (i t-1,i t ) Activate i t-1 oliviergeorgeon.com Principle of Agent 3 (i11,i12) i12 e1 Choose Execute (i11i11) 25/29

Implementation of Agent 2 At the end of time step t – Record or reinforce i c =  i t-1, i t, w  =  pre(i c ), post(i c ), weight  If i c already belongs to the set of existing interactions I t – Weight ++. At the beginning of time step t – Construct the list of activated composite interactions A t A t = { i  I t | pre(i) = i t-1 } – For each activated composite interaction in A t Create a proposition for post(ic).experience with proclivity ic.weight * post(ic).valence For each experience, sum up the proclivity of all its propositions. Choose the experience that has the highest total proclivity. oliviergeorgeon.com26/29

Class Interaction //attributes: Experience experience; Result result; int value; String label; Interaction preInteraction; Interaction postInteraction; int weight; oliviergeorgeon.com PostInteractoin Composite Interaction weight preInteraction 27/29

Decision mechanism oliviergeorgeon.com List propositions = new ArrayList (); for (Interaction activatedInteraction : getActivatedInteractions()){ Proposition proposition = new Proposition( activatedInteraction.getPostInteraction().getExperience(), activatedInteraction.getWeight() * activatedInteraction.getPostInteraction().getValence()); int index = propositions.indexOf(proposition); if (index < 0) propositions.add(proposition); else propositions.get(index).addProclivity(activatedInteraction.getWeight() * activatedInteraction.getPostInteraction().getValence()); } Collections.sort(propositions); If (propositions.size()> 0) proposedExperience = propositions.get(0).getExperience(); 28/29

Class Proposition Class Proposition implements Comparable – // attributs: – Experience experience – Int proclivity – // constructor – Proposition(Experience experience, int Proclivity) – // methods – int compareTo(Proposition proposition) return new Integer(proposition.getProclivity()).compareTo(proclivity); – boolean equals(Object otherProposition) return ((Proposition)otherProposition).getExperience() == this.experience; – void addProclivity(int proclivity) this.proclivity += proclivity; oliviergeorgeon.com29/29