CSCTR Session 11 Dana Retová

 Start bottom-up  Create cognition based on sensori-motor interaction ◦ Cohen et al. (1996) – Building a baby ◦ Cohen & Lederle (1998) - Dynamic maps as representations of verbs

 agent acquiring conceptual knowledge by sensorimotor interaction with its environment  Uses notion of image-schemas  Goal ◦ Produce conceptual structures from sensorimotor interactions ◦ Show that very little prior structure is sufficient do to this (nature vs. nurture)

 Pattern detectors or filters that map sensory streams onto redescriptions or partial representations. ◦ E.g. ANIMATE MOTION

 Simulated agent ◦ Lives in simulated environment – BabyWorld  Implements Neo’s sensations, mental representations, mental and physical activities nad the behavior of objects. 1.Neo – implements everything that Neo does  Learning, moving, mouthing, looking, crying,… 2.StreamsWorld – represents Neo’s environment, implements events that happen around Neo and in response to Neo’s actions ◦ Neo senses its environment through a collection of streams in discrete time steps.

 Streams ◦ Affect, pain, hunger, somatic, haptic, visual, auditory,…  Simple and probabilistic  Only random actions ◦ E.g. If Neo’s eyes alight on a rattle then Neo will grasp the rattle with some probability  At the beginning experience has no apparent structure

1. Changes in token values: Tokens in streams are augmented by noticing when they change value. 2. Scopes: Neo finds pairs of correlated streams called scopes. 3. Base fluents: Neo finds common token-value pairs within scopes. 4. Context fluents: Neo finds base fluents that tend to follow each other in time. 5. Chains: These temporal dependencies are combined into temporal chains, which represent activities. Chains are used for activity-based categorization.

 Agent was able to learn basic concepts ◦ It learned base fluents corresponding to the shape and color of most objects in its envirnoment ◦ It learned activities  Grasping, mouthing, movintgits arm, seeing its arm move

 Classes of objects are differentiated by how we interact with them,  Concepts are abstractions over those classes  Meaning of concepts are in large part predictive models of how interactions with objects will unfold.  In the interactionist view, category distinctions are based on activity

Six lessons from babies (Smith & Gasser. 2005)

 It is claimed, that systems should be build incrementally. (not by selecting random x samples for training, and n-x for evaluation). Can you think of problems, where this does not work?[Martin]  It is said that it seems to be counter intuitive that children fail to locate an object in a room if the model that should help with this task is too similar to the room. Isn’t this just because a very similar model has to be very complex and is therefore harder to understand?[Thomas]

 How could multimodal associations be established in an infant? How does a baby know to look in the direction of the radio when it hears a sound? (In case of a moving car, this would be easy, but if there are no obvious visual clues?) How does it know to look in the direction of its parents when they are talking (prior to it being able to focus well enough to perceive the mouth motions)? [Thomas]

 It is mentioned that babies start with "motor babbling", and then discover activities which trigger interesting events. Can you think of a way to better define what could be "interesting" for a baby?[Christian]  The time babies spend on an activity seems to depend on how much "fun" this activity is (examples in the paper are the mobile attached to a babies ankles, which jingles when the baby moves; and mothers in general, who smile if babies smile and coo if babies coo). In your opinion, what does this "fun" factor depend on? Can we define it?[Thomas]

 Why should we assume that solving the reaching task starts from random, non-goal directed actions? Isn’t it more plausible that, while the babies’ movement might be mostly random, the reaching itself is always goal directed but erroneous?[Simon]  How would you try to come up with the "algorithm" the baby uses in order to explore its world? What kind of strategy does it use? [Christian]

 Must a artificial system be equiped with language?[Martin]  Since a lot of interesting things must be learned by the baby before being able to understand language, or even talk, do you think it is a mistake to try and build language understanding systems, without trying to solve earlier developmental problems first? [Christian]

 Must it be social?[Martin]  It is said that the lesson to be learned from facial-imitation-behavior is that intelligent systems have to be raised in a social world. But isn’t the main function of this facial imitation to allow for empathy in human interaction? Is this therefore really something we should be especially concerned with at the moment? [Simon]

 It is said that it seems to be counter intuitive that children fail to locate an object in a room if the model that should help with this task is too similar to the room. Isn’t this just because a very similar model has to be very complex and is therefore harder to understand?[Simon]