1. Scenario Specification and Problem Finding
Luca Tummolini
ISTC-CNR
MindRACES, First Review Meeting, Lund, 11/01/2006

2. MindRACES, First Review Meeting, Lund, 11/01/2006
Objective
From the Annex 1: “identify 6 scenarios corresponding to the situation in which cognition requires anticipation and implement 3 of them”
Scenarios are used for:
Understanding the role of anticipation in cognitive systems
Identify the challenges
Facilitate future integration
MindRACES, First Review Meeting, Lund, 11/01/2006

3. MindRACES, First Review Meeting, Lund, 11/01/2006
The scenarios
Finding and Looking for
IDSIA, IST, ISTC-CNR, NBU, UW-COGSCI
Anticipation in a dynamic world
LUCS, OFAI, UW-COGSCI
Guards and Thieves
ISTC-CNR, LUCS, NBU, OFAI
The scenarios highlight the role of anticipation in:
Perception and categorization
Selective attention
Deliberation
Action monitoring and control
Skill learning and routinization
Coordination
Emotions
MindRACES, First Review Meeting, Lund, 11/01/2006

4. Scenario 1: Finding and Looking for
Partners involved: IDSIA, IST, ISTC-CNR, NBU, UW-COGSCI
Two different assumptions:
the agent has a model of the environment
the agent learns the model of the environment
The agent:
a pan-and-tilt camera mounted on a fixed base
a simulated fovea (higher resolution at the centre) on a mobile robot
a robotic arm
MindRACES, First Review Meeting, Lund, 11/01/2006

5. Scenario 1: Finding and Looking for
Task 1: Finding a specific object
Find and reach a specific object in the environment (e.g. a green circle).
This task involves:
Selective attention
Anticipation based on memory
Goal-based motor-primitives
Detection of regularities in the spatial relations among objects
1. Aggiungere le tre immagini di IDSIA
MindRACES, First Review Meeting, Lund, 11/01/2006

6. Scenario 1: Finding and Looking for
Task 2: Finding members of a class of objects
Find an object transferring a system of relations from one domain to another
This task involves:
Anticipation by analogy
Detection of relations among objects
Relation extraction and encoding
Categorization of observed objects
MindRACES, First Review Meeting, Lund, 11/01/2006

7. Scenario 1: Finding and Looking for
Task 3: Looking for an object
Look for an object that is hidden in one of the rooms in a believable way
The target location can be probabilistically biased towards certain locations
This task involves:
Prediction by analogy making
Decision making about the rooms to visit
Building models of the world structure and of the regularities between objects
Emotional responses to expected and unexpected events
1. Inserire immagine di boicho
MindRACES, First Review Meeting, Lund, 11/01/2006

8. Scenario 2: Anticipation in a Dynamic World
Partners involved: LUCS, OFAI, UW-COGSCI
This scenario involves prediction of objects with an intrinsic dynamics
The first three tasks involve looking at two games with moving objects and the task is to predict the movement of one or several targets
The last task deals with the prediction of a rolling ball from a developmental perspective
MindRACES, First Review Meeting, Lund, 11/01/2006

9. Scenario 2: Anticipation in a Dynamic World
Task 1: The fish catching game
Goal: anticipate location and velocity of some targets in the scene
The movement of the targets is very regular
Prediction from different point of views
This task involves:
Selective attention
Target tracking based on anticipatory model
Tracking in a fixed environment
Quick relearning of the model when the viewing angle changes
1. Aggiungere immagini del trenino di LUCS, cambiare titolo???
MindRACES, First Review Meeting, Lund, 11/01/2006

10. Scenario 2: Anticipation in a Dynamic World
Task 2: Marble run game
Goal: anticipate location and velocity of some targets in the scene
Paths can be re-arranged
Combination of the continuous dynamics of the ball with a compositional structure
Generalization between different configurations of the set-up
This task involves
Selective attention
Target tracking based on anticipatory model
Tracking of partially occluded and moving targets
MindRACES, First Review Meeting, Lund, 11/01/2006

11. Scenario 2: Anticipation in a Dynamic World
Task 3: Learning previous two games at the same time
Context sensitive learning
Priming effects biasing the predictions
Detection of relevant context
This task involves:
Generalization of target model from one viewing condition to another and between the different games
Context dependent selection of target, predictive models and tracking strategy
Simultaneous modelling, recognition, prediction and estimation of viewer’s position
MindRACES, First Review Meeting, Lund, 11/01/2006

12. Scenario 2: Anticipation in a Dynamic World
Task 4: Basic how-to knowledge
Initial basic interactions with the environment driven by basic instincts and motivations
The robot learns through reinforcement which interactions are allowed by the objects in the environment
This task involves:
The acquisition of object generalisations and concepts
Learning of basic dynamical properties of objects (e.g. predict where a ball will reappear behind an obstacle)
MindRACES, First Review Meeting, Lund, 11/01/2006

13. Scenario 2: Anticipation in a Dynamic World
Task 5: Generalisation
If the ball moves slowly, the robot learns to look for the ball on the right side of the wall
If the ball moves fast, the robot learns to look for the ball on the other end of the wall
If the ball produces a noise, the robot goes to look for it behind the wall
This task involves:
The development of more sophisticated concepts, such as object persistence
Flexible how-to knowledge
MindRACES, First Review Meeting, Lund, 11/01/2006

14. Scenario 3: Guards and Thieves
Partners involved: ISTC-CNR, LUCS, NBU, OFAI
Agents can have two different roles (guards or thieves)
Thieves try to steal valuable resources
Guards protect the valuables restricting access to them (e.g. blocking the entrance)
In some variants of the task, only one of the two roles will be played by an anticipatory cognitive system
In the more complex variants, both roles will be played by anticipatory agents
1. Aggiungere le immagini dello scenario di Lucs che presenta
MindRACES, First Review Meeting, Lund, 11/01/2006

15. Scenario 3: Guards and Thieves
Task 1: Conflict in accessing the valuables - simple
Two agents: one thief and one guard
The valuables are hidden in at least two different places
The session ends either when the thief has collected or found all the valuables or when the guard has blocked the thief.
This task involves:
Recognition of the adversary among the moving objects
Prediction of the adversary behaviour (avoiding/intercepting)
Integrating different levels of action control (e.g. routinary, reasoning), based on different kinds of expectations (e.g. implicit, explicit)
MindRACES, First Review Meeting, Lund, 11/01/2006

16. Scenario 3: Guards and Thieves
Task 2: Conflict in the access to valuables - complex
A social task involving several agents – several thieves and a guard
The session ends either when all the valuables have been collected or found or when the guard has blocked all the thieves
The thieves should be able to distinguish between guards (danger) and fellows (cooperation)
This task involves:
Help and critical help by anticipating other’s needs, actions or capabilities, e.g. by removing obstacles or doing part of other’s work
Delegating by trusting; e.g. an agent can explicitly “ask another one for help”
MindRACES, First Review Meeting, Lund, 11/01/2006

17. Thank you for the attention
MindRACES, First Review Meeting, Lund, 11/01/2006