1. Project ArteSImit Artefact Structural Learning through Imitation (TU München, U Parma, U Tübingen, U Minho, KU Nijmegen) Giorgio Panin - TUM

2. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 2 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Artesimit Project Partners Alois Knoll and colleagues (project coordinator, Robotics, Munich) Giacomo Rizzolatti and colleagues (Neurophysiology, Parma) Peter Thier and colleagues (Cognitive Neuroscience, Tübingen) Wolfram Erlhagen and colleagues (Cognitive Modeling, Minho) Harold Bekkering and colleagues (Cognitive Neuropsychology, Nijmegen)

3. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 3 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Central issue of imitation: do we imitate movements (means) or consequences (goals) of the observed behavior? Movements somehow need to be directly mapped (via-point approach), whereas goals need to be represented. Movement mapping assumes homologue bodies, full access of the whole movement and stable environments. Goal representation assumes an association between an (observed) action and the (observed) consequence. The concept of goal-directed Imitation

4. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 4 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 How to work together? Behavioral measurements (healthy and clinical patients) Brain research (single-cell, fMRI, MEG, EEG) Modeling (Dynamic neural fields) Robotics (Visual analysis, cognition, and artefact control) THE solution: One common paradigm

5. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 5 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Common paradigm Starting position Movement 1 Movement 2 A LED gives information about either the goal or the means

6. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 6 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Motor responses of parietal neurons Parietal Mirror Neuron findings Observation task Visual responses of parietal mirror neurons Motor task

7. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 7 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Architecture of the dynamic model PFC: Goal representations Task input Object properties (e.g. colour) STS: Visual desciption of grip and trajectory PF: Sequence of means F5: Movement primitives Bridge Paradigm

8. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 8 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Visual system in action: Task recognition Complete information acquisition for the decision and planning tasks

9. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 9 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Learning I: Association between object properties and goal

10. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 10 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Learning II: Copying the means of the teacher

11. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 11 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Goal-directed imitation

12. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 12 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Partial information: Cueing of the goal

13. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 13 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Partial information: Cueing of the means

14. __________________________________________________________________________________________________________________________________________________________________________________ Project funded by the Future and Emerging Technologies arm of the IST Programme 14 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689 Summary The project combined... the development of a new paradigm for goal-directed imitation with imaging studies across all modalities new findings in experimental neurophysiology and advanced neuromodelling of the structure of the PFC-(STS-PF-F5)-coupling to develop and implement computer- operational models for goal-directed imitation the development of a powerful model-based hand posture recogniser (using only one B/W camera) into... A complete artefact (robot system) capable of mimicking imitation behaviour of creatures: all project goals have been met!