2009, Pr Fakhreddine Karray 1 Advances in Cognitive Robotics: Achievements and Challenges Fakhri Karray Pattern Analysis and Machine Intelligence Research.

Slides:



Advertisements
Similar presentations
Cognitive Systems, ICANN panel, Q1 What is machine intelligence, as beyond pattern matching, classification and prediction. What is machine intelligence,
Advertisements

Iñaki Merino Albaina MSc Program: Media & Knowledge Engineering Daily supervisors: drs. L.H.T.E. Yamane dr. ir. M.H. Vastenburg SCID group Faculty of Industrial.
ARCHITECTURES FOR ARTIFICIAL INTELLIGENCE SYSTEMS
Breakout session B questions. Research directions/areas Multi-modal perception cognition and interaction Learning, adaptation and imitation Design and.
15 th International Conference on Design Theory and Methodology 2-6 September 2003, Chicago, Illinois Intelligent Agents in Design Zbigniew Skolicki Tomasz.
Chapter Thirteen Conclusion: Where We Go From Here.
Integrating Educational Technology into the Curriculum
Ying Wang EDN 303 Fall Objectives Define curriculum-specific learning Explain the difference between computer, information, and integration literacy.
University of Minho School of Engineering Centre ALGORITMI Uma Escola a Reinventar o Futuro – Semana da Escola de Engenharia - 24 a 27 de Outubro de 2011.
Introduction to Robotics In the name of Allah. Introduction to Robotics o Leila Sharif o o Lecture #2: The Big.
Software Engineering Techniques for the Development of System of Systems Seminar of “Component Base Software Engineering” course By : Marzieh Khalouzadeh.
Agent Mediated Grid Services in e-Learning Chun Yan, Miao School of Computer Engineering Nanyang Technological University (NTU) Singapore April,
Matching brain and body dynamics Daniel Wolpert: – "Why don't plants have brains?" – "Plants don't have to move!" Early phases of embodied artificial intelligence:
Intelligence without Reason
Provisional draft 1 ICT Work Programme Challenge 2 Cognition, Interaction, Robotics NCP meeting 19 October 2006, Brussels Colette Maloney, PhD.
ISTD 2003, Thoughts and Emotions Interactive Systems Technical Design Seminar work: Thoughts & Emotions Saija Gronroos Mika Rautanen Juha Sunnari.
Marakas: Decision Support Systems, 2nd Edition © 2003, Prentice-Hall Chapter Chapter 7: Expert Systems and Artificial Intelligence Decision Support.
Robotics for Intelligent Environments
A.M. Florea, Cognitive systems, COST Action IC0801 – WG1, 15 December, Ayia Napa, Cyprus.
Meaningful Learning in an Information Age
RoboCup: The Robot World Cup Initiative Based on Wikipedia and presentations by Mariya Miteva, Kevin Lam, Paul Marlow.
The Computer Science Course at Omar Al-Mukhtar University, Libya The Computer Science Course at Omar Al-Mukhtar University, Libya User-Centered Design.
Artificial Intelligence
Brain, Mind, Body and Society: Controllability and Uncontrollability in Robotics Motomu SHIMODA, PhD. Kyoto Women’s University.
Robots at Work Dr Gerard McKee Active Robotics Laboratory School of Systems Engineering The University of Reading, UK
Biointelligence Laboratory School of Computer Science and Engineering Seoul National University Cognitive Robots © 2014, SNU CSE Biointelligence Lab.,
Robotica Lezione 1. Robotica - Lecture 12 Objectives - I General aspects of robotics –Situated Agents –Autonomous Vehicles –Dynamical Agents Implementing.
Institute of Perception, Action and Behaviour (IPAB) Director: Prof. Sethu Vijayakumar.
Chapter 14: Artificial Intelligence Invitation to Computer Science, C++ Version, Third Edition.
INTEGRATED SYSTEMS 1205 Technology Education A Curriculum Review Sabine Schnepf-Comeau July 19, 2011 ED 4752.
Lecture 9: Chapter 9 Architectural Design
Com3240 Adaptive Intelligence Epigenetic Robotics Amanda Sharkey Reinforcement Learning Eleni Vasilaki.
Towards Cognitive Robotics Biointelligence Laboratory School of Computer Science and Engineering Seoul National University Christian.
Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved. Decision Support Systems Chapter 10.
Chapter 6 Cognitive and Learning Characteristics © Taylor & Francis 2015.
Beyond Gazing, Pointing, and Reaching A Survey of Developmental Robotics Authors: Max Lungarella, Giorgio Metta.
SEMINAR REPORT ON K.SWATHI. INTRODUCTION Any automatically operated machine that functions in human like manner Any automatically operated machine that.
Fundamentals of Information Systems, Third Edition2 Principles and Learning Objectives Artificial intelligence systems form a broad and diverse set of.
The roots of innovation Future and Emerging Technologies (FET) Future and Emerging Technologies (FET) The roots of innovation Proactive initiative on:
Artificial Intelligence By Michelle Witcofsky And Evan Flanagan.
Sharing Design Knowledge through the IMS Learning Design Specification Dawn Howard-Rose Kevin Harrigan David Bean University of Waterloo McGraw-Hill Ryerson.
Robotics Sharif In the name of Allah. Robotics Sharif Introduction to Robotics o Leila Sharif o o Lecture #2: The.
A Context Model based on Ontological Languages: a Proposal for Information Visualization School of Informatics Castilla-La Mancha University Ramón Hervás.
I Robot.
University of Windsor School of Computer Science Topics in Artificial Intelligence Fall 2008 Sept 11, 2008.
DARPA ITO/MARS Project Update Vanderbilt University A Software Architecture and Tools for Autonomous Robots that Learn on Mission K. Kawamura, M. Wilkes,
Collaboration in eRegion- ICT for Growth and Empowerment Bror Salmelin Head of Unit, New working environments European Commission, DG Information Society.
Riga Technical University Department of System Theory and Design Usage of Multi-Agent Paradigm in Multi-Robot Systems Integration Assistant professor Egons.
Introduction to Artificial Intelligence CS 438 Spring 2008 Today –AIMA, Ch. 25 –Robotics Thursday –Robotics continued Home Work due next Tuesday –Ch. 13:
Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical.
Early Childhood Special Education. Dunst model interest engagement competence mastery.
Chapter 1: Introduction to Neuro-Fuzzy (NF) and Soft Computing (SC)
Introduction to HCI Lecture #1.
RULES Patty Nordstrom Hien Nguyen. "Cognitive Skills are Realized by Production Rules"
Artificial Intelligence: Research and Collaborative Possibilities a presentation by: Dr. Ernest L. McDuffie, Assistant Professor Department of Computer.
Slide no 1 Cognitive Systems in FP6 scope and focus Colette Maloney DG Information Society.
INTRODUCTION TO COGNITIVE SCIENCE NURSING INFORMATICS CHAPTER 3 1.
WP6 Emotion in Interaction Embodied Conversational Agents WP6 core task: describe an interactive ECA system with capabilities beyond those of present day.
Presented by: Kumar Magi. ( 2MM07EC016 ). Contents Introduction Definition Sensor & Its Evolution Sensor Principle Multi Sensor Fusion & Integration Application.
Functionality of objects through observation and Interaction Ruzena Bajcsy based on Luca Bogoni’s Ph.D thesis April 2016.
Overview of Artificial Intelligence (1) Artificial intelligence (AI) Computers with the ability to mimic or duplicate the functions of the human brain.
RoboCup: The Robot World Cup Initiative
Fundamentals of Information Systems, Sixth Edition
IPAB Research Areas and Strengths
Overview of Year 1 Progress Angelo Cangelosi & ITALK team
HI 5354 – Cognitive Engineering
MANAGING KNOWLEDGE FOR THE DIGITAL FIRM
Market-based Dynamic Task Allocation in Mobile Surveillance Systems
BCI Research at the ISRC, University of Ulster N. Ireland, UK
Behavior Based Systems
Presentation transcript:

2009, Pr Fakhreddine Karray 1 Advances in Cognitive Robotics: Achievements and Challenges Fakhri Karray Pattern Analysis and Machine Intelligence Research Lab University of Waterloo, Canada HNICEM’09, Manilla

2009, Pr Fakhreddine Karray 2 Outline  Part1 : Towards Cognition in robotics  Evolution of Robotic Systems : The Need For Cognition  What is Cognition?

2009, Pr Fakhreddine Karray 3 Outline  Part1 : Towards Cognition in robotics  Part2 : Major Aspects of Cognitive Robotics  Line of research in cognitive robotics  Building blocks or components  Theories of cognition and architectures

2009, Pr Fakhreddine Karray 4 Outline  Part1 : Towards Cognition in robotics  Part2 : Major Aspects of Cognitive Robotics  Part3 : Recent Advances  Design of Confabulation Architecture  Novel design of discrete cognitive skills

2009, Pr Fakhreddine Karray 5 Outline  Part1 : Towards Cognition in robotics  Part2 : Major Aspects of Cognitive Robotics  Part3 : Recent Advances  Part4 : Applications  Part5 : Challenges

2009, Pr Fakhreddine Karray 6 Clip from AI motion picture 2001

2009, Pr Fakhreddine Karray 7 IEEE Spectrum Online March 12, 2009 (reporting on Boston Globe article of March 4, 2009)

2009, Pr Fakhreddine Karray 8 Toward Cognition in Robotics Part2 : Major Aspects of Cognitive Robotics Part3 : Recent Advances Part4 : Applications Part5 : Challenges Part 1:

2009, Pr Fakhreddine Karray 9 The Evolutionary Stages Industrial Robotics Service Robots for Personal Use Personal Robotics Service Robotics Evolution of RoboticsWhat is Cognition? Service Robots for Professional Use Toward Cognition in Robotics

2009, Pr Fakhreddine Karray 10 Processing Units Machines Units s 100 s 10 s SI DAI AI Robotics Centralized Control Multiple Machines Machine Multiagent Distributed Robot System Swarm Robotics MEMS-based Multiple Machines Toward Cognition in Robotics Evolution of RoboticsWhat is Cognition?

2009, Pr Fakhreddine Karray 11 The need For Cognition  to learn from experience and apply the learned knowledge to deal with anomalies  to ‘understand’ human and naturally communicate with them  to reason and self-reflect to take new task initiatives We expect from next generation of robots to have the following capabilities : Evolution of RoboticsWhat is Cognition? Toward Cognition in Robotics

2009, Pr Fakhreddine Karray 12 These skills emerge in human as a result of complex, bidirectional interaction among body, brain, and environment  Self-awareness  Perception  Learning Cognitive robots are robots imparted with ‘human-like’ cognitive abilities Major Properties in Human Cognition Evolution of RoboticsWhat is Cognition?  Knowledge  Reasoning  Planning and decision making Toward Cognition in Robotics

2009, Pr Fakhreddine Karray 13 Psychology Neuro- science Biology Cognition Understanding of human cognition involves three major disciplines Toward Cognition in Robotics Evolution of RoboticsWhat is Cognition?

2009, Pr Fakhreddine Karray 14 Understanding of cognitive robotics involves : Psychology Neuro- science Biology Computer Engineering Mechanical/ Mechatronics Engineering Artificial Intelligence Computer Science Toward Cognition in Robotics Evolution of RoboticsWhat is Cognition? Cognitive Robotics

2009, Pr Fakhreddine Karray 15 Cognitive Robotics Toward Cognition in Robotics Evolution of RoboticsWhat is Cognition? Processing Units Machines Units s 100 s 10 s SI DAI AI Robotics Centralized Control Multiple Machines Machine Multiagent Distributed Robot System Swarm Robotics MEMS-based Multiple Machines

2009, Pr Fakhreddine Karray 16 Major properties required for cognition in Robotics  Physical embodiment  Social situatedness  Role of experience  Self-awareness  Value system Toward Cognition in Robotics Evolution of RoboticsWhat is Cognition?

2009, Pr Fakhreddine Karray 17 Cognition Pathways Perception pathway Reasoning pathway Planning pathway Action pathway Sensors Actuators Memory Evolution of RoboticsWhat is Cognition? Toward Cognition in Robotics

2009, Pr Fakhreddine Karray 18 Major Aspects of Cognitive Robotics Part3 : Recent Advances Part4 : Applications Part5 : Challenges Part1 : Toward Cognition in Robotics Part2 :

2009, Pr Fakhreddine Karray 19 Major Aspects of Cognitive Robotics The major aspects in cognitive robotics research  Lines of research  Building blocks of cognitive systems  Various theories of cognition/architectures

2009, Pr Fakhreddine Karray 20 Major Aspects of Cognitive Robotics The research on robot cognition is still in its infancy. A complete theory of robot cognition is not yet in the picture. Group 1 Theory of Robot cognition Group 2Discrete cognitive abilities Lines of research ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 21 Major Aspects of Cognitive Robotics Group 1 : Theories of Robot Cognition FocusPioneering Works Design principle, Architecture for cognition, Hardware and software requirement, Social factors Incremental Intelligence [Brooks 1991] Developmental Engineering [Sandini 1997] Brain-based Device [Krichmar 1998] Autonomous Mental Development [Weng 2000] Cognitive Developmental robotics [Asada 2001] Conscious Machine [Kawamura 2005] ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 22 Major Aspects of Cognitive Robotics Group 2 : Discrete cognitive skills  Visual attention  Theoretical models  Robotic models  Task-specific models  Social cognition  Joint attention  Social imitation  Value system ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 23  Computational perception  Knowledge Representation  Learning  Reasoning and planning  Interaction and communication The major building blocks of Cognitive Robots involve the following components : ComponentsTheories Major Aspects of Cognitive Robotics Line of research

2009, Pr Fakhreddine Karray Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research Perception in cognitive robotics :  allows robot to receive multi-sensory input from external environment  is crucial in realizing self-awareness capability  is connected to all other components Attention is the main mechanism required in this process

2009, Pr Fakhreddine Karray 25 Knowledge representation:  allows ease of information storage and retrieval  performs inference to obtain new information from learned data The robot deploys mechanisms such as self-organization, self- production, and self-maintenance to autonomously develop its own representation with interaction and experience. 2- Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 26 Construct new skills, knowledge and capabilities through:  Exploration: e.g. reaching, grasping, and manipulating what is around it.  Social Interaction: which involves interacting with the user in the learning process.  Observation and Imitation: can be imitation of low level features such as joint trajectories, or higher level features such as complete actions, and behaviors. 2- Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 27 Consists of a set of three simultaneous processes:  Reactive processes: they mimic the reflexive behavior observed in biological systems.  Deliberative processes: they are in charge of realizing what is commonly known as thinking in biological systems (e.g. Motion planning).  Reflective planning: they involve high-level reasoning where robot computes how to perform a given task based on its cognitive abilities, learned knowledge, and embodiments. 2- Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics The human role and the level of interaction will vary but the human is still part of the system. CooperaTION ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 29 Multimodality Adaptivity Multimodal Interaction Multimodal Modules Social Interaction Multimodal & Adpativity Modules Adaptive Interaction Adpativity Modules Direct Manipulation Interaction WIMP-based GUIs S R/R I S R/R I: Service Request/Response Interface S R/R I Alaa Khamis and Mohamed Kamel, and Miguel A. Salichs, “Human-Robot Interfaces for Social Interaction”, International Journal of Robotics and Automation, vol. 22, (206), Knowledge Representation 1- Computational perception 3-Learning 4-Reasoning and planning 5- Interaction and communication Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 30 Major Theories of Cognitive Robotics Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 31 Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics First introduced by Brooks, The main implications of this theory are requirement for :  developmental (incremental) nature of intelligence,  necessity of social situatedness and communication,  and robot capability to integrate multi- modality sensory data. Major Aspects of Cognitive Robotics ComponentsTheoriesLine of research Conscious Machine

2009, Pr Fakhreddine Karray 32 First introduced by Sandini, et al DE has emphasis on four basic principles:  physical embodiment,  social interaction,  development and experience, and  dynamic of development. Major Aspects of Cognitive Robotics Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics ComponentsTheoriesLine of research Conscious Machine

2009, Pr Fakhreddine Karray 33 First introduced by Krichmar et al., 2003 as a platform to test computational models of nervous systems. BBD devices use authentic simulation of the primate nervous systems to realize their cognitive abilities such as reasoning and planning. Major Aspects of Cognitive Robotics Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics ComponentsTheoriesLine of research Conscious Machine

2009, Pr Fakhreddine Karray 34 Was introduced by Weng et al., 2000 AMD requires the cognitive robot to be designed in a task non-specific manner assuming that robot is capable of developing the task specification program by itself. The main principles of AMD are  embodiment,  self-awareness,  self effectiveness,  and developmental program. Major Aspects of Cognitive Robotics Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics ComponentsTheoriesLine of research Conscious Machine

2009, Pr Fakhreddine Karray 35 Was introduced by Asada et al., 2001 CDR argues for the existence of a built-in architecture and puts emphasis on the role of social interaction on autonomous development of representation. CDR focuses on three main issues, embodiment in concert with Books theory, embedded structure (built-in architecture), and interaction design. Major Aspects of Cognitive Robotics Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics ComponentsTheoriesLine of research Conscious Machine

2009, Pr Fakhreddine Karray 36 Incremental Intelligence Developmental Engineering Brain-based Device Autonomous Mental Development Cognitive Developmental Robotics Conscious Machine Was introduced by Kawamura et al., 2005 CM essentially provides a method of implementation for cognitive skills inspired by findings in cognitive psychology. Authors of CM restrict their work into emotion and attention-based control of behavior in designing consciousness. CM takes a multi-agent and behavior based methodology with each (software) agent dedicated to a specific cognitive ability and maintains coordination among these agents using a built-in central architecture called intelligent machine architecture. Part 3: Aspects, Components and Architectures of Cognitive Robotics ComponentsTheoriesLine of research

2009, Pr Fakhreddine Karray 37 Recent Advances Part4 : Applications Part5 : Challenges Part1 : Toward Cognition in Robotics Part2 : Major Aspects of Cognitive Robotics Part3 :

2009, Pr Fakhreddine Karray 38  Design of Confabulation Architecture  Novel design of discrete cognitive skills Besides the theories proposed, some recent advances in Cognitive robotics were made in : Recent Advances

2009, Pr Fakhreddine Karray 39 Confabulation Theory  First development made by Hecht-Nielsen in  claimed to be the first comprehensive theory to describe cognition in humans and animals.  postulates that cognition constitutes four fundamental elements:  Mental object representation: about 4000 thalamocortical modules each comprising 10,000 to 100,000s of symbols  Knowledge links: unidirectional association between two symbols  Confabulation: the symbol with maximum total input excitation is activated  Action command origination: confabulation conclusion launches a set of pre-associated action commands Recent Advances Confabulation ArchitectureDiscreet cognitive skills

2009, Pr Fakhreddine Karray 40 Confabulation Theory provides a hypothesis and neuronal model for acquiring, storing and using knowledge Recent Advances Confabulation ArchitectureDiscreet cognitive skills An apple object and some of the attributes Knowledge link connecting word apple to color

2009, Pr Fakhreddine Karray 41 Recent Advances Motivational example: Sentence Generation Several other centenarians at Maria Manor had talked about trying to live until 2000, but only Wegner made it. Her niece said that Wegner had always been a character – former glove model, buyer for Macy’s, owner of Lydia’s Smart Gifts downtown during the 1950s and ‘60s – and that she was determined to see She was born in the Bronx Borough of New York City. First context sentence Second context sentence Plausible next sentence Confabulation architecture Confabulation ArchitectureDiscreet cognitive skills From R.H.Nielsen, “confabulation theory, The mechanics of though”, Springer 2007

2009, Pr Fakhreddine Karray 42 Recent Advances Design of Discrete Cognitive Skills  Visual attention  Social cognition  Joint attention  Social imitation  A case study : Multimodal Interaction – Multimodal attention  Value system Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 43 Recent Advances Design of Discrete Cognitive Skills Visual Attention  Visual attention is a fundamental building block of cognitive development.  It allows robots to focus their processing power only on the behaviorally relevant information and therefore facilitates their interaction with humans and surrounding environment Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 44 Recent Advances Design of Discrete Cognitive Skills Visual Attention (Recent works)  Recent works involved a bioinspired probabilistic model of visual attention of cognitive robots.  Work uses Bayesian analysis to recursively estimate the orientation of the camera head of a robot such as a visually salient/behaviorally relevant stimuli resides at the center of the visual field  Observes the postulates of Biased Competitive hypothesis (a famous neurodynamic theory of the primates visual attention) for state transition Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 45 Example task in hand: any particular arm motion of the human user Toward Cognition in Robotics Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 46 Recent Advances Design of Discrete Cognitive Skills Visual Attention : Theoretical Models Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 47 Recent Advances Design of Discrete Cognitive Skills Visual Attention : Robotic Model (1/2) Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 48 Recent Advances Design of Discrete Cognitive Skills Visual Attention : Robotic Model (2/2) Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 49 Recent Advances Design of Discrete Cognitive Skills Social Cognition  Multimodal interaction: allow users to move seamlessly between different modes of interaction, from visual to voice to touch, according to changes in context or user preference. Method facilitating social interaction between robots and humans, divided into three categories of research : Confabulation ArchitectureDiscrete cognitive skills  Joint attention: having this ability enables robot to attend to an object of mutual interest intentionally.  Social Imitation: powerful methods of learning and developing social behavior for cognitive robots.

2009, Pr Fakhreddine Karray 50 Recent Advances Design of Discrete Cognitive Skills Social Cognition – Joint Attention Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 51 Recent Advances Design of Discrete Cognitive Skills Social Cognition – Social Imitation (1/2) Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 52 Recent Advances Design of Discrete Cognitive Skills Social Cognition – Social Imitation (2/2) Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 53  Data fusion should be performed at different levels in order to extend its flexibility.  Data fusion and cognitive processes (perception, reasoning, projection) should not be treated independently  The system and/or the user are involved in the fusion model through a refinement process (feedback)  Different factors influence the fusion process, such as memory/database and experience  The fusion process is accomplished within a predefined context/goal The architecture here is based on the following major considerations : Recent Advances Design of Discrete Cognitive Skills a case study: multimodal Interaction Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 54 Recent Advances Toward a Comprehensive DataFusion Architecture Design of Discrete Cognitive Skills Confabulation ArchitectureDiscrete cognitive skills a case study: multimodal Interaction

2009, Pr Fakhreddine Karray 55 Recent Advances Design of Discrete Cognitive Skills Value System Robot’s capability to plan action perception of salient stimuli, which makes it an essential requirement of developing human-like intelligence in robotic systems  Prediction  Task non specific  Developmental  Value based learning Characteristics: Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 56 Recent Advances Design of Discrete Cognitive Skills Value System Confabulation ArchitectureDiscrete cognitive skills

2009, Pr Fakhreddine Karray 57 Applications Part5 : Challenges Part1 : Toward Cognition in Robotics Part2 : Major Aspects of Cognitive Robotics Part3 : Recent Advances Part4 :

2009, Pr Fakhreddine Karray 58 Applications  Health-care and Assistive robotics  Entertainment robotics  Military and Space exploration  Humanoid/social robotics

2009, Pr Fakhreddine Karray 59 Applications Trends in percentage of the elderly (over age 85) in the world Source: Adriana Tapus, Maha J. Mataric, and Brian Scassellati, “Socially Assistive Robotics: The Grand Challenges in Helping Humans Through Social Interaction,” IEEE Robotics & Automation Magazine, MARCH 2007 Health Care and Assistive Robotics

2009, Pr Fakhreddine Karray 60 Applications  Domo (MIT robot): A household assistant for the elderly or wheelchair-bound. It grasps objects and places them on shelves or counters.  ASIBOT: A portable assistive robot for elderly and disease people bringing more freedom in daily tasks as eating, drinking, shaving, make up wearing, tooth cleaning, etc.  Huggable: A robotic companion for health care, education, and social communication. Health Care and Assistive Robotics

2009, Pr Fakhreddine Karray 61 Clip from Information and Robot Technology Research Initiative, Tokyo, Dec. 2008

2009, Pr Fakhreddine Karray 62 Applications  Sparky project on AIBO platform: alleviating loneliness and causing to form attachments for nursing home residents.  CiceRobot: A museum guide cognitive robot.  Soccer robot  Flutist robot: The first member of a humanoid robot orchestra. Entertainment Robotics Video : Violinist Robot

2009, Pr Fakhreddine Karray 63 Clip from Toyota Research labs, 2007

2009, Pr Fakhreddine Karray 64 Applications  Robonaut: A robotic system that can function as an EVA astronaut equivalent.  BEAR: Designed to find, pick up and rescue people in harm's way. Military and Space Exploration Video : Bear Robot

2009, Pr Fakhreddine Karray 65 Clip from Vecna Robotics Inc., 2007

2009, Pr Fakhreddine Karray 66 Applications  Maggie: research platform to study human robot interaction and robot intelligence and autonomy. Humanoid/social robotics Video : Partner/ Cooperative Dancing with Maggie  CB2: child robot with biomimetic body. It is able to develop a behavior similar to a 1 or 2 years old baby.

2009, Pr Fakhreddine Karray 67 Clip from Robotics Lab, Univ. Carlos III, 2007

2009, Pr Fakhreddine Karray 68 Challenges Part1 : Toward Cognition in Robotics Part2 : Major Aspects of Cognitive Robotics Part3 : Recent Advances Part4 : Applications Part5 :

2009, Pr Fakhreddine Karray 69  Need for performance metric to evaluate the cognitive abilities in a robotic system. Difficulty in providing objective measure  Lack of standard measure to qualitatively assess intelligence in human makes the issue more challenging  More work is needed for evaluating objectively cognitive abilities of robots Issue I: Need for Metrics and Evaluation Measure for Cognition Challenges

2009, Pr Fakhreddine Karray 70  A cross-disciplinary collaborative study to integrate the most understood aspects of human cognition in order to come up with a comprehensive theory of robot cognition.  Lack of comprehensive architecture on which to impart cognitive skills for robots  Robustness and adaptability of robot cognition from one environment to the next.  Difficulty in dealing with uncertainty in perception Issue II: Integration of cross-disciplinary knowledge Challenges

2009, Pr Fakhreddine Karray 71  Tighter cooperation involving roboticists, AI specialists, Neuroscientists and psychologists is much needed  Avoid for now the goal of designing a general theory for robot cognition and focus on designing robots with cognitive traits that are task specific  More work needed on understanding the dynamics of cognitive paths in human brains  Further work is needed on the mathematical modeling of developmental cognition in humans Final Thoughts

2009, Pr Fakhreddine Karray 72 Campus University of Waterloo

2009, Pr Fakhreddine Karray 73 University of Waterloo at Glance Canada leading comprehensive University (25,000 Students) Largest Co-op program in the world Largest Math Faculty in the World (3500 students) Largest Engineering Faculty in Canada (3800 grad and undergrad students) Ranked as the “most innovative” in Canada for the past 16 years in a row. Largest employer in Waterloo, RIM provider of the BlackBerry Device (6500 employees) spinoff of University of Waterloo

2009, Pr Fakhreddine Karray 74 Pattern Analysis and Machine Intelligence Lab Electrical and Computer Engineering University of Waterloo Canada

2009, Pr Fakhreddine Karray 75 Pattern Analysis & Machine Intelligence Research Group Current Supporting Institutions/Companies

2009, Pr Fakhreddine Karray 76 Pattern Analysis & Machine Intelligence Research Group  Established in early eighties  Members: M. Kamel, F. Karray, O. Basir, A. Wong  Principal participant in major Provincial and Federal centers of excellence (MMO, CITO, Telelearning-NCE, PRECARN, LORNET).  Extensive experience in promoting fundamental and applied research to government and industry  Graduated over 450 Ph.D and Master students over the past 25 years.  Average annual budget of $2.5 M  Four Spin-off companies: Virtek (1986), PDS (1997), IMS (1999), Vestec (2002).

2009, Pr Fakhreddine Karray 77 Major Research Areas  Cooperative Intelligent Systems  Intelligent Robotics and Cognitive Machines  Computer Vision and Image Processing  Pattern Recognition and Intelligent Agents  Sensors and embedded systems  e-Learning  Applications: Automotive Industry, Aerospace, Financial Sector, Web Services, Biomedical, Health Care Pattern Analysis & Machine Intelligence Research Group

2009, Pr Fakhreddine Karray 78 Clip from Wall e motion picture 2008

2009, Pr Fakhreddine Karray 79 Acknowledgements Dr. Alaa Khamis, Research Asst. professor Dr. Abbas Ahmadi, PostDoc fellow Malek Baklouti, PhD student Momotaz Begum, PhD student Jamil Abu Saleh, PhD student Bahador Khalegui, PhD student Ahmed ElMogy, PhD student and Technical staff of the PAMI Research Lab

2009, Pr Fakhreddine Karray 80 Thank you for your attention

2009, Pr Fakhreddine Karray 81 Recent Advances Design of Discrete Cognitive Skills a case study: multimodal attention Confabulation ArchitectureDiscrete cognitive skills Multi-modal attention: Natural speech command modulates the visual attention behavior of the robot in the form of top-down modulating bias

2009, Pr Fakhreddine Karray 82 Recent Advances Design of Discrete Cognitive Skills Visual Attention (Recent works) Confabulation ArchitectureDiscrete cognitive skills Functional description of the visual attention model

2009, Pr Fakhreddine Karray 83 References [Brooks1991] R. A. Brooks. Intelligence without representation. Artificial Intelligence 47: , [Asada2001] M. Asada, K. F. MacDormanb, H. Ishigurob, and Y. Kuniyoshi. Cognitive developmental robotics as a new paradigm for the design of humanoid robots. Robotics and Autonomous Systems, 37:185 – 193, 2001 [Sandini2001] G. Metta, G. Sandini, L. Natale, and F. Panerai. Development and robotics. In IEEE-RAS International Conference on Humanoid Robots, pages 33 – 42, 2001 [Kawamura2005] K. Kawamura. Cognitive approach to a human adaptive robot development. In Proceedings of IEEE International Workshop on Robot and Human Interactive Communication, pages 210 – 216, 2005 [Weng2004] J.Weng. Developmental robotics: Theory and experiments. International Journal of Humanoid Robotics, 1:199 – 234, [Krichmar2003] J. L. Krichmar and G. M. Edelman. Brain-based devices: intelligent systems based on principles of the nervous system. In IEEE/RSJ International Conference on Intelligent Robot and Systems, pages 940 – 945, [Nielsen2007] R. Hecht-Nielsen. Confabulation Theory. Springer-Verlag, 2007.