Impromptu Teams of Heterogeneous Mobile Robots Ross Mead Dr. Jerry B. Weinberg

Introduction Following the World Trade Center attacks, teams of mobile robots were called to Ground Zero to aid in finding victims. Following the World Trade Center attacks, teams of mobile robots were called to Ground Zero to aid in finding victims. This marks the first time in history that robots were used in a real search and rescue environment. This marks the first time in history that robots were used in a real search and rescue environment.

Introduction The small size of these machines granted them the ability to investigate places where workers could not go. The small size of these machines granted them the ability to investigate places where workers could not go. Rescuers were able to perceive and communicate with victims… Rescuers were able to perceive and communicate with victims…  using visual and audio sensors

Introduction Participating teams came from different organizations across the country… Participating teams came from different organizations across the country…  had not previously met Robots were heterogeneous Robots were heterogeneous  varied in shape, size, and functionality

Introduction Based on their abilities, robot teams were assigned particular objectives… Based on their abilities, robot teams were assigned particular objectives…  Hazardous materials  Medical  Logistics  Planning  Search-and-rescue

Introduction Despite this delegation, did not demonstrate teamwork. Despite this delegation, did not demonstrate teamwork. The overall goal was the same… The overall goal was the same…  The robots were only aware of their own assigned tasks. Missions were carried out individually, and were limited solely to the abilities of that robot… Missions were carried out individually, and were limited solely to the abilities of that robot…  A failure of a single unit would greatly hinder the completion of the overall search and rescue operation.

Introduction How can these robots, which have never interacted, collaborate to accomplish a joint objective? How can these robots, which have never interacted, collaborate to accomplish a joint objective? How can abilities, internal knowledge, perceptual experiences, and tasks be uniformly represented and understood by all members of an impromptu robot team? How can abilities, internal knowledge, perceptual experiences, and tasks be uniformly represented and understood by all members of an impromptu robot team?  composed of independent agents each unknown to the others Once this information is obtained, how can robots reorganize and delegate individual and joint objectives, based on individual abilities and limitations? Once this information is obtained, how can robots reorganize and delegate individual and joint objectives, based on individual abilities and limitations?

Background A homogeneous team of robots has been used in a search and rescue environment. A homogeneous team of robots has been used in a search and rescue environment.  Tejada et al. (2003) Impromptu teams have been applied to the RoboCup robotic soccer league. Impromptu teams have been applied to the RoboCup robotic soccer league.RoboCup  Bowling et al. (2005)

Background A theoretical foundation for teamwork among mobile robots has been provided. A theoretical foundation for teamwork among mobile robots has been provided.  Cohen et al. (1991) A multi-layered approach to heterogeneous teams has been discussed and implemented. A multi-layered approach to heterogeneous teams has been discussed and implemented.  Balch et al. (2002)

Background symbol grounding problem symbol grounding problem  in artificial intelligence (AI), the representation of internal knowledge and how it is meaningfully interpreted as concepts in the physical world Three levels of knowledge representation: Three levels of knowledge representation:  iconic – physically similar to what it represents  indexical – links between icons  symbolic – provides a relationship between icons, indices, and other symbols

Goals/Objectives If heterogeneous robots are to organize and work together as an impromptu team, a language must be developed that allows them to share knowledge in a meaningful way. If heterogeneous robots are to organize and work together as an impromptu team, a language must be developed that allows them to share knowledge in a meaningful way. For consistent interpretation of communicated knowledge within a team, symbols representing the knowledge must be grounded in concepts... For consistent interpretation of communicated knowledge within a team, symbols representing the knowledge must be grounded in concepts...  refer to categories of entities in an environment

Goals/Objectives Symbolic communication protocol based upon the Semantic Web. Symbolic communication protocol based upon the Semantic Web.  extension of World Wide Web  gives information well-defined meaning relates data to the corresponding entity in the real world relates data to the corresponding entity in the real world

Goals/Objectives Symbolic communication protocol based upon the Semantic Web… Symbolic communication protocol based upon the Semantic Web…  extension of World Wide Web  gives information well-defined meaning relates to the corresponding entity in the real world relates to the corresponding entity in the real world

Goals/Objectives

Goals/Objectives Thus, if a library of unified grounded physical concepts was established… Thus, if a library of unified grounded physical concepts was established…  and a robot’s abilities, perceptions, and goals were represented by symbols relating to those concepts … then the robot could meaningfully share its symbols with others based on the agreed concepts. … then the robot could meaningfully share its symbols with others based on the agreed concepts.

Goals/Objectives Information is presented in a structured form… Information is presented in a structured form…  similar to a descriptive sentence   By enforcing this strict syntax, robot attributes can be broken down and related to the traits of others. By enforcing this strict syntax, robot attributes can be broken down and related to the traits of others. This structure can easily be interpreted by both computers and people… This structure can easily be interpreted by both computers and people…  simple to edit, search, and communicate

Goals/Objectives Based on the characteristics of XML… Based on the characteristics of XML…  language used to program the Semantic Web XML is able to encode metadata… XML is able to encode metadata…  information about the information XML is extensible. XML is extensible.

Goals/Objectives XML is designed so that new tags can be defined that describe new concepts and categories of information… XML is designed so that new tags can be defined that describe new concepts and categories of information…  can be shared with others searching for information The communication protocol would be layered beneath a robot’s own independent program… The communication protocol would be layered beneath a robot’s own independent program…  provide meaning to symbols relating to itself and other robots in its immediate area.

Goals/Objectives A task management system is needed to complete individual and joint goals. A task management system is needed to complete individual and joint goals. The team must consider the individual abilities, limitations, costs, and rewards of each robot to delegate tasks and coordinate activities appropriately… The team must consider the individual abilities, limitations, costs, and rewards of each robot to delegate tasks and coordinate activities appropriately…  must be tolerant of communication failure and dysfunctional teammates

Procedures The robot coordination system will initially be implemented and tested in simulation in PyRo. The robot coordination system will initially be implemented and tested in simulation in PyRo.PyRo  programming environment for different types of robots  specifics of actuators and sensors are abstracted program can be shared across different platforms program can be shared across different platforms ultimate tool for heterogeneous robot code portability ultimate tool for heterogeneous robot code portability

Procedures This project will be developed within the framework of the Department of Computer Science (CS) Senior Assignment. This project will be developed within the framework of the Department of Computer Science (CS) Senior Assignment.Department of Computer ScienceSenior AssignmentDepartment of Computer ScienceSenior Assignment  The target task will be the American Association for Artificial Intelligence (AAAI) Robot Scavenger Hunt Challenge. American Association for Artificial Intelligence Robot Scavenger Hunt ChallengeAmerican Association for Artificial Intelligence Robot Scavenger Hunt Challenge

Procedures A robotic system to search the Engineering Building for a list of objects will be developed… A robotic system to search the Engineering Building for a list of objects will be developed…  working with senior CS students Code must be portable and list of objectives must be easy to modify Code must be portable and list of objectives must be easy to modify  working with three ActivMedia robots ActivMedia  Elmer, Taz, Marvin ElmerTazMarvin ElmerTazMarvin

Procedures Working alongside the scavenger hunt project, I will be developing the robot coordination system. Working alongside the scavenger hunt project, I will be developing the robot coordination system. The scavenger hunt is, in many ways, analogous to urban search and rescue… The scavenger hunt is, in many ways, analogous to urban search and rescue…  a successful implementation of the system will show proof-of-concept of impromptu teams in a search and rescue environment

References S. Westphal (2001). “Robots join search and rescue teams”, New Scientist (Online) 19 September March S. Westphal (2001). “Robots join search and rescue teams”, New Scientist (Online) 19 September March R. Murphy (2004) “Human-Robot Interaction in Rescue Robotics,” IEEE Systems, Man and Cybernetics Part C: Applications and Reviews, special issue on Human-Robot Interaction, Vol. 34, No. 2, May R. Murphy (2004) “Human-Robot Interaction in Rescue Robotics,” IEEE Systems, Man and Cybernetics Part C: Applications and Reviews, special issue on Human-Robot Interaction, Vol. 34, No. 2, May M. Bowling & P. McCracken (2005) “Coordination and Adaptation in Impromptu Teams,” In the Proceedings of AAAI-05. M. Bowling & P. McCracken (2005) “Coordination and Adaptation in Impromptu Teams,” In the Proceedings of AAAI-05. S. Tejada, A. Cristina, P. Goodwyne, E. Normand, R. O’Hara, & S. Tarapore (2003) “Virtual Synergy: A Human-Robot Interface for Urban Search and Rescue”. In the Proceedings of the AAAI 2003 Robot Competition, Acapulco, Mexico. S. Tejada, A. Cristina, P. Goodwyne, E. Normand, R. O’Hara, & S. Tarapore (2003) “Virtual Synergy: A Human-Robot Interface for Urban Search and Rescue”. In the Proceedings of the AAAI 2003 Robot Competition, Acapulco, Mexico. P. Cohen & H. Levesque (1991) “Teamwork,” Nous 25:11—24 P. Cohen & H. Levesque (1991) “Teamwork,” Nous 25:11—24 T. Balch & L. Parker (2002) Robot Teams: From Diversity to Polymorphism, A K Peters, Ltd., Natick, Massachusetts. T. Balch & L. Parker (2002) Robot Teams: From Diversity to Polymorphism, A K Peters, Ltd., Natick, Massachusetts. Harnad, S. (2003) The Symbol Grounding Problem, Encyclopedia of Cognitive Science, Nature Publishing Group/Macmillan. Harnad, S. (2003) The Symbol Grounding Problem, Encyclopedia of Cognitive Science, Nature Publishing Group/Macmillan. R. Daviddson (1993) “A Framework for Organization and Representation of Concept Knowledge in Autonomous Agents,” In Scandinavian Conference of Artificial Intelligence— 93, IOS Press. R. Daviddson (1993) “A Framework for Organization and Representation of Concept Knowledge in Autonomous Agents,” In Scandinavian Conference of Artificial Intelligence— 93, IOS Press. Semantic Web. Ed. Eric Miller, et al. 8 March W3C. 11 March Semantic Web. Ed. Eric Miller, et al. 8 March W3C. 11 March

Questions?