1. Cognitive Science 1 Kartik Talamadupula Subbarao Kambhampati J. Benton Dept. of Computer Science Arizona State University Paul Schermerhorn Matthias Scheutz Cognitive Science Program Indiana University Planning for Human-Robot Teaming

2. Cognitive Science MotivationMotivation 2 Early motivation of AI – Autonomous control for robotic agents Plenty of applications – Household Assistance – Search and Rescue – Military Drones and Mules All scenarios involve humans giving orders Planning must co-opt this area

3. Cognitive Science Human-Robot Teaming Teaming – Share the same goal(s) – Autonomous behavior – Communication Role of Planning – Plan generation – Feedback acceptance – Model resolution HUMAN ROBOT PLANNER Planning and Execution Monitoring Human Robot Interaction (HRI) Mixed Initiative Planning (MIP) What are the factors that planners must take into account?

4. Cognitive Science Dimensions Scenario / Environment Inspired by the real world Large amounts of domain knowledge from – Humans with experience – Technical documents and manuals New knowledge may arrive during execution – Planner must handle such contingencies Planner and Robot Features – Determined by the needs of the scenario – E.g.: NASA needs temporal planning

5. Cognitive Science Dimensions Robotic Agent Central Actor – Execute actions – Gather sensory feedback Different types of robots – Various capabilities Gripper Humanoid Mobile Combined

6. Cognitive Science Dimensions Human User Specifies and updates: – Scenario goals – Model (in some cases) Must be in communication with robot/system Novice Uses the robot merely as an assistant Domain Expert Authority on the execution environment System Expert Authority on the integrated AI system

7. Cognitive Science Planning Goal Management Human-Robot Teaming – Utility stems from delegation of goals Support different types of goals – Temporal Goals: Deadlines – Priorities: Rewards and Penalties Bonus Goals: Partial Satisfaction – Trajectory Goals – Conditional Goals Changes to goals on the fly – Open World Quantified Goals [Talamadupula et al., AAAI 2010]

8. Cognitive Science One true model of the world – Robot High + Low Level models – Human User Symbolic model + Add’l knowledge – Planner must take this gap into account Model Maintenance v. Model Revision – Usability v. Consistency issues – Use the human user’s deep knowledge Distinct Models – Using two (or more) models Higher level: Task-oriented model Lower level: Robot’s capabilities Planning Model Management

9. Cognitive Science HRT Tasks: Examples SEARCH AND REPORT RECONNAISSANCEKITCHEN ROBOT ROBOT Mobile Mobile and Manipulator HUMAN (USER) Domain ExpertSystem ExpertNovice MODEL Less DynamicDynamicHighly Dynamic GOALS EvolvingStaticEvolving COMMUNICATION Natural LanguageAPIsNatural Language Feature Task

10. Cognitive Science Case Study Urban Search and Rescue Human-Robot Team in Urban Setting – Find and report location of critical assets – Human: Domain expert; removed from the scene SEARCH AND REPORT Deliver medical supplies Bonus Goal: Find and report injured humans Requirements – Updates to knowledge base – Goal changes [Talamadupula et. al., AAAI 2010] RECONNAISSANCE Gather information High risk to humans – E.g. Bomb defusal Requirements – Support model changes – New capabilities E.g.: Zoom camera

11. Cognitive Science Goal Manager Monitor Planner Plan Problem Updates Updated State Information Initial Model Information Sensory Information Actions System Integration Additional Capabilities Model Update

12. Cognitive Science Model Update: Demo Run 12 Initial Goal End of hallway During Execution Injured humans (boxes) in rooms behind doors New action / effect during execution  Push doors to get inside rooms

13. Cognitive Science Conclusions Human-Robot Teaming from a planning perspective Planning Challenges – Framework for Human-Robot Teaming Problems – Model and Goal Management Need to define the scope of planning for these tasks – What are the main technical problems Huge potential for novel P&Sapplications Companion Robots Military and Service Drones Household Assistants

14. Cognitive Science Future Work Multiple Models – Use two (or more) models to direct the planning Task v. Motion Level (BTAMP Workshops) Classical v. More Expressive Robotic Proactiveness – “Ask” for help – Many sources of knowledge in the real world – Putting the “teaming” in HRT More Application Scenarios – Design planners sensitive to HRT issues System Demo Tuesday 5:30pm Main Conference