December 20, 2015 Decentralized Mission Planning for Heterogeneous Human-Robot Teams Sameera Ponda Prof. Jonathan How Department of Aeronautics and Astronautics Massachusetts Institute of Technology

2 Motivation Modern day complex missions involve networked teams of heterogeneous agents executing several tasks simultaneously: –Unmanned aerial vehicles (UAVs) – target tracking, surveillance –Human operators – classify targets, monitor status –Ground convoys – rescue operations Key Research Questions: –How can we coordinate team behavior to improve mission performance? –How should planning strategies evolve as we acquire more information?

3 Problem Statement Goal: Automate task allocation to improve mission performance –Spatial and temporal synchronization –Reduce costs and improve efficiency Key Technical Challenges: –Combinatorial decision problem – computationally intractable (NP-hard) –Complex agent modeling & constraints (stochastic, non-linear, time-varying) –Limited resources (bandwidth, fuel, etc) –Dynamic networks and communication constraints –Unknown and dynamic environments Agent 2 Task 2 Agent 1 Agent 4 Agent 6 Agent 3 Agent 5 Task 1 Task 3 Task 7 Task 6 Task 5 Task 4 Task 9 Task 8

4 Planning Approaches Optimal solution methods are computationally intractable for large problems –Typically use approximation methods Centralized Planning approach –Mission Control Center (MCC) plans & distributes tasks for all agents –High bandwidth, slow reaction, resource intensive Recent research in Decentralized Planning –Individual agents make their own plans and coordinate with each other –Faster reaction to local information changes –Trade-off between communication and computation Key Questions: –What quantities should the agents agree upon? Information / tasks & plans / objectives / constraints –How do we ensure that the planning is robust to inaccurate information and models? Agent 1 Agent 4 Agent 2 Agent 3 Agent 4 Agent 2 Agent 3 MCC Agent 1

5 Consensus-Based Bundle Algorithm Decentralized task allocation approach called Consensus-Based Bundle Algorithm (CBBA) [Choi, Brunet, How 2009] –CBBA iterates between 2 phases: Bidding & Consensus Core features of CBBA: –Polynomial-time decentralized algorithm with provably good approximate solutions –Consensus on task assignments, not information – guaranteed real-time convergence even with inconsistent information Phase 2: Consensus (all agents) All agents consistent? Yes No Phase 1: Build Bundle & Bid on Tasks (individual agents) 3 2 N 1 Key extensions to CBBA: 1)Temporal constraints – Time-windows of validity for tasks 2)Connectivity issues and constraints 3)Planning for teams with Humans-in-the-loop

6 CBBA with Time-Windows In realistic missions, task scores often depend on arrival times and have associated time-windows of validity: Issue: Planning algorithms usually involve time discretization –Extra planning dimension – computationally intractable! CBBA extended to include time-windows –Solution does not discretize time! –Preserves convergence properties –Planner decides arrival times, producing task schedules for agents Embedded CBBA with Time-Windows into a real-time system architecture Arrival Time Score Arrival Time Score Time-criticalPeak-timeFlat Arrival Time Score e.g. monitor status, security shifts e.g. rescue ops, target tracking e.g. rendezvous, special ops

7 CBBA with Time-Windows CBBA successfully used in real-time fight test environments –Cooperative search, acquisition, and track (CSAT) –Coordination of agents under dynamic network topologies Further information available online at:

8 Connectivity: Network Challenges As agents move around in the environment, expect varying network topologies –Limited communication radius between agents –Potential broken comm links and/or disconnected networks Main issue: Planner cannot converge with a disconnected network, leading to conflicting assignments Developed two solution approaches: –CBBA with Relays – Creates relay tasks to ensure connectivity –CBBA with Network Handling Protocols – Protocols to adjust task lists prior to planning Task 1 Agent 2 Task 2 Agent 1 Agent 4 Agent 6 Agent 3 Agent 5 Task 3 Task 7 Task 6 Task 5 Task 4 Task 9 Task 8 Disconnected Network

9 Connectivity: CBBA with Relays Extended CBBA to include relay tasks – (Published in GlobeComm 2010) –Employs underutilized agents as relays –Key feature: Agents use bid info to predict network structure at select times –Guarantees connectivity –Computationally efficient - converges in real-time CBBA with Relays improves team performance and network connectivity Relay Task

10 Connectivity: Network Protocols If preventing disconnects is too conservative: Network Handling Protocols to adjust task lists for agents prior to planning – (Published in ACC 2010) Local Adjustment improves mission performance with low bandwidth and computation requirements Baseline (no adjustment) Central Adjustment Local Adjustment All tasks available to all agents MCC distributes tasks to networks at each replan MCC distributes new tasks to closest agents (once per task) Low Bandwidth High Bandwidth Low Bandwidth Low Computation High Computation Low Computation Conflicting Assignments – lower mission scores and wasted fuel Guaranteed Deconfliction – higher mission scores and lower fuel consumption

11 Planning for Human-Robot Teams Most modern missions involve human-robot teams –Human operators perform several tasks (e.g. supervisory, target classification, monitoring) –Need to coordinate robotic agents and operators Main Issue: Operator performance is stochastic –Heterogeneous operator capabilities (“slow” vs. “fast”) –Robustness to uncertainty in team performance Recent research has explored modeling operators using probabilistic distributions – [Cummings et al ‘10] Key Challenge: Incorporate uncertainty into planner to increase robustness Log-Normal Distribution for Operator Target Identification Figure from [D. Southern, Masters Thesis, 2010] Predator UAV Operations – Associated Press

Planning for Human-Robot Teams Consider a time-critical mission with operators performing target classification As expected vs. actual service times differ, planner performance degrades –Adding a margin of conservatism can mitigate this problem –Tradeoff between late penalties and number of tasks assigned Simulation Observations: –Performance is best when expected & actual are close (ridge line) –Steeper drop for overestimating (optimistic) vs. underestimating (conservative) –Conservative Planning performs better than Optimistic Planning Developing a Robust Planning Framework –Explicitly embed PDFs of plan parameters –Adapt as estimates improve Time Tasks Agent Schedule Late! 12

Planning for Human-Robot Teams Currently performing Human-in-the-loop experiments at Cornell –CBBA used to allocate targets to agents (MIT) –Image processing and sensor fusion used to update target PDFs (Cornell) –Human-in-the-loop for target classification and PDF updates through HRI (Cornell) 13

14 Conclusions Explored strategies to coordinate team behavior to improve mission performance Extended the Consensus-Based Bundle Algorithm (CBBA) to address the demands of more realistic multi-agent mission planning –Included task time-windows of validity –Addressed connectivity issues and communication constraints –Explored planning for heterogeneous human-robot teams Current research and expected thesis contributions: 1)Robust decentralized planning framework Embed distributions of parameters into planner Preserve computational tractability and scalability (e.g. avoid discretization, explore efficient sampling techniques) 2)Flexible planner structure that adapts to dynamic uncertainty representations Modular uncertainty representations (Nonparametric Bayesian models, etc) Modify planning strategy without recomputing all scenarios 3)Efficient strategies for information consensus to improve planner performance Decide what information and when to share (e.g. hyperparameter consensus) Cooperative decentralized strategies to update global distributions