1. Controlling and Configuring Large UAV Teams Paul Scerri, Yang Xu, Jumpol Polvichai, Katia Sycara and Mike Lewis Carnegie Mellon University and University of Pittsburgh

2. Context Aim to build large heterogeneous teams for complex tasks –Robots, agents, people –10,000s of actors Multiagent version of Belief Desire Intention approach to autonomous behavior –Builds on key abstraction of a Team Oriented Plan Defines the activities that must take place and interactions between those activities –Supported by extensive theoretical (logical) work Key algorithms are NP-complete or worse –Heuristics required for scalability

3. Specific Target Application: Wide Area Search Munitions (WASMs) Part munition, part unmanned aerial vehicle –Single use –Variety of sensors –Limited fuel supply, approximately 30 minutes –Communicate with each other, manned aircraft Concept of Operations (under development) –Small number of manned aircraft –Potentially other ground forces –100s of WASMs, performing a variety of missions Attack Search Battle damage assessment Decoys Communication relays Flight test planned September, 2005 –1 real and 3 simulated WASMs

4. Token-Based Coordination Token: self contained packet capable of being sent around team –Information content –Control content Local models –Team members use receipt of tokens to create local models of other team members What sorts of things are they/are they not working on? What sorts of things might they need to know? –Local models are used to improve the routing of future tokens Token “flows” implement coordination No brittle, non-scalable “message protocols”

5. Token Based Algorithms Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

6. Token Based Algorithms Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings Information about environment passed around in tokens, when agent receives tokens matching plan pre-conditions plan is initiated Very high probability all applicable plans are initiated Liao et al, 2004

7. Token Based Algorithms Information about initiated plans shared in tokens Very high probability some agent gets to find out about any duplicate plans Liao et al, 2004 Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

8. Token Based Algorithms Responsibility to perform a role encapsulated in token Only team member with token can perform role Team member must have capability > threshold to perform role Threshold calculated from estimates of likely role allocation outcome Okamoto, 2004 Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

9. Token Based Algorithms Locally sensed information shared via token to get information to team members performing role effected by information Does not require sensing agent to know who needs information Xu et al, 2004 Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

10. Token Based Algorithms Access to resource represented by token Only team member with token can use resource Team member must have need > threshold to keep resource Threshold changes dynamically as it moves around the team, seeing resource need Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

11. Token Based Algorithms Uncertain sensor readings encapsulated in tokens and sent around the team Very high probability that at least one team member gets related sensor readings and fuses for higher confidence Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

12. Token Based Algorithms Assumptions used to make decisions are attached to tokens resulting from that decision Very high probability agents with contradictory information see the assumptions and initiate checking process Plan instantiation Removal of duplicate plans Role allocation Information sharing Resource allocation Sensor fusion Recovering from faulty sensor readings

13. Layered Team Member Architecture Bottom: Local Reasoning Team member’s local actions are restricted by the tokens they have –E.g., without resource token, cannot use resource Middle: Coordination Reasoning Movement of tokens around the team implements the coordination –Flows of tokens –Local models inform token routing Top: Meta-reasoning Ensures token flows work effectively Token Local Routing Model Meta

14. Synergies Between Token Algorithms Overall performance depends on how well tokens are routed –Team members use local models to improve routing Observation: Execution of algorithms shares information that other algorithms can exploit –E.g., if role for strike near Pittsburgh was allocated to WASM X, then air space resources around Pittsburgh likely of interest to WASM X Implementation: Use all tokens to improve local models –E.g., role tokens change local models, resource tokens move according to local models Result: When all algorithms are working together the system actually performs better

15. Implementing Synergies - Example When receiving an role token from neighbor i –Probability of sending information token to i changed proportionally to similarity between “plan initiated” token and information token –Probability of sending resource token to i changed inverse proportionally to similarity between role and resource token

16. Meta-Reasoning for Token Flows Specific tokens are meta-reasoned about –Identify tokens that are not behaving as expected –Bring to human attention –Examples Tokens that “live” too long Tokens that travel too much –See Scerri et al, AIAA 2004 Overall flows can be controlled to optimize for specific criteria –By controlling the flows, we control how the coordination works

17. Neural Networks for Modeling and Controlling Token Flows Use a simple input/output feed-forward neural networks to represent a team performance model –Three-layer FFNN is capable of representing any arbitrary functions Extend to Dynamic Networks concept to cope with dynamic behaviors –This kind of network enlarges the capacity to deal with non- deterministic problem Learn the model with genetic algorithms methodology –Excellent for dealing with very huge and noisy training data set –Based on around 1,000,000 simulation runs

18. Configuring Algorithms with NN Offline –Change environment and algorithm parameters, observe expected performance Online –Use NN “in reverse” to find parameter settings for specific optimization criteria As environment changes As requirements change –Allows tradeoff between performance of all algorithms

19. Configuration Interface

20. Results: Token Based Coordination Total Reward Messages Fully IntegratedRandom

21. Results: Configuring Algorithms

22. Results: Online Control

23. Machinetta: Bringing it all Together Encapsulate token-based coordination approach into reusable software module –Called a proxy Proxies provide domain independent coordination algorithms –Do not provide domain specific communication and interface code –Available in the public domain Machinetta used to demonstrate coordination of up to 500 distributed, heterogeneous team members in several distinct domains –Demonstrates that token based coordination is feasible –Biggest teams developed to date?

24. Conclusions and Future Work Token-based coordination as a feasible alternative paradigm for large teams Additional layer over token flows gives high levels of control Future Work –Can we make more precise mathematical models? Markov chains?