Agent-based Composition of Behavior Models Katia Sycara (PI) Start date: 10/02/02 Gita Sukthankar Anupriya Ankolekar The Robotics Institute Carnegie Mellon University

Talk Outline Vision Limitations of Current Models Research Objectives Research Approach Expected Impact Accomplishments Deliverables

Fully automated, high fidelity Computer Generated Forces have enormous value for military simulation and training High fidelity CGFs provide realistic adversaries and team mates Utilize multi-agent architectures to go beyond current limited behaviors to adaptive opponent/teammates with human-like unpredictability Can learn from experience Embodying Team behaviors Can be used for shipboard and embedded training Training can be conducted using standard computer equipment (e.g. PCs) Will be cost-effective and affordable Automated CFGs reduce the training manning requirements Agent-oriented software engineering techniques promote modularity and reuse

Limitations of Current Models Current CGF training models are limited and inflexible –They exhibit a small hard-coded set of behaviors –They do not allow the coach to easily customise the training experience –They are hard to develop and troubleshoot Current human performance modeling techniques –Have not been successfully scaled to complex tasks –Have not been applied to modeling teams –Models are expensive to construct –Models do not allow reuse

Research Objectives Develop techniques that: –Enable CGFs to increase range of behaviors to incorporate smart human-like strategies and adaptation –Allow efficient reuse and composition of CGF models –Allow the development of models of adversaries and team mates that are consistent with human behavior modeling –Reduce model construction time and cost

Research Approach Integration of multi-agent architectures and software engineering techniques to increase CGF sophistication and enable reuse Leverage our expertise in the development of intelligent agents to increase the autonomy, range of behaviors and long-term strategic level thinking of the CGFs Use knowledge bases of composable CGF plan fragments that encapsulate particular behaviors Use libraries of reusable software components and connectors to create executable code COTS game engines and state of the art animations provide a realistic and affordable simulation platform deployable for classroom, shipboard, and embedded training (PC’s with game software) Demonstration Domain: Urban Warfare

What’s unique about our approach? The combination of semantically rich agent representation and software engineering development methodology The multi-agent architectural approach enables modeling of team behaviors This approach will result in affordable, coachable teams of realistic training forces

Functional Architecture Belief Editor Plan Editor Simulation Environment Trainee Trainer Reasoner Knowledge Structures Reasoner CGF Model In te rn al E ve nt s

Armies Fight in Teams and so must their Training Simulations Teamwork in Open Environments [Sycara et al.] incorporates heterogeneous teams and dynamic team formation –Teams are not assumed to be fixed in size or team members abilities –Model accommodates dynamic role assignment according to current situation and individual capability –Model accommodates discovery and incorporation into the team of new appropriate team members (adapts to the loss of members) –Teams can be formed/reformed dynamically during execution in response to incoming/changing goals and environment –Negotiation of team goals and commitments –Has been applied to Joint Mission Planning (Agent Storm)

Our approach enables reuse at multiple levels Individual CGFs can be adapted for different scenarios and domains Programmers reuse already developed CGF behavior fragments to construct new CGFs Our multi-agent architecture (RETSINA) is a proven model of software development that has been reused across multiple domains

Composition Composition of agents at task level –SE language: an agent is a computational process (an “smart” component). An agent can be viewed as a unit of planning and execution –Thus, composition of plan fragments and associated code Manage interdependencies between plan fragments by matching preconditions, beliefs, commitments, constraints (at reactive and cognitive levels) Manage interdependencies between code by matching inputs and outputs Promising approach from Software Engineering –Use a library of adapters and connectors to manage interdependencies and repair violated dependencies between composed agents

Appropriate representation facilitates reuse and composition of pre-existing plans Explored room Defeated enemies Team formed Unexplored room Knowledge base of pre-developed plan fragments CLEAR ENTRY GAIN DOMINANT POSITION CLEAR INTERIOR OF x CLEAR AREA x IF DOOR LOCKED SHOOT BOLT IF DOOR CLOSED KICK DOOR IF WIDE ENTRY STRAFE ENTRY HUG WALL… … … Executable actions communicated to UT and executed by CGF Abstract plan fragments

Appropriate representation facilitates reuse and composition of pre-existing plans Plan fragment reuse and composition in similar new situations Clearing Room CLEAR BUILDING CLEAR ROOM CLEAR BUILDING INTERIOR CLEAR ENTRY GAIN DOMINANT POSITION CLEAR ROOM INTERIOR Explored room Defeated enemies Team formed Unexplored room DOOR LOCKED: SHOOT BOLT HUG WALL… … … Executable actions that are communicated to UT and executed by CGF Abstract plan fragments

Appropriate representation facilitates reuse and composition of pre-existing plans Plan fragment reuse and composition in similar new situations Clearing Cave CLEAR CAVE CLEAR ENTRY GAIN DOMINANT POSITION CLEAR CAVE INTERIOR WIDE ENTRY: STRAFE ENTRY HUG WALL… Explored cave Defeated enemies Team formed Unexplored cave … … Executable actions that are communicated to UT and executed by CGF Abstract plan fragments

Realistic and Affordable Simulation Environment: UnrealTournament (UT) UT Engine (C/C++) Gamebots TCP/IP Interface Urban Scenario CGF Agent CGF Agent CGF Agent CGF Agent

We can embed CGFs into larger tactical simulations OneSAF To simulate larger military entities, behaviors, & capabilities UT Game Engine To provide real-time high quality graphics and detailed local behavior Correlated entities Correlated terrain

SAF Manager SAF entities Show entities information Show Current PDUs in OTB Show UT entities Show network information Show existing OTB simulation

Advantages of our Approach Reuse –knowledge base of plan fragments and beliefs supports reuse in new situations Modularity –agent-based architecture provides modularity of CGF plans and behaviors Composition –matching algorithms enable the matching of plan fragments and behaviors so they can be composed to form more intelligent adversaries and team mates, as situations warrant Verification –our representation formalism can be used for formal model- checking and verification of desirable properties of the software, thus reducing development time

Expected Impact If successful, our research will provide Reprogrammable and Instructable CGF teams which: –Can be “Coached” by training instructor using a simple GUI to provide trainee appropriate combat experiences –Exhibit realistic team behaviors –Considerably reduce development time and cost while increasing behavior realism –Can be embedded in larger simulations (e.g. OneSAF)

Accomplishments Developed initial Agent Representation Scheme Developed initial algorithm that matches current situation to previously developed plan fragments for reuse. Implemented initial teamwork scenario in Unreal Tournament. Publications: –Sycara, K. et al. “Integrating Agents into Human Teams”, In Salas E. (ed.) Team Cognition, Erlbaum Publishers, In Press. –Sycara K. et al. “Ontologies in Agent Architectures”, In S. Staab and R. Studer (eds.) Handbook on Ontologies in Information Systems, Springer In Press.

Hand Signal Behaviors Cover AreaListenWait 1234 Hand signals are important for team communication in urban warfare since the enemy is often in close proximity. Extensions to Gamebots allow AI control over these new behaviors.

Composition: L-Shaped Corridor and Room Clearing ABCD MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing ABCD MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing ACD B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing ACDB MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CDAB MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD B A MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD A B MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD B A MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD B A MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-Shaped Corridor and Room Clearing CD B A MCWP Military Operations on Urbanized Terrain (MOUT)

Composition: L-shaped Corridor + Stacked 2- Man Room Clearing

Milestones and Deliverables 4/30/03- 9/30/03 Develop initial scenarios for CGF deployment Develop initial agent teamwork representation Implement the initial scenarios in Unreal Tournament 10/01/03-12/30/03 Evaluate the resulting CGFs for realism Refine teamwork representation as a result 1/01/04 – 3/30/04 Develop techniques for agent behavior reuse Continue development and testing of teamwork schemes Implement them and test them in new situations

Milestones and Deliverables (2) 4/01/04- 6/30/04 Evaluate the resulting CGFs from previous quarter for realism and ease of development Develop and test mechanisms for agent behavior composition 7/01/04 – 9/30/04 Develop techniques for resolution of mismatches in agent descriptions Develop techniques for propagation of constraints across plans and agent beliefs 10/01/04 – 12/30/04 Implement techniques from previous quarter in Unreal Tournament and test in new situations Develop techniques for belief propagation across team members

Milestones and Deliverables (3) 1/01/05- 3/30/05 Develop indexing scheme for agent behaviors Develop techniques for dynamic retrieval of agent behaviors and reuse 4/01/05 – 6/30/05 Implement dynamic retrieval and reuse of agent behaviors in new situations Design and implement coach’s GUI 7/01/05 – 9/30/05 Test control of CGFs from coach’s GUI Demonstrate embedding of CGFs in OneSAF

Hand Signal Behaviors