1. © HVR Consulting Services Ltd AI for OOTW Representing Plausible Behaviour in OOTW Simulators

2. © HVR Consulting Services Ltd Steve Curram Mirek Gliniecki Modelling and Analysis Group HVR Consulting Services Ltd Selborne House, Mill Lane Alton, Hampshire GU34 2QJ steve.curram@hvr-csl.co.uk Mark Cusack Nick Exon Distributed Technologies Group QinetiQ Ltd St Andrews Road Malvern, Worcestershire WR14 3PS cusack@signal.qinetiq.com

3. © HVR Consulting Services Ltd Background Purpose of overall study is to create simulators that allow stress testing of Command Information Systems and the people who use them in an OOTW situation Features: Multiple factions and agencies Multiple factions and agencies Highly political environment Highly political environment Restrictive RoE Restrictive RoE Media coverage and public opinion Media coverage and public opinion Multiple information sources Multiple information sources

4. © HVR Consulting Services Ltd Background (2) Purpose of simulators for use by commanders is to provide a challenging and thought provoking environment Purpose of simulators for use by commanders is to provide a challenging and thought provoking environment The simulators are not designed to be analysis tools The simulators are not designed to be analysis tools They will conform to the High Level Architecture (HLA) using QinetiQ’s BeanXS environment They will conform to the High Level Architecture (HLA) using QinetiQ’s BeanXS environment Use Java and JavaBeans™ technology Use Java and JavaBeans™ technology AI is only one component of the study AI is only one component of the study Quick and cheap to set up Quick and cheap to set up

5. © HVR Consulting Services Ltd Study Approach Identification of behaviour types and requirements of AI for OOTW simulators Identification of behaviour types and requirements of AI for OOTW simulators Review of technology and case studies to generate shortlist of approaches for further investigation Review of technology and case studies to generate shortlist of approaches for further investigation Develop stand-alone AI objects for analysis Develop stand-alone AI objects for analysis Test AI objects in an integrated environment Test AI objects in an integrated environment Prototype in a small simulation model Prototype in a small simulation model

6. © HVR Consulting Services Ltd Aims of the AI Provide plausible behaviour in simulation from a player’s perspective Provide plausible behaviour in simulation from a player’s perspective –computer generated actors –occurrence of events Give player something to think about Give player something to think about Be responsive to player’s actions Be responsive to player’s actions Not to be too transparent - player should be immersed in environment rather than trying to beat the AI Not to be too transparent - player should be immersed in environment rather than trying to beat the AI

7. © HVR Consulting Services Ltd It is more important that behaviour appears plausible to a player rather than for the mechanism to conform to any psychological theories It is more important that behaviour appears plausible to a player rather than for the mechanism to conform to any psychological theories The AI should be reasonably quick and easy to set up The AI should be reasonably quick and easy to set up Probably more akin to the requirements for commercial computer games than traditional military models Probably more akin to the requirements for commercial computer games than traditional military models It’s Artificial Intelligence, BUT

8. © HVR Consulting Services Ltd Approaches Considered Random Sampling - does not take account of conditions, though not entirely predictable - could be combined with other approaches Random Sampling - does not take account of conditions, though not entirely predictable - could be combined with other approaches Knowledge-Based Systems (& related) - can be too predictable if rules are simple, while ease of setting up falls rapidly as rules become more complex Knowledge-Based Systems (& related) - can be too predictable if rules are simple, while ease of setting up falls rapidly as rules become more complex Fuzzy Logic - rules are generally simpler but more powerful than KBSs, needs some experience in choice of fuzzy regions, linguistic terms can help trainers in tweaking scenarios Fuzzy Logic - rules are generally simpler but more powerful than KBSs, needs some experience in choice of fuzzy regions, linguistic terms can help trainers in tweaking scenarios Case-Based Reasoning - full systems can be time consuming to set up and have overhead of a shell, simpler versions may be useful for recognising when scripted events can occur Case-Based Reasoning - full systems can be time consuming to set up and have overhead of a shell, simpler versions may be useful for recognising when scripted events can occur

9. © HVR Consulting Services Ltd Approaches Considered (2) Finite State Machines - commonly used in computer games, simple to set up, can get predictable for player Finite State Machines - commonly used in computer games, simple to set up, can get predictable for player Fuzzy State Machines - adaptation of Finite State Machines using fuzzy rules, becoming popular in computer games, membership functions could be used for random sampling Fuzzy State Machines - adaptation of Finite State Machines using fuzzy rules, becoming popular in computer games, membership functions could be used for random sampling Bayesian Belief Networks - powerful representation of factors on behaviour, can be difficult to set up, difficulty of validation Bayesian Belief Networks - powerful representation of factors on behaviour, can be difficult to set up, difficulty of validation Neural Networks (various types) - adaptive, powerful representation but time consuming and difficult to set up and validate Neural Networks (various types) - adaptive, powerful representation but time consuming and difficult to set up and validate Genetic Algorithms - adaptive, but too slow for interactive simulation Genetic Algorithms - adaptive, but too slow for interactive simulation

10. © HVR Consulting Services Ltd Short-listed Approaches Finite State Machine (FSM) Finite State Machine (FSM) –for simple behaviour –create generic FSM object to which states and transitions are added –easy to set up

11. © HVR Consulting Services Ltd Short-listed Approaches Fuzzy State Machine (FuSM) Fuzzy State Machine (FuSM) –for more complex behaviour –linguistic or numerical inputs –smoother transitions between states –create generic FuSM to which terms and fuzzy rules can be added –state can be selected from centre of gravity or using random sampling from fuzzy region

12. © HVR Consulting Services Ltd Implementation Considerations Classes developed in Java and converted to JavaBeans™ Classes developed in Java and converted to JavaBeans™ Will be used in simulations along with other JavaBeans™ Will be used in simulations along with other JavaBeans™ They need to pick up messages and hold their own state They need to pick up messages and hold their own state Need to respond to a request or simulation time Need to respond to a request or simulation time Behaviour should be able to be time-scaleable Behaviour should be able to be time-scaleable

13. © HVR Consulting Services Ltd Fuzzy State Machine Fuzzy State Machine and test rig (shown) developed Fuzzy State Machine and test rig (shown) developed Allows absolute rules - outcome is state membership values Allows absolute rules - outcome is state membership values Allows change rules - outcome is amount of change to current state value Allows change rules - outcome is amount of change to current state value States and rules can be read- in using a scripting language States and rules can be read- in using a scripting language JavaBean version developed JavaBean version developed Wrapper/Interpreter allows FuSMBean to interact with rest of simulation Wrapper/Interpreter allows FuSMBean to interact with rest of simulation

14. © HVR Consulting Services Ltd Behaviour Type - Attitude Attitude of an actor towards other actors Attitude of an actor towards other actors –may differ at different levels, e.g. faction leaders towards UN, or faction soldier towards UN –use Finite State Machine or Fuzzy State Machine Neutral Mistrustful Neutral Mistrustful

15. © HVR Consulting Services Ltd Behaviour Type - Negotiation Negotiation between actors Negotiation between actors –manual: player has role as a mediator, and suggests compromise solutions –automatic: compromise solutions based on global approach to negotiation settings –response of actors depends on the weighted distance from their ideal solution and their attitude towards the negotiations –makes use of Fuzzy State Machines –likely to be iterative process over time (may be short or long periods)

16. © HVR Consulting Services Ltd Behaviour Type - Negotiation

17. © HVR Consulting Services Ltd Vignette Development (WIP) Two warring factions Two warring factions Refugees from both population groups Refugees from both population groups Refugee camps for both groups Refugee camps for both groups –populations, food, medicine, death rates UN convoys supplying camps UN convoys supplying camps Road blocks run by militia factions Road blocks run by militia factions Media coverage Media coverage Includes: crowd movement, terrain, path- finding, attitude, negotiation Includes: crowd movement, terrain, path- finding, attitude, negotiation

18. © HVR Consulting Services Ltd Observations So Far Generic and flexible FSM and FuSMs can be created and converted to JavaBeans ™ Generic and flexible FSM and FuSMs can be created and converted to JavaBeans ™ Wrappers control interaction with other JavaBeans ™ in the simulation Wrappers control interaction with other JavaBeans ™ in the simulation Specific behaviour loaded in from XML Specific behaviour loaded in from XML Types of behaviour can be stored in libraries Types of behaviour can be stored in libraries Experimenting with messaging Experimenting with messaging Refining rules - want sufficient power with simplest set possible Refining rules - want sufficient power with simplest set possible

19. © HVR Consulting Services Ltd Questions ?