1. Evolving Agents in a Hostile Environment Alex J. Berry

2. Overview Motivation Background The Approach Map Generation Evolutionary Algorithm Experiments

3. Training First Responders VEnOM Labs is developing a suite to train First Responders Is the training effective? How can we make the training more effective?

5. Goal To develop a system to allow for friendly and hostile AI agents in the training environment. To develop a system to create better agents for training First Responders.

6. Simulation of Adaptive Agents in a Hostile Environment [HW95] Thomas Haynes Simple Agents Mines and Energy Experiments  Single Agent, Static and Random Environment  Multiple Agent, Static and Random Environment

7. The Complicator Aliases: Dr. T, Dr. Tauritz Input: 2+2=4 Output:

8. The Approach Randomly Generated Grid Environment Three Types of Agents:  First Responders  Terrorists  Victims Genetic Programming to Evolve the Agents

9. Randomly Generated Maps Any Dimension Percentage walls Bit Array to Hold the Data

10. Demo

11. What’s an Agent to do? Victims  Move Randomly  Remember Things  Forget Things  Survive Terrorists  Kill Victims  Kill First Responders  Lay Traps  Disguise Themselves  Not Get Caught First Responders  Help Victims  Find and Disarm Traps  Survive  Catch Terrorists

12. Evolutionary Algorithm Two Agents to Evolve  First Responder  Terrorist Two Competing Evolving Populations Genetic Programming for the Evolutionary Implementation

13. What An Individual Looks Like Terminals  Current Grid Location (C)  Surrounding Grid Locations (S)  Rand (R) Non-Terminals  If-Then-Else Threat And, Or, Not Victim, First Responder, Terrorist, Trap Valid Move Actions  Save  Kill  Move  Place Trap  Remove Trap  Not (Action) to invert an Action

14. Sample Individuals Move

15. Genetic Programming Evaluation First Responder  Victims Helped  Terrorists Caught  Traps Found  Traps Removed  Survival  Amount of the Map explored Terrorist  Kills using Traps  Kills on Contact  Effective Disguises  Amount of Time Survived

16. Experiments Static Environment Evolution Random Environment Evolution Varying Ratios of First Responders, Victims, and Terrorists Evolving one Population at a Time

17. Summary Looking at agents operating in a hostile environment.  First Responders, Terrorists, and Victims Evolving first responders and terrorist using genetic programming techniques.

18. Future Work and Questions Other Evolutionary Approaches  LCS  GP-LCS Hybrid Integration into a 3D environment Playable Human Mode Representations of Real Buildings Test effectiveness of adding this to Affective Intensity Experiment Integration of other types of Traps and sensors to detect those traps