1. A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft Santiago Ontanon, Gabriel Synnaeve, Alberto Uriarte, Florian Richoux, David Churchill, Mike Preuss Coffee Meeting October 2014

2. Economy – Gathering resources – Building a base Military – Training units – Researching technologies Traits – Simultaneous – Real Time – Partially observable (Fog of War) – Non-deterministic E.x. Starcraft, Command and Conquer, Total Annihilation What is a Real Time Strategy game?

3. Game Complexity and State Space – Checkers 10^31, Chess 10^47 – For Starcraft, with naive assumptions Assume 200x200 tiled game map Assume, each player has maximum 30 units + buildings Assume Each unit and building has no internal states (skills and abilities) 16384^400 = 10^1685 Excellent for testing real-world AI – High dimensionality ( compute real-world state space) – There is no dominant strategy ( no optimal solution in our world) – No sensor problems, finite worlds, limited parameters, other players are humans Why should we be interested?

4. Developing AI Bots for Starcraft No single solution exists yet, approaches break the RTS AI Problem into sub problems (Task Decomposition) Strategy Terrain Analysis Reconnaissance Tactics Reactive Control Micro Macro Early Game - Usually build-orders and scouting your enemy Mid Game – Obtaining the map control, and countering strategies Late Game – Controlling all resources, high tech units, strategies with depleted resources

5. State of the Art Bots Resource Management Decision Making under Uncertainity SpatioTemporal Reasoning Opponent Modeling Tactics are generated with FiniteStateMachines Mostly scripted and hardcoded strategies (rush, expand base, build invisible units) Huge codebases, many developing modules... But still thousands of hour programmer effort < 1 hour of progamer exploitation (Bakaryu vs Skynet 3.30-4.30) Challenges BroodwarBotQ Architecture, Synnaeve et al.

6. State of the Art Bots Starcraft Bot Ladder ( 2010-2014) How can bots learn from each other? How can we utilize reinforcement learning for developing strategies?

7. Solved/Open Problems of RTS AI Unit Pathfinding Given source and destination, how to optimally move a unit? Solved by optimized A* variants. Unit Microing in Combat Avoiding enemy fire, attack prioritization, chasing units, retreating, using choke points Efficient Learning Techniques. In chess, game tree search strategies, in RTS not possible. Learning Strategies. How to discover new strategies/patterns from game replays that provide substrantial amount of information? Tactical Adaptation. How to adapt what is happening in the game? (e.x. Observing opponent’s expansion bases, should strengthen military power) Synnaeve et al. INRIA, CNRS) Walling tactic from a human player to close the entrance of the base. AI should adapt to this by building dropships to drop units inside the base.

8. Final Comments Thank you & Questions?