1. AI and Computer Games (informational session) Lecture by: Dustin Dannenhauer Professor Héctor Muñoz-Avila Computer Science and Eng.

2. AI and Game Design We want to understand what kinds of in-game tasks AI can perform This will help us understand what kinds of possibilities and limitations we can expect of AI when designing games Some games have failed because of unrealistic expectations - amount of effort required to create good AI or even its feasibility –E.g., the poor AI of the automated companion in the Daikatana game we discussed before In this lecture we will focus on “what” instead of “how” –Recommended courses to learn about “how”: CSE 348 AI Game Programming CSE 327 AI Theory and Practice

3. What is AI? Systems that think like humans Systems that think rationally Systems that act like humans Systems that act rationally Categories for definitions of AI

4. Pathfinding (1) Red team versus blue team Player controlling red team sets blue flag as destination Pathfinding can be used to plan the movement (1) of red units automatically Pathfinding is a well understood problem

5. Pathfinding Frequently games use a collection of nodes (“waypoints”) and edges connecting those nodes Algorithms are devised that find a path through the waypoints for two given nodes Challenges arise when there are too many waypoints

6. Memory Available The amount of memory available has increased Speed to process this memory has also increased

7. Taken Advantage of Memory Available Pre-compute all possible paths (figure from AI Game Programming Wisdom) So finding any (start,destination) path can be done quickly This is why pathfinding works so well in many commercial games If map changes (e.g., “terraforming”), an AI technique called “heuristic search” can be used. But this is used localized in a region to run efficiently

8. Pathfinding in Games Many examples: Most RTS use standard “point and click” command telling units where to go NPCs in RPGs can follow the player –Although not “perfect”: See video (9:40-10:10). Amazon (avatar) is followed by the lance mercenary (NPC). Sometimes the mercenary falls behindvideo NPCs in FPS games can navigate by themselves in complex environments (e.g., human vs AI death match in Unreal Tournament) –See video FFA, frag count, 16 players (only one is human)video

9. Planning Planning: automated generation of a sequence of actions to achieve a goal Example of an action: Example of a plan controlling an NPC Attack positioned Weapon ready Enemy attacked preconditions effects Take-cover Load-weaponattack

10. Planning in Games NPCs in FEAR are controlled using planning (video on image) 40:00-1:40 NPCs takes cover, flank One could have created an FSM for this same behavior But for play variety purposes we will need to create many FSMs Planning can generate these behaviours automatically –This is what FEAR does

11. What-If Analysis (1) What-if analysis to counter the opponent Red team is controlled by AI The AI can analyze and predict what blue will do and counter- plan accordingly In this example: red sends force (2) to delay blue This works only on small spaces (maps) (2)

12. Minimax Idea: Build a tree describing all possible gaming moves between two opponents Analyze tree looking ahead N moves into the “horizon” and select best move Well understood: Nash equilibrium Challenges:  game tree can be too large Ways around:  Parallelism & heuristics: Deep Blue  Abstraction

13. What-if Analysis in Games Used in some Poker playing games –Because tree is so large, state abstraction is needed (e.g., grouping hands that have similar expected strength) Demonstrated in RTS games for micromanagement  Mini-battle between small groups of opposing units  Many such encounters take place in a match  Proficiency in these mini- battles is crucial

14. Machine Learning (1) Suppose red is a automated player Machine learning enables an automated player to learn Suppose blue had strong defenses along path (1) As a result, the approach (1) fails The automated player learns form this and tries path (3) (3)

15. Machine Learning Multiples ways to learn in games: –Learn by observing other players –Learn by processing gameplay logs –Learn while playing Why machine learning in games? Among others: –Self-correction Automatically fixing exploits –Creativity Responding intelligently to new situations –Scalability Better entertainment for strong players Better entertainment for weak players

16. Recall: Domination Games A number of fixed domination locations. When a team member steps into one of these locations, the status of the location changes to be under the control of his/her team. The team gets a point for every five seconds that each domination location remains under the control of that team. The game is won by the first team that gets a pre-specified amount of points. We used Unreal Tournament© a team-based FPSUnreal Tournament

17. Before/After Learning Before (video) After After (video) Bad: NPCs concentrate on one flag Better: NPCs focus on two flags

18. Machine Learning in Games Player trains a creature The creature learns from player’s feedback: –If player wants to reward creature’s actions  scratches creature's belly –If player wants to punish creature’s actions  slap create Over time creature behaves consistently with feedback received (video linked from figure)

19. Final remarks AI has been successfully applied in a number of games –We have mentioned some: Fear, Black and White Aside from heuristic pathfinding techniques, use of AI has been limited in commercial games Some reasons for this: –Game designers are worried about the AI doing unexpected (e.g., “silly”) things –Lack of understanding about its capabilities –Historic limitations in computing power (this is becoming less of an issue) But there are a lot of exciting possibilities: videovideo