1. CS 4730 AI and Pathfinding CS 4730 – Computer Game Design Some slides courtesy Tiffany Barnes, NCSU

2. CS 4730 AI Strategies Reaction vs. Deliberation When having the NPC make a decision, how much thought goes into the next move? How is the AI different in: –Frozen Synapse –Kingdom Hearts –Civilization –Halo 2

3. CS 4730 AI Strategies Reaction-Based –Fast, but limited capabilities Implementations –Finite-State Machines –Rule-Based Systems –Set Pattern 3

4. CS 4730 AI Strategies Deliberation-Based –Much slower, but more adaptable Implementations –A* / Dijkstra –Roadmaps –Genetic Algorithms 4

5. CS 4730 Deliberation Goal is to emphasize making the best possible decision by searching across all possibilities Thus, deliberation tends to use various search algorithms across data structures that contain the option space 5

6. CS 4730 “Sense – Plan – Act” Sense (or perceive) a sufficiently complete model of the world Plan by searching over possible future situations that would result from taking various actions Act by executing the best course of action Each possible outcome is effectively scored by a “metric of success” that indicates whether the choice should be taken or not 6

7. CS 4730 Deliberative vs. Reactive These are NOT mutually exclusive! You can have reactive policies for immediate threats –Incoming PC fire –Environmental destruction And you can have deliberative policies for long- term planning –Build orders and positioning 7

8. CS 4730 Core Questions How do you represent knowledge about the current task, environment, PC, etc? How do you find actions that allow the goal to be met? 8

9. CS 4730 Knowledge Representation A decision tree is a common way to represent knowledge The root node is the current state of the game state WRT the NPC / AI Thus, deliberative AI techniques are effectively versions of search and shortest-path algorithms! 9

10. CS 4730 Tic-Tac-Toe What is the decision tree for Tic-Tac-Toe? 10

11. CS 4730 The Goal State The objective of the decision tree model is to move from the initial game state (root of the tree) to the goal state of the NPC –What might a goal state be? When searching through the decision tree space, which path do we take? 11

12. CS 4730 Cost and Reward Every choice has a cost –Ammo –Movement –Time –Increase vulnerability to attack Every choice has a reward –Opportunity to hit PC –Capture a strategic point –Gain new resources 12

13. CS 4730 Minimax Algorithm Find the path through the decision tree that yields the best outcome for one player, assuming the other player always makes a decision that would lead to the best outcome for themselves 13

14. CS 4730 Naïve Search Algorithms Breadth-First Search –At each depth, explore every option at the next depth Depth-First Search –Fully explore one possible path to its “conclusion”, then backtrack to check other options What are the problems with these techniques in gaming? 14

15. CS 4730 Breadth-First Search Expand Root node –Expand all Root node ’ s children Expand all Root node ’ s grandchildren Problem: Memory size Root Child1 Child2 Root Child1 Child2 GChild1 GChild2 GChild3 GChild4

16. CS 4730 Uniform Cost Search Modify Breadth-First by expanding cheapest nodes first Minimize g(n) cost of path so far Root Child1 Child2 GChild1 9 GChild2 5 GChild3 3 GChild4 8

17. CS 4730 Depth First Search Always expand the node that is deepest in the tree Root Child1 GChild1 GChild2 Root Child1 Root Child1 GChild1

18. CS 4730 Adding a Heuristic Simple definition: a heuristic is a “mental shortcut” to ignore non-useful states to limit the search space and make the decision tree more reasonable What metrics might we use to determine “the value” of a potential option? What metrics might we use to determine “the cost” of a potential option? 18

19. CS 4730 Adding a Heuristic Creating an AI heuristic forms the basis of how the NPCs will behave –Will they ignore enemies that are farther than X away? –Will they avoid water? –Will they always move in the straightest path to the PC? 19

20. CS 4730 Cheaper Distance First! 20

21. CS 4730 Greedy Search Expand the node that yields the minimum cost –Expand the node that is closest to target –Depth first –Minimize the function h(n) the heuristic cost function

22. CS 4730 Greedy Search 22

23. CS 4730 Greedy Search 23

24. CS 4730 Greedy Search Greedy gives us (often) a sub-optimal path, but it’s really cheap to calculate! How can we improve on this? Add another aspect to the function – the cost of the node + the heuristic distance 24

25. CS 4730 A* A best-first search (using heuristics) to find the least-cost path from the initial state to the goal state The algorithm follows the path of lowest expected cost, keeping a priority queue of alternate path segments along the way 25

26. CS 4730 A* Search Minimize sum of costs g(n) + h(n) –Cost so far + heuristic to goal Guaranteed to work –If h(n) does not overestimate cost Examples –Euclidean distance

27. CS 4730 A* 27

28. CS 4730 A* 28

29. CS 4730 A* 29

30. CS 4730 Navigation Grid 30

31. CS 4730 Pathfinding in “real life” These algorithms work great when the game is grid based –Square grid –Hex grid For more “open” games, like FPSs: –Path nodes are placed on the map that NPCs can reach –Navigation mesh layers are added over the terrain –Often done automatically in advanced engines 31

32. CS 4730 Navigation Mesh Instead of using discrete node locations, a node in this instance is a convex polygon Every point inside a valid polygon can be considered “fair game” to move into Navigation meshes can be auto generated by the engine, so easier to manage than nodes Can also handle different sized NPCs by checking collisions 32

33. CS 4730 Navigation Mesh 33

34. CS 4730 Groups Groups stay together –All units move at same speed –All units follow the same general path –Units arrive at the same time Obstruction Goal

35. CS 4730 Groups Need a hierarchical movement system Group structure –Manages its own priorities –Resolves its own collisions –Elects a commander that traces paths, etc Commander can be an explicit game feature

36. CS 4730 Formations Groups with unit layouts –Layouts designed in advance Additional States –Forming –Formed –Broken Only formed formations can move

37. CS 4730 Formations Schedule arrival into position –Start at the middle and work outwards –Move one unit at a time into position –Pick the next unit with Least collisions Least distance –Formed units have highest priority Forming units medium priority Unformed units lowest

38. CS 4730 Formations 1 2 3 4 5 6 7 8 9 123 4 5 67 8 9 1 2 3 4 5 6 7 8 9 Not so good… Better…

39. CS 4730 Formations: Wheeling Only necessary for non-symmetric formations 123 4 5 1 2 3 4 5 Break formation here Stop motion temporarily Set re-formation point here

40. CS 4730 Formations: Obstacles 123 4 5 Scale formation layout to fit through gaps 12345 123 4 5 123 4 5 Subdivide formation around small obstacles

41. CS 4730 Formations Adopt a hierarchy of paths to simplify path- planning problems High-level path considers only large obstacles –Perhaps at lower resolution –Solves problem of gross formation movement –Paths around major terrain features

42. CS 4730 AI That Learns Imagine a player in Madden calls a particular play on offense over and over and over The heuristic values for certain states should change to reflect a more optimal strategy Now, the adjustment of heuristic values represents long-term strategy (to a degree) 42

43. CS 4730 AI That Evolves Neural networks add in a mutation mechanic Bayesian networks can also learn and add inferences How much processing power can we use for this? Is it better to truly have a “learning” AI, or should we just adjust some game parameters? 43