1. Game Playing and Game Programming (Chapter 6)
BUSI 0088 Handout 5 - Game Playing and Game Programming

2. BUSI 0088 Handout 5 - Game Playing and Game Programming
Outline
Strategies
The minimax algorithm
alpha-beta pruning
Expectiminimax
Applications of AI in game programming
BUSI 0088 Handout 5 - Game Playing and Game Programming

3. Game vs Search Problems
“Unpredictable” opponent”  solution is a strategy specifying a move for every opponent play
Because of time limits, it is unlikely to find all possibilities. Must approximate based on algorithms.
BUSI 0088 Handout 5 - Game Playing and Game Programming

4. BUSI 0088 Handout 5 - Game Playing and Game Programming
Types of Games
Single player vs two players vs multiple players
Zero-sum vs non-zero-sum
Perfect information vs imperfect information
Deterministic vs chance
BUSI 0088 Handout 5 - Game Playing and Game Programming

5. Some Examples of 2-player Games
deterministic
chance
chess, checkers, go, tic-tac-toe
monopoly, backgammon
bridge, poker, scrabble
perfect information
imperfect information
BUSI 0088 Handout 5 - Game Playing and Game Programming

6. BUSI 0088 Handout 5 - Game Playing and Game Programming
Game Trees
Considering games with 2 players called MAX and MIN. MAX moves first, and then they take turns moving until the game is over.
A game can be formally defined as a search problem with the following components:
Initial state: the board position and identifies the player to move.
Successor function: a list of (move, state) pairs
Terminal test: which determines when the game is over
Utility function: Gives a numeric value for the terminal states. For example, in chess, the outcome is win, loss, or draw, which can be given a value 1, -1, or 0.
BUSI 0088 Handout 5 - Game Playing and Game Programming

7. BUSI 0088 Handout 5 - Game Playing and Game Programming
Game Trees
The initial state and the legal moves for each side define the game tree for the game.
Consider the game of tic-tac-toe. From the initial state, MAX has nine possible moves.
MAX and MIN take turns until we reach leaf nodes corresponding to terminal states — one player has three in a row or all the squares are filled.
BUSI 0088 Handout 5 - Game Playing and Game Programming

8. Game Tree for Tic-Tac-Toe
*The utility values are for MAX
BUSI 0088 Handout 5 - Game Playing and Game Programming

9. Game Tree for a Trivial 2-ply Game
MAX moves first and can choose the path a1, a2, or a3. MIN then makes the second move. The utility values are the values for MAX.
BUSI 0088 Handout 5 - Game Playing and Game Programming

10. BUSI 0088 Handout 5 - Game Playing and Game Programming
Optimal Strategy
What is the optimal strategy?
Roughly speaking, an optimal strategy leads to the best outcome assuming one is playing against the best opponent.
BUSI 0088 Handout 5 - Game Playing and Game Programming

11. BUSI 0088 Handout 5 - Game Playing and Game Programming
Minimax
Given a game tree, the optimal strategy can be determined by examining the minimax value of each node
Consider zero-sum games: MAX’s heaven is MIN’s hell.
The minimax value of a node is the utility (for MAX) of being in the corresponding state, assuming that both players play optimally from there to the end of the game.
Obviously, the minimax value of a terminal state is just its utility. At other nodes, MAX will prefer to move to a state of maximum value, whereas MIN prefers a state of minimum value.
BUSI 0088 Handout 5 - Game Playing and Game Programming

12. BUSI 0088 Handout 5 - Game Playing and Game Programming
Minimax
MINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node
maxsSuccessors(n) MINIMAX-VALUE(s) if n is a MAX node
minsSuccessors(n) MINIMAX-VALUE(s) if n is a MIN node
BUSI 0088 Handout 5 - Game Playing and Game Programming

13. Minimax Example with the 2-ply game
The minimax value at B = min(3, 12, 8) = 3
The minimax value at C = min(2, 4, 6) = 2
The minimax value at D = min(14, 5, 2) = 2
The minimax value at A = max(3, 2, 2) = 3

14. BUSI 0088 Handout 5 - Game Playing and Game Programming
The Minimax Algorithm
Uses a simple recursive computation of the minimax values of each successor state by performing a complete depth-first search of the game tree.
Chooses the move with the highest minimax value.
BUSI 0088 Handout 5 - Game Playing and Game Programming

15. BUSI 0088 Handout 5 - Game Playing and Game Programming
The Minimax Algorithm
If the maximum depth of the tree is m and there are b legal moves at each node, the time complexity of the minimax algorithm is O(bm)
For real games, the time cost is impractical, e.g., in chess, b  35 and m  100
Suppose the computer has 100 seconds to “think”, and it can explore 10,000 nodes per second  106 nodes per move
bm = 106, b = 35  m = 4
4-ply  human novice
8-ply  typical PC, human master
12-ply  Deep Blue, Kasparov
BUSI 0088 Handout 5 - Game Playing and Game Programming

16. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-Beta Pruning
To compute the correct minimax decision without looking at every node in the game tree.
Prunes away branches that cannot possibly influence the final decision.
Always returns the same move as the standard minimax.
BUSI 0088 Handout 5 - Game Playing and Game Programming

17. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
BUSI 0088 Handout 5 - Game Playing and Game Programming

18. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
BUSI 0088 Handout 5 - Game Playing and Game Programming

19. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
BUSI 0088 Handout 5 - Game Playing and Game Programming

20. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
BUSI 0088 Handout 5 - Game Playing and Game Programming

21. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
BUSI 0088 Handout 5 - Game Playing and Game Programming

22. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
Good ordering of moves can improve effectiveness of pruning.
With “perfect ordering”, time complexity = O(bm/2)
Double depth of search
Can easily reach depth 8 and play good chess
BUSI 0088 Handout 5 - Game Playing and Game Programming

23. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
Why is it called Alpha-Beta Pruning (- pruning)?
 = the value of the best choice we have found so far for MAX
 = the value of the best choice we have found so far for MIN
Alpha-beta search updates the values of  and  as it goes along and prunes the remaining branches at a node as soon as the value of the current node is known to be worse than the current  or  value for MAX or MIN, respectively.
BUSI 0088 Handout 5 - Game Playing and Game Programming

24. BUSI 0088 Handout 5 - Game Playing and Game Programming
Alpha-beta Pruning
In general, if m is better than n for the player, we will never get to n in play.
BUSI 0088 Handout 5 - Game Playing and Game Programming

25. Nondeterministic Games
In many games there are chances introduced by dice, card-shuffling, etc.
A simple example with coin-flipping:
BUSI 0088 Handout 5 - Game Playing and Game Programming

26. BUSI 0088 Handout 5 - Game Playing and Game Programming
Expectiminimax
Expectiminimax gives optimal play.
Just like minimax, except that we must also handle chance nodes.
EXPECTIMINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node
maxsSuccessors(n) EXPECTIMINIMAX-VALUE(s)
if n is a MAX node
minsSuccessors(n) EXPECTIMINIMAX-VALUE(s)
if n is a MIN node
sumsSuccessors(n) P(s)  EXPECTIMINIMAX-VALUE(s)
if n is a chance node
P(s) = the probability that s will be chosen based on the chance node
BUSI 0088 Handout 5 - Game Playing and Game Programming

27. BUSI 0088 Handout 5 - Game Playing and Game Programming
State-of-the-Art
Checkers
In 1952, Arthur Samuel of IBM developed a checkers program that learned its own evaluation function by playing itself thousands of times. Starting as a novice, it improved very well after a few days and defeated Samuel. 10 years later it defeated Robert Nealy, a human champion. Samuel’s computer had 10,000 words of main memory, magnetic tape for long-term storage, and a MHz processor.
In 1994, the human world champion was defeated by a computer defining perfect play for all possible positions involving 8 or fewer pieces on board (around 443 billion positions).
BUSI 0088 Handout 5 - Game Playing and Game Programming

28. BUSI 0088 Handout 5 - Game Playing and Game Programming
State-of-the-Art
Chess: In 1997, Deep Blue, developed at IBM, defeated Kasparov in a six-game exhibition match. Deep Blue searches 200 million positions per seconds, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply.
Othello: Defeated human world champion by six games to none. It is generally acknowledged that humans are no match for computers at Othello.
Go: The board is 19x19, making it too daunting for regular search methods (b > 300). There are some better programs these days.
BUSI 0088 Handout 5 - Game Playing and Game Programming

29. Other Things to Consider
We have only considered two-player zero-sum game. Game theory can be applied in non-zero-sum games and multiplayer games. Collaboration (teaming up) may be needed in multiplayer games.
Risk analysis: how much risk should the computer take?
High level reasoning or planning can eliminate the search space significantly.
BUSI 0088 Handout 5 - Game Playing and Game Programming

30. Other AI Applications in Game Programming
Finite State machines
Behaviors of computer characters
Prediction and Learning
Artificial Life
BUSI 0088 Handout 5 - Game Playing and Game Programming

31. BUSI 0088 Handout 5 - Game Playing and Game Programming
State Machines
Finite State Machines (FSMs) are the most commonly used techniques in AI game programming.
FSM is a concise, nonlinear description of how an object can change its state over time.
BUSI 0088 Handout 5 - Game Playing and Game Programming

32. BUSI 0088 Handout 5 - Game Playing and Game Programming
Finite State Machines
Example of a simple FSM for a computer-programmed opponent (CPO)
Enemy in sight
Fight
Walk around
Enemy dead
Food in sight
Finish eating
Low energy
Cornered
Flee
Eat Food
Food in sight
BUSI 0088 Handout 5 - Game Playing and Game Programming

33. BUSI 0088 Handout 5 - Game Playing and Game Programming
Finite State Machines
Finite State Machines can be improved by using randomness and probabilities  Fuzzy State Machines
BUSI 0088 Handout 5 - Game Playing and Game Programming

34. BUSI 0088 Handout 5 - Game Playing and Game Programming
Behavior
Pathfinding is often needed in game programming. E.g., asking a CPO (computer-programmed opponent) or NPC (non-player character) to go to a specific position on the map.
Usually based on A* search
Need to avoid obstacles
Need to make sure the entities are not trapped in loop
Need to control the entities to move together (flocking)
BUSI 0088 Handout 5 - Game Playing and Game Programming

35. BUSI 0088 Handout 5 - Game Playing and Game Programming
Behavior
The characters need to be aware of:
The player (the most important!)
Environment (don’t walk through a wall)
Team management (ensure that the members are working as a team)
BUSI 0088 Handout 5 - Game Playing and Game Programming

36. BUSI 0088 Handout 5 - Game Playing and Game Programming
Behavior
Behavior can be goal-directed (using the agent concept).
For example, in Age of Empires II, the following rule was used in the AI Script by the CPO of the game:
(defrule
(goal resource-needed WOOD)
(current-age == dark-age)
(civilian-population >= 10)
(not (strategic-number sn-wood-gatherer-percentage = 40) )
=>
(set-strategic-number sn-wood-gatherer-percentage 40)
(set-strategic-number sn-food-gatherer-percentage 60) )
BUSI 0088 Handout 5 - Game Playing and Game Programming

37. BUSI 0088 Handout 5 - Game Playing and Game Programming
Behavior
Sometimes the computer can’t be too tough; otherwise the player will never win.
Artificial stupidity is often introduced, usually in the easy mode of the game.
Missing the first attack
Bad aiming skills
Warning the player before attacking
Not attacking at the same time
BUSI 0088 Handout 5 - Game Playing and Game Programming

38. Prediction and Learning
Needs to predict what the users will do next (e.g., punching in a fighting game, passing or shooting the ball in soccer, running in a certain direction).
In the past, many computer games cheat by having more resources (e.g., Age of Empires) or allowing illegal moves (e.g., Street Fighter). This is less appreciated by players now and more sophisticated AI techniques are used in games.
BUSI 0088 Handout 5 - Game Playing and Game Programming

39. Prediction and Learning
For example, in a street fighting game, the player’s actions can be accumulated into a move history. It is possible to record the player’s movement for statistical prediction. If the following statistics are obtained, then we can better predict the player’s moves.
Player sequence Occurrences Frequency
Low Kick, Low Punch, Uppercut %
Low Kick, Low Punch, Low Punch %
Low Kick, Low Punch, Low Kick %
Virtua Fighter 4 uses a similar technique.
BUSI 0088 Handout 5 - Game Playing and Game Programming

40. Prediction and Learning
In recent years, there are more AI techniques (like neural networks, genetic algorithms, reinforcement learning) being used for learning in game programming.
Some examples
Creatures (GA and NN)
Cloak, Dagger, and DNA (GA, can be freely downloaded at:
Black & White (NN)
Colin McRae Rally 2.0 (NN)
World Championship Snooker 2003 (NN)
Fields of Battle (simulated annealing and NN)
BUSI 0088 Handout 5 - Game Playing and Game Programming

41. BUSI 0088 Handout 5 - Game Playing and Game Programming
Artificial Life
Making artificial lives in computers.
Examples:
Princess Maker
The Sims
Petz (the Dogz and Catz screensavers).
Creatures
Theme Hospital
Each A-life has his own goals, behaviors, personalities, emotions, and even a family and career.
BUSI 0088 Handout 5 - Game Playing and Game Programming

42. BUSI 0088 Handout 5 - Game Playing and Game Programming
Artificial Life
In The Sims, even the objects have their own behaviors. For example, a smart microwave oven knows what it can accomplish (cook food) and how it should be used (open door, place food inside, close door, …).
Agents can use the objects with which they were never programmed to interact.
BUSI 0088 Handout 5 - Game Playing and Game Programming