1. Games as Game Theory Systems (Ch. 19)

2. Game Theory It is not a theoretical approach to games Game theory is the mathematical study of decision making –Origins in the field of economics –Examples of “games”: Bankruptcy Investment It concerns with how to make decisions: –Idea: reason with the consequences of decisions (i.e., choosing a game action) madeconsequences Contrary to the book claim’s game theory techniques have been used to build successful game playing algorithms –Specifically for turn-based games

3. Game Trees Two players (MAX and MIN) alternate moves (turn-based) The tree indicates all possible movements by the players The root of the tree indicates the starting state The children of the root indicate possible moves by MAX The children of the children indicate possible moves by MIN Tree continues alternating until it reaches end-game situations

4. … First example MAX moves … MIN moves (which depend on MAX’s last move)

5. Game Trees (2) Concepts: State: node in tree Terminal node: game over Utility function: value for outcome of the game. MAX: 1 st player, maximizing its own utility MIN: 2 nd player, minimizing Max’s utility Utility : 1  Max won 0  Max and Min draw -1  Max lost (Min won)

6. Minimax Finding perfect play for deterministic, perfect information games Idea: choose move to position with: highest utility for MAX = best achievable payoff against a rational opponent Example: Utility is a number between 2 and 14

7. Properties of Minimax Always find an optimal strategy for MAX (or MIN)? Size of game tree? –b: branching factor (maximum number of moves a player can make in his/her turn) –m: # turns in an average game Yes (if tree is finite and opponent is rational) approximately b m For Tic-Tac-Toe, b ≈ 9, m ≈10. Thus, the size of the tree is 9 10. Any computer nowadays will find the optimal strategy:  Always draw against a rational opponent For chess, b ≈ 35, m ≈60. Thus, the size of the tree is 35 60. Too large for computers today. Therefore tree can only be partially constructed. Look ahead of 6 turns for most computers Evaluation functions are used to measure utility of non- terminal states looking ahead

8. Evaluation Function –Is an estimate of the actual utility –Typically represented as a linear function: EF(state) = w 1 f 1 (state) + w 2 f 2 (state) + … + w n f n (state) –Example: Chess weight: Piece  Number  (w 1 ) Pawn  1  (w 2 ) Knight  3  (w 3 ) Bishop  3  (w 4 ) Rook  5  (w 5 ) Queen  9 Function; state  Number  f 1 = #(pawns,w)  #(pawns,b)  f 2 = #(knight,w)  #(knight,b)  f 3 = #(bishop,w)  #(bishop,b)  f 4 = #(rook,w)  #(rook,b)  f 5 = #(knight,w)  #(knight,b)

9. Example: Evaluation Function “all things been equal” White moves, Who is winning? Is this consistent with Evaluation function? Black Yes!

10. Evaluation Function (2) Obviously, the quality of the AI player depends on the evaluation function Conditions for evaluation functions:  If n is a terminal node,  Computing EF should not take long  EF should reflect chances of winning EF(n) = Utility(n) If EF(state) > 3 then is almost-certain that blacks win

11. Horizon: Cutting Off Search When to cutoff minimax expansion? Potential problem with cutting off search: Horizon problem Solution:  Fixed depth limit  Iterative deepening until times runs out  Decision made by opponent is damaging but cannot be “seen” because of cutoff  Quiescent: states that are unlikely to exhibit wild swings in the values of the evaluation functions

12. Example: Horizon Problem “all things been equal” White moves, Who is winning? Is this consistent with Evaluation function? White No!

13. In practice Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, 24 processors, heuristics to guide construction of game tree with help of human grand masters Othello: human champions refuse to compete against computers. Computers are too good. Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

14. Saddle Points –Saddle point: Developing a strategy for playing the game that ensures victory (or non defeat) every time Example: Tic-Tac-Toe –Results in loss of meaningful play It is a theoretical possibility but if in practice can’t figured it out then it is ok But sometimes is discovered by players as emergent behavior –“Exploits” Victory! … … …

15. Summoning Necromancer Some Design principles: –Rather weak character (not much of direct damage; can’t sustain a lot of damage) –Uses curses to weaken foe’s attacks/defenses –Killed monsters can be revived to serve the necromancer But they last 2 minutes only (negative feedback!) –Reviving monsters, casting curses cost mana Mana is limited and renews itself very slowly (negative feedback!) http://www.youtube.com/watch?v=UJdB-ydfUho Saddle point: people killed shamans mobs which could revive other mobs! (positive feedback)

16. Recent Research Use Minimax in RTS games In such games there are frequent battles taken place between small armies Who should attack who? –What happens if X attacks A first and then B –Easily represented as a minimax tree