Turn-Based Games Héctor Muñoz-Avila sources: Wikipedia.org Russell & Norvig AI Book; Chapter 5 (and slides) My own

Turn-Based Strategy Games Early strategy games was dominated by turn- based games Derivate from board games Chess The Battle for Normandy (1982) Nato Division Commanders (1985) Turn-based strategy: game flow is partitioned in turns or rounds. Turns separate analysis by the player from actions “harvest, build, destroy” in turns Two classes: Simultaneous Mini-turns

Turn-Based Games Continues to be A Popular Game Genre At least 3 sub-styles are very popular: –“Civilization”-style games Civilization IV came out last week –Fantasy-style (RPG) Heroes of Might and Magic series –Poker games Poker Academy

Some Historical Highlights 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program 1956 Maniac versus Human 1970 Hamurabi. A game about building an economy for a kingdom The Battle for Normandy (1982) 1987 Pirates! 1990 Civilization 1995 HoMM 1996 Civilization II The best game ever? … 2005 Civilization IV 2006 HoMM V

Side-tracking: Game Design: Contradicting Principles Principle: All actions can be done from a single screen. Classical example: Civ IVCiv IV But: HoMM uses two interfaces: HoMM IVHoMM IV

Coming back: How to Construct Good AI? Idea: Lets just use A* and define a good heuristic for the game  Search space: a bipartite tree  After all didn’t we use it with the 9-puzzle game? Problems with this idea:  Adversarial: we need to consider possible moves of our opponent (s)  Time limit: (think Chess)

Types of Adversarial TBGs (from AI perspective) Perfect information Imperfect information Deterministic Chance Chess, Go, rock- paper-scissors Battleships, Stratego Backgammon, monopoly Civilization, HoMM Bridge, Poker

Game tree (2-player, deterministic, turns) Concepts: State: node in search space Operator: valid move Terminal test: game over Utility function: value for outcome of the game MAX: 1 st player, maximizing its own utility MIN: 2 nd player, minimizing Max’s utility

Minimax Finding perfect play for deterministic games Idea: choose move to position with highest minimax value = best achievable payoff against best play E.g., 2-play game:

Minimax algorithm

Properties of minimax Complete? Optimal? Time complexity? –b: branching factor –m: # moves in a game Yes (if tree is finite) Yes (against an optimal opponent) O(b m ) For chess, b ≈ 35, m ≈100 for "reasonable" games Therefore, exact solution is infeasible

Minimax algorithm with Imperfect Decisions evaluationFunction(state) Cutoff-test(state)

Evaluation Function –Is an estimate of the actual utility –Typically represented as a linear function: EF(state) = w 1 f 1 (state) + w 2 f 2 (state) + … + w n f n (state) –Example: Chess weight: Piece  Number  (w 1 ) Pawn  1  (w 2 ) Knight  3  (w 3 ) Bishop  3  (w 4 ) Rook  5  (w 5 ) Queen  9 Function; state  Number  f 1 = #(pawns,b)  #(pawns,w)  f 2 = #(knight,b)  #(knight,w)  f 3 = #(bishop,b)  #(bishop,w)  f 4 = #(rook,b)  #(rook,w)  f 5 = #(knight,b)  #(knight,w)

Evaluation Function (2) Obviously, the quality of the AI player depends on the evaluation function Conditions for evaluation functions:  If n is a terminal node,  Computing EF should not take long  EF should reflect chances of winning EF(n) = Utility(n) If EF(state) > 3 then is almost-certain that blacks win

Cutting Off Search

α-β pruning example

Properties of α-β Pruning does not affect final result Good move ordering improves effectiveness of pruning With "perfect ordering," time complexity = O(b m/2 )  doubles depth of search A simple example of the value of reasoning about which computations are relevant (a form of metareasoning)

Why is it called α-β? α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max If v is worse than α, max will avoid it  prune that branch Define β similarly for min

The α-β algorithm

Resource limits Suppose we have 100 secs, explore 10 4 nodes/sec  10 6 nodes per move Standard approach: cutoff test: e.g., depth limit (perhaps add quiescence search) evaluation function = estimated desirability of position

Evaluation functions For chess, typically linear weighted sum of features Eval(s) = w 1 f 1 (s) + w 2 f 2 (s) + … + w n f n (s) e.g., w 1 = 9 with f 1 (s) = (number of white queens) – (number of black queens), etc.

Cutting off search MinimaxCutoff is identical to MinimaxValue except 1.Terminal? is replaced by Cutoff? 2.Utility is replaced by Eval Does it work in practice? b m = 10 6, b=35  m=4 4-ply lookahead is a hopeless chess player! –4-ply ≈ human novice –8-ply ≈ typical PC, human master –12-ply ≈ Deep Blue, Kasparov

Deterministic games in practice Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply. Othello: human champions refuse to compete against computers, who are too good. Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

Summary Games are fun to work on! They illustrate several important points about AI perfection is unattainable  must approximate good idea to think about what to think about