Artificial Intelligence AIMA §5: Adversarial Search Rodney Nielsen Many / most of these slides were adapted from: AI MA, Russell and Norvig

Adversarial Search Games Perfect Play Minimax algorithm α-β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information

Games vs. Search Problems Unpredictable Opponent  Solution is a strategy specifying a move for every possible opponent reply Time limits  unlikely to find goal, must approximate

Types of Games Deterministic Chance Perfect Information Chess, Checkers, Go, Othello Backgammon, Monopoly Imperfect Information Battleship, Blind TicTacToe Bridge, Poker, Scrabble, Nuclear War

Game tree (2-player, deterministic, turns)

Minimax Perfect play for deterministic, perfect-information games Idea: choose move to position with highest minimax value = best achievable payoff against opponent’s best play E.g., 2-ply game:

Minimax Algorithm

Properties of Minimax Complete? Optimal? Search type? Yes, if tree is finite Optimal? Yes, against an optimal opponent. Otherwise? Search type? Depth-first exploration Complexity? Time: O(bm) Space: O(bm) For chess, b≈35 for “reasonable” games  Exact solution completely infeasible But do we need to explore every path?

α-β Pruning

α-β Pruning X ≥ -∞ ≥ -∞ ≤ 3 ≤ 3 ≥ 3 ≥ 3 ≤ 3 ≤ 3 ≤ 2 ≥ 3 ≥ 3 ≤ 3 ≤ 2 ≤ 14 ≤ 3 ≤ 2 ≤ 2

Game Objective Function Utility Functions Defn: Zero-sum game

α-β Pruning Efficiency / Space and Time Complexity Move ordering Killer moves Transposition table Heuristic evaluation function

Move Ordering X ≥ -∞ ≥ -∞ ≤ 3 ≤ 3 ≥ 3 ≥ 3 ≤ 3 ≤ 3 ≤ 2 ≥ 3 ≥ 3 ≤ 3 ≤ 2 ≤ 14 ≤ 3 ≤ 2 ≤ 2

Move Ordering Move ordering Transposition table Killer moves Need to avoid repeated evaluation (as in state space searches)

Heuristic Evaluation Functions Estimates the expected utility value Instead of calculating exact utility Based on probabilities Chess example (two pawns versus one): 72% wins, 20% losses, and 8% draws f = 0.72*1 + 0.2*0 + 0.08*½ = 0.76 Order terminal states same as utility function Use exact utility at terminal states

Heuristic Evaluation Functions Use features of non-terminal state to estimate utility Creates equivalence classes Often a weighted linear function of features Eval(s) = w1f1(s) + w1f1(s) + … + wdfd(s) Examples Chess features ? Sweep Fives features

Cutting Off Search if Cutoff-Test(state, depth) return Eval(state) When should you cut off the search Depth Quiescence

Cutting Off Search When should you cut off the search Depth Quiescence Horizon Effect

Forward Pruning Beam Search Only explore best n moves past a certain depth

Search versus Lookup Pre-compute and store the best move and its value for some states

Stochastic Games What to do when future states are based on a stochastic process? Calculate the expected value Based on an exhaustive evaluation, or Based on Monte Carlo simulation

Partially Observable Games Belief States Monte Carlo Simulations - again

State-of-the-Art Chess Checkers Othello Poker Go Bridge?