University College Cork (Ireland) Department of Civil and Environmental Engineering Course: Engineering Artificial Intelligence Dr. Radu Marinescu Lecture 5

University College Cork (Ireland) Department of Civil and Environmental Engineering Today’s class o Game-playing R. Marinescu October-2009page 2

University College Cork (Ireland) Department of Civil and Environmental Engineering Overview o Computer programs which play 2-player games game-playing as search with the complication of an opponent o General principles of game-playing and search evaluation functions mini-max principle alpha-beta-pruning heuristic techniques o Status of Game-Playing Systems in chess, checkers, backgammon, Othello, etc, computers routinely defeat leading world players o Applications? think of “nature” as an opponent economics, war-gaming, medical drug treatment R. Marinescu October-2009page 3

University College Cork (Ireland) Department of Civil and Environmental Engineering History R. Marinescu October-2009page – Shannon paper 1951 – Turing paper 1958 – Bernstein program – Simon-Newell program (α – β McCarthy?) 1961 – Soviet program – MacHack 6 (MIT AI Lab) 70’s – NW Chess ’s – Crazy Blitz 90’s – Belle, Hitech, Deep Thought, Deep Blue

University College Cork (Ireland) Department of Civil and Environmental Engineering Solving 2-player games o Two players, perfect information o Examples: e.g., chess, checkers, tic-tac-toe o Configuration of the board = unique arrangement of “pieces” o Games as a Search Problem States = board configurations Operators = legal moves Initial State = current configuration Goal = winning configuration payoff function = gives numerical value of outcome of the game R. Marinescu October-2009page 5

University College Cork (Ireland) Department of Civil and Environmental Engineering Game tree search o Game tree: encodes all possible games o We are not looking for a path, only the next move to make (that hopefully leads to a winning position) o Our best move depends on what the other player does R. Marinescu October-2009page 6

University College Cork (Ireland) Department of Civil and Environmental Engineering Move generation R. Marinescu October-2009page 7

University College Cork (Ireland) Department of Civil and Environmental Engineering Example: partial game tree for Tic-Tac-Toe R. Marinescu October-2009page 8 MAX (O) MIN (X) MAX (O) MIN (X)

University College Cork (Ireland) Department of Civil and Environmental Engineering Scoring function R. Marinescu October-2009page 9 We will use the same scoring function for both players, simply negating the values to represent the opponent's scores.

University College Cork (Ireland) Department of Civil and Environmental Engineering Example: scoring function for Tic-Tac-Toe R. Marinescu October-2009page 10 Leaves are either win (+1), loss (-1) or draw (0)

University College Cork (Ireland) Department of Civil and Environmental Engineering Min-Max: an optimal procedure o Designed to find the optimal strategy for MAX player and find best move: 1. Generate the whole game tree to leaves 2. Apply scoring (payoff) function to leaves 3. Back-up values from leaves toward the root:  a MAX node computes the max of its child values  a MIN node computes the min of its child values 4. When value reaches the root: choose max value and the corresponding move. R. Marinescu October-2009page 11

University College Cork (Ireland) Department of Civil and Environmental Engineering Min-Max R. Marinescu October-2009page 12

University College Cork (Ireland) Department of Civil and Environmental Engineering Properties of Min-Max o Complete? Yes (if tree is finite) o Optimal? Yes (against an optimal opponent) o Time complexity? O(b d ) o Space complexity? O(bd) (depth-first exploration) o For chess, b ≈ 35, m ≈100 for "reasonable" games  exact solution completely infeasible Chess:  b ~ 35 (average branching factor)  d ~ 100 (depth of game tree for typical game)  b d ~ ~ nodes!! Tic-Tac-Toe  ~5 legal moves, total of 9 moves  5 9 = 1,953,125  9! = 362,880 (Computer goes first)  8! = 40,320 (Computer goes second) R. Marinescu October-2009page 13

University College Cork (Ireland) Department of Civil and Environmental Engineering Min-Max o Naïve Min-Max assumes that it is possible to search the full tree, where the scoring function of the leaves is either win (+1), loss (-1) or draw (0) o However: For most games, it is impossible to develop the whole search tree o Instead develop part of the tree (up to a certain depth or a number of plys) and evaluate promise of leaves using a static evaluation function. R. Marinescu October-2009page 14

University College Cork (Ireland) Department of Civil and Environmental Engineering Static evaluation function o The static evaluation function: Estimates how good the current board configuration is for a player. Typically, one figures how good it is for the player, and how good it is for the opponent, and subtracts the opponents score from the players Othello: Number of white pieces - Number of black pieces Chess: Value of all white pieces - Value of all black pieces o Typical values from -infinity (loss) to +infinity (win) or [-1, +1]. o If the board evaluation is X for a player, it’s -X for the opponent o Example: Evaluating chess boards, Checkers, Tic-tac-toe R. Marinescu October-2009page 15

University College Cork (Ireland) Department of Civil and Environmental Engineering Evaluation functions: chess R. Marinescu October-2009page 16 For Chess, typically a linear weighted sum of features e.g., weight of the piece (Q = 9, P=1, K=3, B=3.5, R=5)

University College Cork (Ireland) Department of Civil and Environmental Engineering Evaluation functions: Tic-Tac-Toe R. Marinescu October-2009page 17 Another example E(n) =

University College Cork (Ireland) Department of Civil and Environmental Engineering Backup values R. Marinescu October-2009page 18 Two-ply minimax applied to the opening move of tic-tac-toe

University College Cork (Ireland) Department of Civil and Environmental Engineering Backup values R. Marinescu October-2009page 19 Two-ply minimax applied to the X’s second move

University College Cork (Ireland) Department of Civil and Environmental Engineering Backup values R. Marinescu October-2009page 20 Two-ply minimax applied to X’s move near end game

University College Cork (Ireland) Department of Civil and Environmental Engineering Impact of tree depth R. Marinescu October-2009page 21

University College Cork (Ireland) Department of Civil and Environmental Engineering Deep Blue o 32 SP2 processors Each with 8 dedicated chess processors = 256 CP o 50 – 100 billion moves in 3 min ply search R. Marinescu October-2009page 22

University College Cork (Ireland) Department of Civil and Environmental Engineering α – β pruning o In Min-Max there is a separation between node generation and evaluation. R. Marinescu October-2009page 23 Backup Values

University College Cork (Ireland) Department of Civil and Environmental Engineering α – β pruning o Idea: Do depth-first search to generate partial game tree Give static evaluation function to leaves Compute bound on internal nodes o Alpha (α), Beta (β) bounds: α value for MAX node means that max real value is at least alpha β for MIN node means that MIN can guarantee a value below beta o Computation: Alpha of a MAX node is the maximum value of its seen children Beta of a MIN node is the lowest value seen of its child node R. Marinescu October-2009page 24

University College Cork (Ireland) Department of Civil and Environmental Engineering When to prune o Pruning Below a MIN node whose beta value is lower than or equal to the alpha value of its ancestors. Below a MAX node having an alpha value greater than or equal to the beta value of any of its MIN nodes ancestors. R. Marinescu October-2009page 25

University College Cork (Ireland) Department of Civil and Environmental Engineering α – β pruning R. Marinescu October-2009page 26

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 27

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 28

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 29

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 30

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 31

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 32

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 33

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 34

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 35

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 36

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 37

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 38

University College Cork (Ireland) Department of Civil and Environmental Engineering Example R. Marinescu October-2009page 39

University College Cork (Ireland) Department of Civil and Environmental Engineering α – β properties o Guaranteed same value as Min-Max o In a perfectly ordered tree, expected work is O(b d/2 ), vs O(b d ) for Min-Max, so can search twice as deep with the same effort o With good move orderings, the actual running time is close to the optimistic estimate R. Marinescu October-2009page 40

University College Cork (Ireland) Department of Civil and Environmental Engineering Game program o Move generator (ordered moves) 50% o Static evaluation 40% o Search control 10% R. Marinescu October-2009page 41 Openings End games databases [all in place by late 60’s]

University College Cork (Ireland) Department of Civil and Environmental Engineering Move generator o Legal moves o Ordered by Most valuable victim Least valuable aggressor o Killer heuristics R. Marinescu October-2009page 42

University College Cork (Ireland) Department of Civil and Environmental Engineering Static evaluation o Initially - very complex o 70’s - very simple (material) o Now - Deep searchers: moderately complex (hardware) PC programs: elaborate, hand tuned R. Marinescu October-2009page 43

University College Cork (Ireland) Department of Civil and Environmental Engineering Other games o Backgammon Involves randomness – dice rolls Machine-learning based player was able to draw the world champion human player o Bridge Involves hidden information – other players’ cards – and communication during bidding Computer players play well but do not bid well o Go No new elements but huge branching factor No good computer players exist R. Marinescu October-2009page 44

University College Cork (Ireland) Department of Civil and Environmental Engineering Observations o Computers excel in well-defined activities where rules are clear Chess Mathematics o Success comes after a long period of gradual refinement R. Marinescu October-2009page 45

University College Cork (Ireland) Department of Civil and Environmental Engineering Summary o Game playing is best modeled as a search problem o Game trees represent alternate computer/opponent moves o Evaluation functions estimate the quality of a given board configuration for the MAX player. o Min-Max is a procedure which chooses moves by assuming that the opponent will always choose the move which is best for them o Alpha-Beta is a procedure which can prune large parts of the search tree and allow search to go deeper o For many well-known games, computer algorithms based on heuristic search match or out-perform human world experts o Reading: R&N Chapter 6 R. Marinescu October-2009page 46