1. AIMA Code: Adversarial Search CSE-391: Artificial Intelligence University of Pennsylvania Matt Huenerfauth February 2005

2. Game Class class Game:... def terminal_test(self, state): "Return True if this is a final state for the game.“ def utility(self, state, player): "Return the value of this final state to player." def to_move(self, state): "Return the player whose move it is in this state.“ def successors(self, state): "Return a list of legal (move, state) pairs."

3. MINIMAX ● Search a game tree all the way to the end. – Start at initial game state. – Exhaustively expand children nodes (all the possible moves in the game from each board). – When you hit an “end state,” decide if it was good/win or bad/loss and give it an appropriate value. ● Then pass up the MIN or the MAX of the values of the children at each level (depending on who’s turn it was). ● At the top level, make the best move: follow the child that passes up the best value.

4. argmax(list,func) – see utils.py def argmax(list, func): "Return element of list with highest func(list[i]) score" best = list[0]; best_score = func(best) for x in list: x_score = func(x) if x_score > best_score: best, best_score = x, x_score return best

5. We’ll see this line of code on the next slide… state – will be a state of the game… call_min – is a function we’ll define later… argmax(game.successors(state), lambda ((a, s)): call_min(s) ) ● Successors is a list of children of this state: (action, state) pairs. ● The lambda function created takes two arguments. – It ignores the first and passes the second to call_min(). – Applied to (action, state) pairs, it calls call_min on each state. ● Argmax returns pair with highest value for call_min(state). Use of argmax(list,func)…

6. At the root of the MINIMAX search at some state in a game. def minimax_decision(state, game):# In a state in a game. player = game.to_move(state)# Who’s turn is it? action, state = \ argmax(game.successors(state), # What successor returns lambda ((a, s)): call_min(s))# maximum call_min()? return action# Move to that state. We first decide whose turn it is. Then our function will tell us what move to make that will move us to the successor state that: - has the MAXIMUM return value - when we call the CALL_MIN function on it. We’re going to see a trend here pretty soon…

7. Take the MAX of your MINs, and the MIN of your MAXs…

8. def call_max(state): if game.terminal_test(state):# Check if game over. return game.utility(state, player) v = -infinity for (a, s) in game.successors(state):# Return the maximum v = max(v, call_min(s))# of the call_min’s. return v def call_min(state): if game.terminal_test(state):# Check if game over. return game.utility(state, player) v = infinity for (a, s) in game.successors(state):# Return the minimum v = min(v, call_max(s))# of the call_max’s. return v

9. def call_max(state): if game.terminal_test(state): return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): v = max(v, call_min(s)) return v def call_min(state): if game.terminal_test(state): return game.utility(state, player) v = infinity for (a, s) in game.successors(state): v = min(v, call_max(s)) return v = take maximum = take minimum Initial call to minimax_decision GAME OVER

10. Depth Limited MINIMAX ● Most games are so complex that searching all of the possible moves in the game until you reach “GAME OVER” on every branch takes too long. – So, we set a limit on how far down to look. – Now, we’ll need a function that can look at a game state that’s not yet finished. It needs to guess how good it thinks that state of the game is. – As we discussed in class, an “Evaluation Function” looks at non-terminal state, and returns a value of how promising that state looks.

11. So, we add a depth limit. Each time we make a call to the next level down, we’ll decrement this depth limit. def minimax_decision(state, game, d):# We’re in some state during a game. player = game.to_move(state)# Who’s turn is it? action, state = \# Return the successor of current argmax(game.successors(state),# board that returns the highest lambda ((a, s)): call_min(s, d-1))# value when we call the return action# call_min function on it. Depth Limited MINIMAX

12. def call_max(state, d): if d == 0: # Use eval. func. if hit limit. return game.evaluation_function(state, player) if game.terminal_test(state):# Check if game over. return game.utility(state, player) v = -infinity for (a, s) in game.successors(state):# Return the maximum v = max(v, call_min(s, d-1))# of the call_mins. return v def call_min(state, d): if d == 0: # Use eval. func. if hit limit. return game.evaluation_function(state, player) if game.terminal_test(state):# Check if game over. return game.utility(state, player) v = infinity for (a, s) in game.successors(state):# Return the minimum v = min(v, call_max(s, d-1))# of the call_maxs. return v

13. = call_max = call_min = take maximum = take minimum Initial call to minimax_decision GAME OVER

14. = call_max = call_min = take maximum = take minimum Initial call to minimax_decision GAME OVER EVALUATION FUNCTION!!!

15. Alpha Beta Search Doing a little pruning…

16. Alpha Beta ● Think about this moment of the search process… 23 24 16 23 5 Imagine we already looked at all the stuff down this part of tree… Do we even need to bother looking at these two last red ones here? Could their blue parent ever get picked up top?

17. Alpha Beta ● The blue node will return at least a 24. So, it’s branch will never get picked by the top-level node. 23 24 16 23 5 Imagine we already looked at all the stuff down this part of tree… Do we even need to bother looking at these two last red ones here? Could their blue parent ever get picked up top?  24

18. Alpha Beta ● So, don’t bother looking at the blue node’s other children. They just don’t matter. 23 24 16 23 5 Imagine we already looked at all the stuff down this part of tree…  24

19. Alpha Beta ● Continue with the rest of the search. 23 24 16 23 5 Imagine we already looked at all the stuff down this part of tree…  24

20. Alpha Beta Search ● A version of MINIMAX that incorporates this handy trick. – It lets us “prune” branches from our tree. – This saves us a lot of searching through useless parts of the tree. ● How do we modify the MINIMAX code to do this? (For now, let’s forget about the depth limit stuff… easy to add later.)

21. First Step: Thread alpha and beta values through all the calls… Alpha: highest value seen in any call_max on the path from root to the current node. Beta: lowest value seen in any call_min on the path from root to current node. Implementing Alpha Beta

22. def call_max(state, alpha, beta): if game.terminal_test(state):return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): v = max(v, call_min(s, alpha, beta)) if v >= beta: return v# stop checking kids if any  beta alpha = max(alpha, v)# alpha= highest value seen in a max return v# no pruning trick if we got to here def call_min(state, alpha, beta): if game.terminal_test(state): return game.utility(state, player) v = infinity for (a, s) in game.successors(state): v = min(v, call_max(s, alpha, beta)) if v <= alpha: return v# stop checking kids if any  alpha beta = min(beta, v)# beta= lowest value seen in a min return v # no pruning trick if we got to here

23. At the root… def alpha_beta(state, game): player = game.to_move(state) action, state = \ argmax(game.successors(state), lambda ((a, s)): call_min(s, -infinity, infinity)) return action Alpha: Starts at –infinity so we know it will get set to new value first time it’s “max”ed inside a call_max function. Beta: Starts at infinity. Same reason for min.

24. Wrap Up ● Game Class – Utility(), Evaluation_Function(), Is_Move() ● MINIMAX – Search the whole game tree to the end each time. ● Depth-Limit MINIMAX – Stop before hitting “GAME OVER” in all branches. ● Alpha-Beta – Cool pruning trick to eliminate useless parts of tree.