1. 11 Artificial Intelligence CS 165A Tuesday, October 23, 2007  Adversarial search – games (Ch 6)  Knowledge and reasoning (Ch 7) 1

2. 22 Notes HW#2 posted tomorrow, due Tuesday Multiplayer games and minimax search…

3. 3 Minimax search The minimax decision maximizes the utility under the assumption that the opponent seeks to minimize it (and uses the same evaluation function) Generate the tree of minimax values –Then choose best (maximum) move –Don’t need to keep all values around  Good memory property Depth-first search is used to implement minimax –Expand all the way down to leaf nodes –Recursive implementation (Fig. 6.3) 3

4. 4 Note: Your opponent is not always optimal What’s your best move? 17252894634957 ABCD 1243 4 Minimax says C but what if opponent isn’t perfectly rational?

5. 5 Minimax properties Optimal? Complete? Time complexity? Space complexity? Yes, against an optimal opponent Yes, if the tree is finite Exponential: O( b m ) 5

6. 6 But this could take forever… For chess, b  35 and m  100 for “reasonable” games –So optimality, completeness are kind of irrelevant! Rather, cut the search off early and apply a heuristic evaluation function to the leaves –h(n) estimates the expected utility of the game from a given position (node) n The performance of a game-playing program depends on the quality (and speed!) of its evaluation function 6

7. 7 Heuristics (Evaluation function) Typical evaluation function for game: weighted linear function –h(s) = w 1 f 1 (s) + w 2 f 2 (s) + … + w n f n (s) –weights · features [dot product] For example, in chess –W = { 1, 3, 3, 5, 8 } –F = { # pawns advantage, # bishops advantage, # knights advantage, # rooks advantage, # queens advantage } –Is this what Deep Blue used? –What are some problems with this? More complex evaluation functions may involve learning –Adjusting weights based on outcomes –Perhaps non-linear functions –How to choose the features? 7

8. 8 Cutting off search Suppose we have 100 seconds per move (e.g., in chess) and we can explore 10 4 nodes per second Options: –Stop search after 10 6 nodes –Choose a depth d that will typically take < 100 sec –Do iterative deepening search until allocated time runs out E.g., in chess: –b m = 10 6, b = 35  m = 4 (can look 4 ply ahead) –4-ply: Human novice level (1.5M) –8-ply: Typical PC, good human (2.2T) –12-ply: Deep Blue, Kasparov (?) (3x10 18 ) 8

9. 9 Cutting off search (cont.) This strategy of cutting off search and applying a heuristic evaluation function does not always work well in practice What’s more, a blind application of the evaluation function can be disastrous –E.g., queen to be captured at ply N+1 One solution to the cutoff problem: use quiescent search –Evaluation function is only valid for quiescent states (ones that are likely to be reasonably steady in the near future) –Have to search further to evaluate non-quiescent states (e.g., right before a capture) –Watch out for the horizon effect 9

10. 10 Pruning What’s really needed is “smarter,” more efficient search –Don’t expand “dead-end” nodes! Pruning – eliminating a branch of the search tree from consideration Alpha-beta pruning, applied to a minimax tree, returns the same “best” move, while pruning away unnecessary branches –Many fewer nodes might be expanded –Hence, smaller effective branching factor –…and deeper search –…and better performance  Remember, minimax is depth-first search 10

11. 11 Alpha-beta pruning Consider traversing a minimax game tree in a depth-first manner General principle: Prune nodes (drop them and their descendants from consideration) if they cannot do better than previously generated nodes Alpha-beta search does a minimax search with pruning, using these parameters to describe bounds: –  - best choice so far for MAX –  - best choice so far for MIN Update  and  as search progresses –Prune remaining branches at a node if current value of node is less than current  value for MAX, or more than current  value for MIN 11

12. 12 Alpha-beta pruning (cont.) 12 If m is better than n for Player, neither it not its descendents will ever be a candidate solution (so they can be pruned.)

13. 13 Alpha-beta example 13 A BCD 312821131452 [ ??, ?? ] Worst I could do at this node Best I could do at this node

14. 14 Alpha-beta example 14 A BCD 312821131452 [3, 3] [-inf, 2][2, 2]

15. 15 Non-deterministic games Games that include an element of chance –E.g., Backgammon, Monopoly, Risk, Scrabble  In backgammon, roll  s P(s) of dice determines legal moves –Human (i.e., imperfect!) opponent A deterministic minimax value is not possible –You do not know what a “perfect” opponent will do! –Instead of an exact minimax value, calculate an expected value of a node (“Expectiminimax”)  Weighted average of utility over all possible rolls of dice  Concept of expected utility –  s P(s) u(s), where P(s) is probability of state s u(s) is utility of state s.  Start with actual utilities of terminal nodes

16. 16 Non-deterministic games How to represent –Add “chance” nodes to tree with MAX and MIN nodes –Represent possible paths (e.g., roll of the dice) –Calculate the average utility over all the possible chance events –Numerical scale of utility function is important Aside: Complex deterministic games can be modeled as deterministic games with chance nodes

17. 17 Schematic diagram of backgammon MAX CHANCE MIN CHANCE MAX Possible moves Possible dice rolls Possible opponent moves Possible dice rolls

18. 18 Expectiminimax calculation P(s 1 ) = 0.3 P(s 2 ) = 0.2 P(s 3 ) = 0.5 Expectiminimax (A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1 Expectiminimax (B) = 0.3*2 + 0.2*2 + 0.5*2 = 2.0 Choose move A AB 6 45222 0.30.20.50.30.20.5

19. 19 Expectiminimax calculation P(s 1 ) = 0.3 P(s 2 ) = 0.2 P(s 3 ) = 0.5 Expectiminimax(A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1 Expectiminimax(B) = 0.3*2 + 0.2*9 + 0.5*9 = 6.9 Choose move B AB 6 45299 0.30.20.50.30.20.5

20. 20 Examples Tomorrow’s discussion sessions will include examples of minimax, alpha-beta pruning, and expectiminimax 20

21. 21 Knowledge and Reasoning Logic

22. 22 Reminder: Three parts of an AI Program? AI programs can generally be thought of as comprising three separated parts –Data / knowledge (“knowledge base”) –Operations / rules (“production rules”) –Control What are these three parts in M&C, TTT, vacuum world? We need to consider how to represent knowledge and how to reason about what we know (with respect to what we want)\ –More high-level and flexible representations and reasoning abilities

23. 23 Knowledge and reasoning We want more powerful methods for –Representing Knowledge – more general methods for representing facts about the world and how to act in world –Carrying out Reasoning – more general methods for deducing additional information and a course of action to achieve goals –Focus on knowledge and reasoning rather than states and search  Note that search is still critical This brings us to the idea of logic

24. 24 Logic – the basis of reasoning Logics are formal languages for representing information such that conclusions can be drawn –I.e., to support reasoning Syntax defines the sentences in the language –Allowable symbols –Rules for constructing grammatically correct sentences Semantics defines the meaning of sentences –Correspondence (isomorphism) between sentences and facts in the world –Gives an interpretation to the sentence –Sentences can be true or false with respect to the semantics Proof theory specifies the reasoning steps that are sound (the inference procedure)

25. 25 Knowledge-based agents A knowledge-based agent uses reasoning based on prior and acquired knowledge in order to achieve its goals Two important components: –Knowledge Base (KB)  Represents facts about the world (the agent’s environment) –Fact = “sentence” in a particular knowledge representation language (KRL)  KB = set of sentences in the KRL –Inference Engine – determines what follows from the knowledge base (what the knowledge base entails)  Inference / deduction –Process for deriving new sentences from old ones  Sound reasoning from facts to conclusions

26. 26 Knowledge base (KB) for agents The KB agent must be able to –Represent states, actions, etc. –Incorporate new percepts –Update the internal representation of the world –Deduce hidden properties of the world –Deduce appropriate actions, given the goals Can be viewed at different levels of explanation –Knowledge level  Abstract. What does the agent know? What might you TELL or ASK the agent? How does the agent reason? –Logical level  Sentences in the KRL, inference rules –Implementation level  Data structures, implementation efficiency

27. 27 Basic approach to representing KB agents Basic activities that must be supported: –KB is consulted and updated when making percepts about world –KB is consulted and updated when making choices of actions in world (use of reasoning mechanisms) –KB is consulted and updated in response to perceived changes following actions All represented in logical terms

28. 28 Knowledge Base Inference engine Domain specific content; facts ASK TELL Domain independent algorithms; can deduce new facts from the KB KB Agent