2. All rights reservedL. Manevitz Lecture 31 Artificial Intelligence A/O* and Minimax L. Manevitz

3. All rights reservedL. Manevitz Lecture 32 Goals AO* Game Playing Algorithm : –Mini-max. –Mini-max with . –References: Slides follow development in Nilsson; material is in Rich, Nilsson, or Winston.

4. All rights reservedL. Manevitz Lecture 33 A/O* algorithm Data Structure –Graph –Marked Connectors (down; unlike A*) –Costs q() maintained on nodes –SOLVED markings –Note: We’ll discuss on acyclic graphs.

5. All rights reservedL. Manevitz Lecture 34 AND Connector Or Connector

6. All rights reservedL. Manevitz Lecture 35 And/OR Graph

7. All rights reservedL. Manevitz Lecture 36 Solution Subgraph

8. All rights reservedL. Manevitz Lecture 37 Solution Subgraph

9. All rights reservedL. Manevitz Lecture 38 Solution Subgraph G’ of G from n to terminals T If n is in T, G’ is just the singleton n. Otherwise n has one connector to set of nodes n1, n2, …, nk. –There is a Solution graph from each ni to T. –G’ is n, that connector, the nodes n1, …, nk –Plus … the solution graphs from each of the ni.

10. All rights reservedL. Manevitz Lecture 39 Heuristic Values: estimated cost to solution set 0 2 2 4 1 1 0 0 4

11. All rights reservedL. Manevitz Lecture 310 Montone Restriction h(n) <= c + h(n1) + h(n2) + … h(nk) Where c is cost of connector between n and set of n1, …, nk. This guarantees that h(n) <= h*(n).

12. All rights reservedL. Manevitz Lecture 311 Cost (q(n) ) Values If n has no successors q(n) = h (n) Otherwise working from bottom, –q(n) = connector cost + sum of q(successors) Pick smallest of above; and mark direction. If that direction has all successors SOLVED then n is marked SOLVED.

13. All rights reservedL. Manevitz Lecture 312 Basic Idea of A/O* First top-down graph growing picks out best available partial solution sub-graph from explicit graph. One leaf node of this graph is expanded Second, bottom-up cost-revising, connector-marking, SOLVE-labeling.

14. All rights reservedL. Manevitz Lecture 313 AO* Algorithm 1.Create G = ; q(s) = h (s) If s  TERM mark s SOLVED 2.Until s labeled SOLVED do: 1.Compute G’ partial solution subgraph of G by tracing down marked connectors in G from s. 2.Select n in G’, n not in TERM, n a leaf. 3.Expand n, place successors in G, for each successor not already in G let q(successor)=h (successor). Label SOLVED all successors in TERM. (If no successors, reset q(n) := infinity ).

15. All rights reservedL. Manevitz Lecture 314 AO* Algorithm cont. 4.Let S := {n}. 5.Until S =  do : 1.Remove a node, m, from S which has no descendent in G also in S (minimal node). 2.Revise cost for m, (check each connector from m) q(m)=min [c +q(n1)+…+q(nk )]. Mark chosen connector. If all successors their connectors are SOLVED then mark m SOLVED. 3.If m SOLVED or changed q(m) then add to S all “preferred” parents of m. 4.End. 6.End.

16. All rights reservedL. Manevitz Lecture 315 Tracing the Algorithm 3 2 1 1

17. All rights reservedL. Manevitz Lecture 316 Tracing the Algorithm 4 5 4 1 1 4

18. All rights reservedL. Manevitz Lecture 317 Tracing the Algorithm 5 5 4 1 2 0 0 4 2

19. All rights reservedL. Manevitz Lecture 318 Tracing the Algorithm 5 5 4 1 2 0 0 4 2

20. All rights reservedL. Manevitz Lecture 319 Tracing the Algorithm 5 5 4 1 2 0 0

21. All rights reservedL. Manevitz Lecture 320 Tracing the Algorithm 5 5 4 1 2 0 0

22. All rights reservedL. Manevitz Lecture 321 Optimality of A/O*? If h (n) <= c + h (n1) + … h (nk) for all nodes n and connectors THEN OPTIMAL path. If h(term) = 0 this implies h (n) <= h*(n) (optimistic)

23. All rights reservedL. Manevitz Lecture 322 Note Acyclicity means that S is empty eventually. Choice of non-terminal node in partial solution graph to expand? –Best with HIGHEST h* (must expand anyhow; quicker to change mind)

24. All rights reservedL. Manevitz Lecture 323 Game Playing Algorithm What are games ? Notion of Winning Strategy –Von – Neumann Morgernstern Theorem Use of heuristics to pick best move. Look ahead : So use backed up values to choose the best move. Mini-max algorithm.

25. All rights reservedL. Manevitz Lecture 324 Maximilian vs. Minerva Winning Strategy (7,Min) (6,1,Max)(5,2,Max)(4,3,Max) (4,2,1,Min)(3,2,2,Min)(3,3,1,Min)(5,1,1,Min) (4,1,1,1,Max)(3,2,1,1,Max)(2,2,2,1,Max) (3,1,1,1,1,Min)(2,2,1,1,1,Min) (2,1,1,1,1,1,Max)

26. All rights reservedL. Manevitz Lecture 325 Maximilian vs. Minerva (7,Min) (6,1,Max)(5,2,Max)(4,3,Max) (4,2,1,Min)(3,2,2,Min)(3,3,1,Min)(5,1,1,Min) (4,1,1,1,Max)(3,2,1,1,Max)(2,2,2,1,Max) (3,1,1,1,1,Min)(2,2,1,1,1,Min) (2,1,1,1,1,1,Max)

27. All rights reservedL. Manevitz Lecture 326 Mini-max Search See if limit of search has been reached : Yes : Compute static evaluation of current position + return value. Otherwise : If maximizing level – apply Mini-max to each child. Return maximum of results. If minimizing level – apply Mini-max to each child. Return minimum of results.

28. All rights reservedL. Manevitz Lecture 327 Example X X OO X X OOX X X OO X X X OO XX X OO XX X OO X

29. All rights reservedL. Manevitz Lecture 328 Example cont. e(p) = number of directions open for Max – number of directions open for Min e(p) = inf if win for Max e(p) = - inf if win for Min e(p) = 6 – 4 = 2 X O

30. All rights reservedL. Manevitz Lecture 329 Example cont. X X OOX X X OOX X X OOX X X OOX X X OOX O OO O 2-1=13-1=22-1=13-1=2 Min move 1

31. All rights reservedL. Manevitz Lecture 330 Example cont. X X OO X O - inf3-2=12-2=03-2=1 Min move - inf X X OO X X X OO X X X OO X X X OO X OO O

32. All rights reservedL. Manevitz Lecture 331 Example cont. - inf2-2=0 Min move - inf X X OO XX X OO XX X OO XX X OO XX X OO X 3-2=1 O O O O

33. All rights reservedL. Manevitz Lecture 332 Example cont. - inf2-1=13-1=2 Min move - inf 3-1=2 X X OO XX X OO XX X OO XX X OO XX X OO X O O O O

34. All rights reservedL. Manevitz Lecture 333 Example cont. - inf2-1=13-1=2 Min move - inf 2-1=1 X X OO X X X OO X X X OO X X X OO X X X OO X O O OO

35. All rights reservedL. Manevitz Lecture 334 Example cont. X X OO X X OOX X X OO X X X OO XX X OO XX X OO X - inf 1 Max move1

36. All rights reservedL. Manevitz Lecture 335 Mini-max Algorithm Mini-max (Position,Depth) 1.If DeepEnough (Position,Depth) 1.then Return <Static (Position,Depth  2.Successors := MoveGen (Position,Depth)

37. All rights reservedL. Manevitz Lecture 336 Mini-max Algorithm cont. 3.If Depth Odd (Max node) 1.then Loop until Successors =  1.Best = - inf 2.Remove m from Successors 3. := Minimax (m,Depth+1) 4.If Value > Best then : 1.Best := Value 2.BestPath := m^Path 5.End Loop 2.Return

38. All rights reservedL. Manevitz Lecture 337 Mini-max Algorithm cont. 4.If Depth Even (Min node) 1.then Loop until Successors =  1.Best = inf 2.Remove m from Successors 3. := Minimax (m,Depth+1) 4.If Value < Best then : 1.Best := Value 2.BestPath := m^Path 5.End Loop 2.Return

39. All rights reservedL. Manevitz Lecture 338  Mini-max   –values at Max nodes never decrease.   values at Min nodes never increase.

40. All rights reservedL. Manevitz Lecture 339 Cut - Off Below any MIN node having  value smaller than  value of an ancestor. Below any MAX node having  value greater than  value of an ancestor.

41. All rights reservedL. Manevitz Lecture 340 Computing    of Max node : current largest backed up value of child.   of Min node : current smallest backed up value of child.

42. All rights reservedL. Manevitz Lecture 341  Mini-max Keep track of  values and send on to children : 1)Search discontinued below any Min node with  an  of one of it’s ancestors. Set final value of node to be this  value. 2)Search discontinued below any Max node with  a  of one of it’s ancestors. Set final value of node to be this  value.

43. All rights reservedL. Manevitz Lecture 342 Example –  Cut-Off Max Min 27 =2 >=2   <=1 1 This is a  Cut-Off

44. All rights reservedL. Manevitz Lecture 343  Mini-max Algorithm 1.Determine if the level is : 1.The top level. 2.The limit of search has been reached. 3.The level is a minimizing level. 4.The level is a maximizing level

45. All rights reservedL. Manevitz Lecture 344  Mini-max Algorithm cont. 2.If the level is the top level : 1.Let alpha be –inf and let beta be inf. 3.If the limit of search has been reached : compute the static value of the current position relative to the appropriate player + Return result.

46. All rights reservedL. Manevitz Lecture 345  Mini-max Algorithm cont. 4.If the level is a minimizing level : 1.Until all children are examined with Minimax or alpha is bigger then beta : 1.Set beta to the smaller of the given beta values and the smallest value so far reported by Minimax working on the children. 2.Use Minimax on the next child of the current position, handing this new application of Minimax the current alpha and beta. 2.Report beta.

47. All rights reservedL. Manevitz Lecture 346  Mini-max Algorithm cont. 5.If the level is a maximizing level : 1.Until all children are examined with Minimax or alpha is bigger then beta : 1.Set alpha to the larger of the given alpha values and the biggest value so far reported by Minimax working on the children. 2.Use Minimax on the next child of the current position, handing this new application of Minimax the current alpha and beta. 2.Report alpha.

48. All rights reservedL. Manevitz Lecture 347 ≥≥  Max node Min node 05-33  ≤≤ 3  -302-23 ≥≥ ≥≥ ≤≤ 525-501  =2 ≥≥ ≤≤ ≥≥ ≥≥

49. All rights reservedL. Manevitz Lecture 348 Max node Min node 05 -33 3 02 -23 52 -55 01 51-30 3 0 145 Cut Off ≥≥ β=1 ≥≥ β=-3 β≤1 ≥≥ ≥≥ β=3 ≥≥ β≤-3 β≤3 ≥≥ β=4 ≥≥ β≤4 ≥≥ ≥≥

50. All rights reservedL. Manevitz Lecture 349 Example Max node Min node 0 5 -3 3 3 0 2 -23 52 -55 0151 -30 -55 -33 2 3 001 -2 2 -3 3 1 45 Cut Off      

51. All rights reservedL. Manevitz Lecture 350 Example Max node Min node 0 5 -3 3 3 0 2 -23 52 -55 0151 -30 -55 -33 2 3 001 -2 2 -3 3 1 45 Cut Off      

52. All rights reservedL. Manevitz Lecture 351 Other Adjustments Progressive Deepening (Analyze depth1,depth2,…). Heuristic Pruning (Order moves plausibly). Futility Cut-Off. Heuristic Continuation (Waiting for Quiescence). Horizon effect. Secondary Search.

53. All rights reservedL. Manevitz Lecture 352 Homework Number 1 Implement BOTH the Mini-max and The Alpha-Beta Algorithm for a Beloved Game. Suggestion : Othello (not allowed), Checkers, Chinese Checkers, Othello (for losers), Chess etc. Due date (Penalty for lateness).

54. All rights reservedL. Manevitz Lecture 353 Other Adjustments Progressive Deepening (Analyze depth1,depth2,…). Heuristic Pruning (Order moves plausibly). Futility Cut-Off. Heuristic Continuation (Waiting for Quiescence). Horizon effect. Secondary Search.

55. All rights reservedL. Manevitz Lecture 354 Homework Number 1 Implement BOTH the Mini-max AND The Alpha-Beta Algorithm for a Beloved Game. Suggestion : Checkers (Damka), Othello (losers), regular Othello not allowed, Chess etc. Due date Dec 20 (before class) (Penalty for lateness): -4 up to one week. then -4 each class (before class)