2. Do you drive? Have you thought about how the route plan is created for you in the GPS system? How would you implement a cross-and-nought computer program? Konstanz, May 2014 1AI Search Algorithms – Tree Searches

3. Artificial Intelligence Search Algorithms Dr Rong Qu School of Computer Science University of Nottingham Nottingham, NG8 1BB, UK rxq@cs.nott.ac.uk Course Introduction & Tree Search

4. AI Search Algorithms – Tree Searches Problem Space Many problems exhibit no detectable regular structure to be exploited, they appear “chaotic”, and do not yield to efficient algorithms The concept of search plays an important role in science and engineering In one way, any problem whatsoever can be seen as a search for “the right answer” Konstanz, May 2014 3

5. AI Search Algorithms – Tree Searches Problem Space 4 Konstanz, May 2014

6. AI Search Algorithms – Tree Searches Problem Space Search space Set of all possible solutions to a problem Search algorithms Take a problem as input Return a solution to the problem Konstanz, May 2014 5

7. AI Search Algorithms – Tree Searches Problem Space Search algorithms Exact (exhaustive) tree search Uninformed search algorithms Depth First, Breadth First Informed search algorithms A*, Minimax Meta-heuristic search Modern AI search algorithms Genetic algorithm, Simulated Annealing Konstanz, May 2014 6

8. AI Search Algorithms – Tree Searches7 References Artificial intelligence: a guide to intelligent systems. Addison-Wesley, 2002. Negnevitsky Good AI textbook Easy to read while in depth Available from library

9. Konstanz, May 2014 AI Search Algorithms – Tree Searches8 References Metaheuristics – From Design to Implementation, Talbi, 2009 Quite recent Seems quite complete, from theory to design, and implementation

10. Konstanz, May 2014 AI Search Algorithms – Tree Searches9 References Artificial Intelligence – A Modern Approach (AIMA) (Russell/Norvig), 1995 & 2003 “Artificial Intelligence (AI) is a big field and this is a big book” (Preface to AIMA) Most comprehensive textbook in AI Web site: http://aima.cs.berkeley.edu/ Textbook in many courses Better to be used as reference book (especially for tree search) You don’t have to learn and read the whole book

11. Konstanz, May 2014 AI Search Algorithms – Tree Searches10 Other Teaching Resources Introduction to AI http://www.cs.nott.ac.uk/~rxq/g51iai.htm Uniform cost search Depth limited search Iterative deepening search alpha-beta pruning: game playing Other basics of AI incl. tree search AI Methods http://www.cs.nott.ac.uk/~rxq/g52aim.htm Search algorithms

12. BRIEF INTRO TO SEARCH SPACE Konstanz, May 2014 AI Search Algorithms – Tree Searches11

13. AI Search Algorithms – Tree Searches Problem Space Often we can't simply write down and solve the equations for a problem Exhaustive search of large state spaces appears to be the only viable approach How? Konstanz, May 2014 12

14. Konstanz, May 2014 AI Search Algorithms – Tree Searches13 The Travelling Salesman Problem TSP: A salesperson has to visit a number of cities (S)He can start at any city and must finish at that same city The salesperson must visit each city only once The cost of a solution is the total distance traveled Given a set of cities and distances between them Find the optimal route, i.e. the shortest possible route The number of possible routes is : A B C D E

15. Konstanz, May 2014 AI Search Algorithms – Tree Searches14 Combinatorial Explosion A 10 city TSP has 181,000 possible solutions A 20 city TSP has 10,000,000,000,000,000 possible solutions A 50 City TSP has 100,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000 possible solutions There are 1,000,000,000,000,000,000,000 litres of water on the planet* *Mchalewicz, Z, Evolutionary Algorithms for Constrained Optimization Problems, CEC 2000 (Tutorial)

16. Konstanz, May 2014 AI Search Algorithms – Tree Searches15 Combinatorial Explosion A 50 City TSP has 1.52 * 10 64 possible solutions A 10GHz computer might do 10 9 tours per second Running since start of universe, it would still only have done 10 26 tours Not even close to evaluating all tours! One of the major unsolved theoretical problems in Computer Science

17. Konstanz, May 2014 AI Search Algorithms – Tree Searches16 Combinatorial Explosion

18. TREE SEARCHES Konstanz, May 2014 AI Search Algorithms – Tree Searches17

19. Konstanz, May 2014 AI Search Algorithms – Tree Searches18 Search Trees Does the tree under the following root contain a node “I”? All you get to see at first is the root and a guarantee that it is a tree The rest is up to you to discover during the process of search F

20. Konstanz, May 2014 AI Search Algorithms – Tree Searches19 Search Trees A tree is a graph that – is connected but becomes disconnected by removing any edge (branch) – has precisely one path between any two nodes Unique path makes them much easier to search so we will start with search on trees JBC D E F G A H I

21. Konstanz, May 2014 20 Search Trees Depth of a node Depth of a tree Examples: TSP vs. game Tree size Branching factor b = 2 (binary tree) Depth d dnodes at d, 2 d total nodes 011 123 247 3815 41631 53263 664127 JBC D E F G A H I Exponentially - Combinatorial explosion AI Search Algorithms – Tree Searches

22. Konstanz, May 2012 AI Search Algorithms – Tree Searches21 Search Trees Heart of search techniques Nodes: states of problem Root node: initial state of problem Branches: moves by operator Branching factor: number of neighbourhoods

23. Konstanz, May 2014 AI Search Algorithms – Tree Searches22 Search Trees – Example I

24. Konstanz, May 2014 AI Search Algorithms – Tree Searches23 Search Trees – Example II 1 st level: 1 root node (empty board) 2 nd level: 8 nodes 3 rd level: 6 nodes for each of the node on the 2 nd level (?) …

25. AI Search Algorithms – Tree Searches Implementing a Search State The state in the state space to which this node corresponds Parent-Node the node that generated the current node Operator the operator that was applied to generate this node Path-Cost the path cost from the initial state to this node Konstanz, May 2014 24

26. BLIND SEARCHES Breadth First Search Konstanz, May 2014 AI Search Algorithms – Tree Searches25

27. Konstanz, May 2014 AI Search Algorithms – Tree Searches26 Breadth First Search - Method Expand Root Node First Expand all nodes at level 1 before expanding level 2 OR Expand all nodes at level d before expanding nodes at level d+1 Queuing function Adds nodes to the end of the queue

28. Konstanz, May 2012 AI Search Algorithms – Tree Searches27 Breadth First Search - Implementation A B CC D E D E F G A BC DE D E F G node = REMOVE-FRONT(nodes) If GOAL-TEST[problem] applied to STATE(node) succeeds then return node nodes = QUEUING-FN(nodes, EXPAND(node, OPERATORS[problem]))

29. Konstanz, May 2014 28 A CDEFB GHIJKLMNOP QRSTUVWXYZ The example node set Initial state Goal state A L Press space to see a BFS of the example node set AI Search Algorithms – Tree Searches

30. Konstanz, May 2014 29 A CDEFB GHIJKL QRSTU A BCD We begin with our initial state: the node labeled A. Press space to continue This node is then expanded to reveal further (unexpanded) nodes. Press space Node A is removed from the queue. Each revealed node is added to the END of the queue. Press space to continue the search. The search then moves to the first node in the queue. Press space to continue. Node B is expanded then removed from the queue. The revealed nodes are added to the END of the queue. Press space. Size of Queue: 0 Nodes expanded: 0Current Action:Current level: n/a Queue: EmptyQueue: ASize of Queue: 1 Nodes expanded: 1 Queue: B, C, D, E, F Press space to begin the search Size of Queue: 5 Current level: 0Current Action: Expanding Queue: C, D, E, F, G, HSize of Queue: 6 Nodes expanded: 2Current level: 1 We then backtrack to expand node C, and the process continues. Press space Current Action: BacktrackingCurrent level: 0Current level: 1 Queue: D, E, F, G, H, I, JSize of Queue: 7 Nodes expanded: 3Current Action: ExpandingCurrent Action: BacktrackingCurrent level: 0Current level: 1 Queue: E, F, G, H, I, J, K, LSize of Queue: 8 Nodes expanded: 4Current Action: ExpandingCurrent Action: BacktrackingCurrent level: 0Current level: 1Current Action: Expanding NM Queue: F, G, H, I, J, K, L, M, NSize of Queue: 9 Nodes expanded: 5 E Current Action: BacktrackingCurrent level: 0Current Action: ExpandingCurrent level: 1 OP Queue: G, H, I, J, K, L, M, N, O, PSize of Queue: 10 Nodes expanded: 6 F Current Action: BacktrackingCurrent level: 0Current level: 1Current level: 2Current Action: Expanding Queue: H, I, J, K, L, M, N, O, P, Q Nodes expanded: 7 G Current Action: BacktrackingCurrent level: 1Current Action: Expanding Queue: I, J, K, L, M, N, O, P, Q, R Nodes expanded: 8 H Current Action: BacktrackingCurrent level: 2Current level: 1Current level: 0Current level: 1Current level: 2Current Action: Expanding Queue: J, K, L, M, N, O, P, Q, R, S Nodes expanded: 9 I Current Action: BacktrackingCurrent level: 1Current level: 2Current Action: Expanding Queue: K, L, M, N, O, P, Q, R, S, T Nodes expanded: 10 J Current Action: BacktrackingCurrent level: 1Current level: 0Current level: 1Current level: 2Current Action: Expanding Queue: L, M, N, O, P, Q, R, S, T, U Nodes expanded: 11 K Current Action: BacktrackingCurrent level: 1 LLLL Node L is located and the search returns a solution. Press space to end. FINISHED SEARCH Queue: EmptySize of Queue: 0 Current level: 2 BREADTH-FIRST SEARCH PATTERN L Press space to continue the search AI Search Algorithms – Tree Searches

31. Konstanz, May 2014 AI Search Algorithms – Tree Searches30 Evaluating Search Algorithms Evaluating against four criteria Optimal Complete Time complexity Space complexity

32. Konstanz, May 2014 AI Search Algorithms – Tree Searches31 Evaluating Breadth First Search Evaluating against four criteria Complete? Yes Optimal? Yes

33. Konstanz, May 2014 AI Search Algorithms – Tree Searches32 Evaluating Breadth First Search Evaluating against four criteria Space Complexity O(b d ), i.e. number of leaves Time Complexity 1 + b + b 2 + b 3 +... + b d-1 i.e. O(b d ) b: the branching factor d: is the depth of the search tree Note : The space/time complexity could be less as the solution could be found anywhere before the d th level.

34. 33 Time and memory requirements for breadth-first search, assuming a branching factor of 10, 100 bytes per node and searching 1000 nodes/second Evaluating Breadth First Search Konstanz, May 2014 AI Search Algorithms – Tree Searches

35. Konstanz, May 2014 AI Search Algorithms – Tree Searches34 Breadth First Search - Observations Very systematic If there is a solution, BFS is guaranteed to find it If there are several solutions, then BFS will always find the shallowest goal state first and if the cost of a solution is a non-decreasing function of the depth then it will always find the cheapest solution

36. Konstanz, May 2014 AI Search Algorithms – Tree Searches35 Breadth First Search - Observations Space is more of a factor to breadth first search than time Time is still an issue Who has 35 years to wait for an answer to a level 12 problem (or even 128 days to a level 10 problem) It could be argued that as technology gets faster then exponential growth will not be a problem But even if technology is 100 times faster we would still have to wait 35 years for a level 14 problem and what if we hit a level 15 problem!

37. BLIND SEARCHES Depth First Search Konstanz, May 2014 AI Search Algorithms – Tree Searches36

38. Konstanz, May 2014 AI Search Algorithms – Tree Searches37 Depth First Search - Method Expand Root Node First Explore one branch of the tree before exploring another branch Queuing function Adds nodes to the front of the queue

39. Konstanz, May 2014 AI Search Algorithms – Tree Searches38 Depth First Search - Observations Space complexity Only needs to store the path from the root to the leaf node as well as the unexpanded nodes For a state space with a branching factor of b and a maximum depth of m, DFS requires storage of bm nodes Time complexity is b m in the worst case

40. Konstanz, May 2014 AI Search Algorithms – Tree Searches39 Depth First Search - Observations If DFS goes down a infinite branch it will not terminate if it does not find a goal state If it does find a solution there may be a better solution at a lower level in the tree Therefore, depth first search is neither complete nor optimal

41. Konstanz, May 2014 AI Search Algorithms – Tree Searches40 A CDEFB GHIJKLMNOP QRSTUVWXYZ The example node set Initial state Goal state A L Exercise: - Show the queue status of BFS and DFS

42. HEURISTIC SEARCHES Konstanz, May 2014 AI Search Algorithms – Tree Searches41

43. Konstanz, May 2014 AI Search Algorithms – Tree Searches42 Blind Search vs. Heuristic Searches Blind search Randomly choose where to search in the search tree When problem get large, not practical any more Heuristic search Explore the node which is more likely lead to goal state

44. Konstanz, May 2014 AI Search Algorithms – Tree Searches43 Heuristic Searches - Characteristics Heuristic searches work by deciding which is the next best node to expand Has some domain knowledge Use a function to tell us how close the node is to the goal state Usually more efficient than blind searches Sometimes called an informed search There is no guarantee that it is the best node Heuristic searches estimate the cost to the goal from its current position. It is usual to denote the heuristic evaluation function by h(n)

45. Konstanz, May 2012 AI Search Algorithms – Tree Searches44 Heuristic Searches – Example H sld (n) = straight line distance between n and the goal location Go to the city which is nearest to the goal city

46. Konstanz, May 2014 AI Search Algorithms – Tree Searches45 Heuristic Searches - Greedy Search So named as it takes the biggest “bite” it can out of the problem That is, it seeks to minimise the estimated cost to the goal by expanding the node estimated to be closest to the goal state Implementation is achieved by sorting the nodes based on the evaluation function: f(h) = h(n)

47. Konstanz, May 2014 AI Search Algorithms – Tree Searches46 Heuristic Searches - Greedy Search It is only concerned with short term aims It is possible to get stuck in an infinite loop It is not optimal It is not complete Time and space complexity is O(b m ); where m is the depth of the search tree

48. Konstanz, May 2012 AI Search Algorithms – Tree Searches47 Performed well, but not optimal Greedy Search

49. Konstanz, May 2012 AI Search Algorithms – Tree Searches48 Heuristic Searches vs. Blind Searches goal outline of graph increasing cost start This region is basically wasted effort goal outline of graph increasing cost start

50. Konstanz, May 2014 AI Search Algorithms – Tree Searches49 Heuristic Searches vs. Blind Searches goal outline of graph increasing cost start Want to achieve this but stay complete optimal If bias the search “too much” then could miss goals or miss shorter paths

51. HEURISTIC SEARCHES A* Search Konstanz, May 2014 AI Search Algorithms – Tree Searches50

52. Konstanz, May 2014 51 The A* algorithm Combines the cost so far and the estimated cost to the goal That is evaluation function f(n) = g(n) + h(n) An estimated cost of the cheapest solution via n goal outline of graph start A B gAgA gBgB hAhA hBhB AI Search Algorithms – Tree Searches

53. Konstanz, May 2014 AI Search Algorithms – Tree Searches52 The A* algorithm A search algorithm to find the shortest path through a search space to a goal state using a heuristic f = g + h f - function that gives an evaluation of the state g - the cost of getting from the initial state to the current state h - the cost of getting from the current state to a goal state

54. Konstanz, May 2014 AI Search Algorithms – Tree Searches53 The A* algorithm – admissible heuristic It can be proved to be optimal and complete providing that the heuristic is admissible. That is the heuristic must never over estimate the cost to reach the goal h(n) must provide a valid lower bound on cost to the goal But, the number of nodes that have to be searched still grows exponentially

55. Konstanz, May 2012 AI Search Algorithms – Tree Searches54 Straight Line Distances to Bucharest TownSLD Arad366 Bucharest0 Craiova160 Dobreta242 Eforie161 Fagaras178 Giurgiu77 Hirsova151 Iasi226 Lugoj244 TownSLD Mehadai241 Neamt234 Oradea380 Pitesti98 Rimnicu193 Sibiu253 Timisoara329 Urziceni80 Vaslui199 Zerind374 We can use straight line distances as an admissible heuristic as they will never overestimate the cost to the goal. This is because there is no shorter distance between two cities than the straight line distance.

56. 55 ANIMATION OF A*. Arad Oradea Zerind Fagaras Neamt Iasi Vaslui Hirsova Eforie Urziceni Giurgui Pitesti Sibiu Dobreta Craiova Rimnicu Mehadia Timisoara Lugoj 87 92 142 86 98 86 211 101 90 99 71 75 140 118 111 70 75 120 138 146 97 80 140 Bucharest 99+178=277 80+193=273 140+366=506 177+98=275 226+160=386(R) 310+0=310 (F) Optimal route is (80+97+101) = 278 miles 1.Sibiu 278+0=278 (R,P) 2.Rimnicu3.Pitesti4.Fagaras5.Bucharest 278 GOAL!! Fringe in RED Visited in BLUE Nodes Expanded 0+253=253 Annotations: “g+h=f” could use 211? 315+160=475(P) Konstanz, May 2014

57. AI Search Algorithms – Tree Searches56 The A* algorithm Clearly the expansion of the nodes is much more directed towards the goal The number of expansions is significantly reduced Exercise: Draw the search tree of A* for the 8-puzzle using the two heuristics

58. 8 puzzle problem Online demo of A* algorithm for 8 puzzle Noyes Chapman’s 15 puzzle Initial StateGoal State The A* Algorithm – An example Konstanz, May 2014 57AI Search Algorithms – Tree Searches

59. H1H1 = the number of tiles that are in the wrong position H2H2 = the sum of the distances of the tiles from their goal positions using the Manhattan Distance We need admissible heuristics (never over estimate) Both are admissible but which one is better? Possible Heuristics in A* Algorithm The A* Algorithm – An example Konstanz, May 2014 58AI Search Algorithms – Tree Searches

60. 3 134 82 765 6 134 862 75 134 862 75 5 134 82 765 4 134 862 75 6 134 862 75 4 134 862 75 5 134 862 75 6 The A* Algorithm – An example 59 Konstanz, May 2014 H 1 = the number of tiles that are in the wrong position (=4) H 2 = the sum of the distances of the tiles from their goal positions using the Manhattan Distance (=5)

61. 6 134 862 75 134 862 75 5 134 82 765 4 134 862 75 6 5 134 82 765 14 832 765 5 134 82 765 3 13 824 765 2 134 825 76 4 13 824 765 1 What’s wrong with the search? Is it implementing the A* search? 60 Konstanz, May 2014 AI Search Algorithms – Tree Searches H 2 = the sum of the distances of the tiles from their goal positions using the Manhattan Distance (=5)

62. Konstanz, May 2014 61AI Search Algorithms – Tree Searches

63. Konstanz, May 2014 AI Search Algorithms – Tree Searches62 A* is optimal and complete, but it is not all good news It can be shown that the number of nodes that are searched is still exponential to the size of most problems This has implications not only for the time taken to perform the search but also the space required Of these two problems the search complexity is more serious The A* Algorithm – An example

64. GAME TREE SEARCHES MiniMax Algorithm Konstanz, May 2014 AI Search Algorithms – Tree Searches63

65. AI Search Algorithms – Tree Searches Game Playing Up till now we have assumed the situation is not going to change whilst we search Shortest route between two towns The same goal board of 8-puzzle, n-Queen Game playing is not like this Not sure of the state after your opponent move Goals of your opponent is to prevent your goal, and vice versa Konstanz, May 2014 64

66. AI Search Algorithms – Tree Searches Game Playing Konstanz, May 2012 65 Wolfgang von Kempelen “The Turk” 18th Century Chess Automaton 1770-1854

67. AI Search Algorithms – Tree Searches Game Playing Konstanz, May 2012 66

68. AI Search Algorithms – Tree Searches Game Playing - Minimax Game Playing An opponent tries to thwart your every move 1944 - John von Neumann outlined a search method (Minimax) maximise your position whilst minimising your opponent’s Konstanz, May 2014 67

69. AI Search Algorithms – Tree Searches Game Playing - Minimax In order to implement we need a method of measuring how good a position is Often called a utility function Initially this will be a value that describes our position exactly Konstanz, May 2014 68

70. DEFG = terminal position= agent = opponent 4-5 1-72-3-8 1 MAX MIN 412-3 MAX 1-3 B C A Assume we can generate the full search tree Of course for larger problem it’s not possible to draw the entire tree Game starts with computer making the first move Then the opponent makes the next move Now we can decide who win the game Assume positive: computer wins We know absolutely who will win following a branch The idea is computer wants to force the opponent to lose, and maximise its own chance of winning

71. DEFG = terminal position= agent = opponent 4-5 1-72-3-8 1 MAX MIN 412-3 MAX 1-3 B C A Now the computer is able to play a perfect game. At each move it’ll move to a state of the highest value. Question: who will win this game, if both players play a perfect game? 70

72. AI Search Algorithms – Tree Searches Game Playing - Minimax Nim Start with a pile of tokens At each move the player must divide the tokens into two non-empty, non-equal piles + + + Konstanz, May 2014 71

73. AI Search Algorithms – Tree Searches Game Playing - Minimax Nim Starting with 7 tokens, draw the complete search tree At each move the player must divide the tokens into two non-empty, non-equal piles Konstanz, May 2014 72

74. 7 6-15-24-35-1-14-2-13-2-23-3-1 4-1-1-13-2-1-12-2-2-1 3-1-1-1-12-2-1-1-1 2-1-1-1-1-1 Konstanz, May 2014 73AI Search Algorithms – Tree Searches

75. Game Playing - Minimax Conventionally, in discussion of minimax, have two players “MAX” and “MIN” The utility function is taken to be the utility for MAX Larger values are better for “MAX” Assuming MIN plays first, complete the MIN/MAX tree Assume that a utility function of 0 = a win for MIN 1 = a win for MAX Konstanz, May 2014 74

76. AI Search Algorithms – Tree Searches Game Playing - Minimax Player MAX is going to take the best move available Will select the next state to be the one with the highest utility Hence, value of a MAX node is the MAXIMUM of the values of the next possible states i.e. the maximum of its children in the search tree Konstanz, May 2014 75

77. AI Search Algorithms – Tree Searches Game Playing - Minimax Player MIN is going to take the best move available for MIN i.e. the worst available for MAX Will select the next state to be the one with the lowest utility higher utility values are better for MAX and so worse for MIN Hence, value of a MIN node is the MINIMUM of the values of the next possible states i.e. the minimum of its children in the search tree Konstanz, May 2014 76

78. AI Search Algorithms – Tree Searches Game Playing - Minimax A “MAX” move takes the best move for MAX so takes the MAX utility of the children A “MIN” move takes the best for min hence the worst for MAX so takes the MIN utility of the children Games alternate in play between MIN and MAX Konstanz, May 2014 77

79. 7 6-15-24-3 5-1-14-2-13-2-23-3-1 4-1-1-13-2-1-12-2-2-1 3-1-1-1-12-2-1-1-1 2-1-1-1-1-1 MIN MAX 0 1 0 0 0 1 0101 111 1 Konstanz, May 2014 78AI Search Algorithms – Tree Searches

80. Game Playing - Minimax Efficiency of the search Game trees are very big Evaluation of positions is time-consuming How can we reduce the number of nodes to be evaluated? “alpha-beta search” Konstanz, May 2014 79

81. AI Search Algorithms – Tree Searches Game Playing - Minimax Konstanz, May 2014 80 Consider a variation of the two player game Nim: The game starts with a stack of 5 tokens. At each move a player removes one, two or three tokens from the pile, leaving the pile non-empty. A player who has to remove the last token loses the game. (a) Draw the complete search tree for this variation of Nim. (b) Assume two players, min and max. Max plays first. If a terminal state in the search tree developed above is a win for min, a utility function of -1 is assigned to that state. A utility function of 1 is assigned to a state if max wins the game. Apply the minimax algorithm to the search tree.

82. APPENDIX Konstanz, May 2014 AI Search Algorithms – Tree Searches81 A brief history of AI game playing

83. AI Search Algorithms – Tree Searches Game Playing Game Playing has been studied for a long time Babbage (1791-1871) Analytical machine tic-tac-toe Turing (1912-1954) Chess playing program Within 10 years a computer will be a chess champion Herbert Simon, 1957 Konstanz, May 2014 82

84. AI Search Algorithms – Tree Searches Game Playing Why study game playing in AI Games are intelligent activities It is very easy to measure success or failure Do not require large amounts of knowledge They were thought to be solvable by straightforward search from the starting state to a winning position Konstanz, May 2014 83

85. AI Search Algorithms – Tree Searches Game Playing - Checkers Arthur Samuel 1952 – first checker program, written for an IBM 701 1954 - Re-wrote for an IBM 704 10,000 words of main memory Konstanz, May 2014 84

86. AI Search Algorithms – Tree Searches Game Playing - Checkers Arthur Samuel Added a learning mechanism that learnt its own evaluation function by playing against itself After a few days it could beat its creator And compete on equal terms with strong human players Konstanz, May 2014 85

87. AI Search Algorithms – Tree Searches Game Playing - Checkers Jonathon Schaeffer – Chinook, 1996 In 1992 Chinook won the US Open Plays a perfect end game by means of a database And challenged for the world championship http://www.cs.ualberta.ca/~chinook/ Konstanz, May 2014 86

88. AI Search Algorithms – Tree Searches Game Playing - Checkers Jonathon Schaeffer – Chinook, 1996 Dr Marion Tinsley World championship for over 40 years, only losing three games in all that time Against Chinook he suffered his fourth and fifth defeat But ultimately won 21.5 to 18.5 Konstanz, May 2014 87

89. AI Search Algorithms – Tree Searches Game Playing - Checkers Jonathon Schaeffer – Chinook, 1996 Dr Marion Tinsley In August 1994 there was a re-match but Marion Tinsley withdrew for health reasons Chinook became the official world champion Konstanz, May 2014 88

90. AI Search Algorithms – Tree Searches Game Playing - Checkers Jonathon Schaeffer – Chinook, 1996 Uses Alpha-Beta search Did not include any learning mechanism Schaeffer claimed Chinook was rated at 2814 The best human players are rated at 2632 and 2625 Konstanz, May 2014 89

91. AI Search Algorithms – Tree Searches Game Playing - Checkers Chellapilla and Fogel – 2000 “Learnt” how to play a good game of checkers The program used a population of games with the best competing for survival Learning was done using a neural network with the synapses being changed by an evolutionary strategy Input: current board position Output: a value used in minimax search Konstanz, May 2014 90

92. AI Search Algorithms – Tree Searches Game Playing - Checkers Chellapilla and Fogel – 2000 During the training period the program is given no information other than whether it won or lost (it is not even told by how much) No strategy and no database of opening and ending positions The best program beats a commercial application 6-0 The program was presented at CEC 2000 (San Diego) and prize remain unclaimed Konstanz, May 2014 91

93. AI Search Algorithms – Tree Searches Game Playing - Chess No computer can play even an amateur-level game of chess Hubert Dreyfus, 1960’s Konstanz, May 2014 92

94. AI Search Algorithms – Tree Searches Game Playing - Chess Shannon - March 9 th 1949 - New York Size of search space (10 120 - average of 40 moves) 10 120 > number of atoms in the universe 200 million positions/second = 10 100 years to evaluate all possible games Age of universe = 10 10 Searching to depth = 40, at one state per microsecond, it would take 10 90 years to make its first move Konstanz, May 2014 93

95. AI Search Algorithms – Tree Searches Game Playing - Chess 1957 – AI pioneers Newell and Simon predicted that a computer would be chess champion within ten years Simon: “I was a little far-sighted with chess, but there was no way to do it with machines that were as slow as the ones way back then” 1958 - First computer to play chess was an IBM 704 about one millionth capacity of deep blue Konstanz, May 2014 94

96. AI Search Algorithms – Tree Searches Game Playing - Chess 1967 : Mac Hack competed successfully in human tournaments 1983 : “Belle” attained expert status from the United States Chess Federation Mid 80’s : Scientists at Carnegie Mellon University started work on what was to become Deep Blue Sun workstation, 50K positions per second Project moved to IBM in 1989 Konstanz, May 2014 95

97. AI Search Algorithms – Tree Searches Game Playing - Chess May 11th 1997, Gary Kasparov lost a six match game to deep blue, IBM Research 3.5 to 2.5 Two wins for deep blue, one win for Kasparov and three draws (http://www.research.ibm.com/deepblue/meet/html/d.3.html) Konstanz, May 2014 96

98. AI Search Algorithms – Tree Searches Game Playing - Chess Still receives a lot of research interests Computer program to “learn” how to play chess, rather than being “told” how it should play Research on game playing at School of CS, Nottingham Konstanz, May 2014 97

99. AI Search Algorithms – Tree Searches Game Playing – Go* A significant challenge to computer programmers, not yet much helped by fast computation Search methods successful for chess and checkers do not work for Go, due to many qualities of the game Larger area of the board (five times the chess board) New piece appears every move - progressively more complex *wikipedia: http://en.wikipedia.org/wiki/Go_(game) Konstanz, May 2014 98

100. AI Search Algorithms – Tree Searches Game Playing – Go* A significant challenge to computer programmers, not yet much helped by fast computation Search methods successful for chess and checkers do not work for Go, due to many qualities of the game A material advantage in Go may just mean that short-term gain has been given priority Very high degree of pattern recognition involved in human capacity to play well … *wikipedia: http://en.wikipedia.org/wiki/Go_(game) Konstanz, May 2014 99

101. APPENDIX Konstanz, May 2014 AI Search Algorithms – Tree Searches100 Alfa Beta pruning

102. A BC DE 658 MAX MIN 6>=8 MAX <=6 HIJK = agent= opponent On discovering util( D ) = 6 we know that util( B ) <= 6 On discovering util( J ) = 8 we know that util( E ) >= 8 STOP! What else can you deduce now!? STOP! What else can you deduce now!? Can stop expansion of E as best play will not go via E Value of K is irrelevant – prune it! STOP! What else can you deduce now!? Konstanz, May 2014 101AI Search Algorithms – Tree Searches

103. A BC DEFG 658 MAX MIN 6>=8 MAX 6 HIJKLM = agent= opponent 21 2 <=2 >=6 Konstanz, May 2014 102AI Search Algorithms – Tree Searches

104. A BC DEFG 658 MAX MIN 6>=8 MAX 6 HIJKLM = agent= opponent 21 2 2 >=6 Konstanz, May 2014 103AI Search Algorithms – Tree Searches

105. A BC DEFG 658 MAX MIN 6>=8 MAX 6 HIJKLM = agent= opponent 21 2 2 6 alpha cutoff beta cutoff Alpha-beta Pruning Konstanz, May 2014 104AI Search Algorithms – Tree Searches

106. APPENDIX Konstanz, May 2014 AI Search Algorithms – Tree Searches105 General Search

107. AI Search Algorithms – Tree Searches General Search Function GENERAL-SEARCH (problem, QUEUING-FN) returns a solution or failure nodes = MAKE-QUEUE(MAKE-NODE(INITIAL- STATE[problem])) Loop do If nodes is empty then return failure node = REMOVE-FRONT(nodes) If GOAL-TEST[problem] applied to STATE(node) succeeds then return node nodes = QUEUING-FN (nodes,EXPAND(node,OPERATORS[problem])) End End Function Konstanz, May 2014 106

108. AI Search Algorithms – Tree Searches General Search Function GENERAL-SEARCH (problem, QUEUING-FN) returns a solution or failure nodes = MAKE-QUEUE(MAKE-NODE(INITIAL- STATE[problem])) Loop do If nodes is empty then return failure node = REMOVE-FRONT(nodes) If GOAL-TEST[problem] applied to STATE(node) succeeds then return node nodes = QUEUING-FN (nodes,EXPAND(node,OPERATORS[problem])) End End Function Konstanz, May 2014 107

109. AI Search Algorithms – Tree Searches General Search Function GENERAL-SEARCH (problem, QUEUING-FN) returns a solution or failure nodes = MAKE-QUEUE(MAKE-NODE(INITIAL- STATE[problem])) Loop do If nodes is empty then return failure node = REMOVE-FRONT(nodes) If GOAL-TEST[problem] applied to STATE(node) succeeds then return node nodes = QUEUING-FN (nodes,EXPAND(node,OPERATORS[problem])) End End Function Konstanz, May 2014 108