Presentation is loading. Please wait.

Presentation is loading. Please wait.

Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Similar presentations


Presentation on theme: "Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)"— Presentation transcript:

1 Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

2 Games Computers (cannot) Play Checkers: Why was it considered beaten? Two approaches to Checkers Games in ASAP Poker (if time) Contents

3 Games Computers (cannot) Play Arthur Samuel started to look at Checkers 2 The determination of weights through self- play ( adapted, remained fixed) 39 Features Included look-ahead via mini-max Computers & Game Playing : A Potted History 2 Samuel A. Some studies in machine learning using the game of checkers. IBM J. Res. Develop. 3 (1959),

4 Games Computers (cannot) Play Samuelss program defeated Robert Nealy, although the victory is surrounded in controversy Was he state champion? Did he lose the game or did Samuel win? Computers & Game Playing : A Potted History

5 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : Just about to make move 16

6 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) Forced Jump

7 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program)

8 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) Strong (Try to keep) Trapped Only advance to Square 28

9 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program)

10 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program)

11 Games Computers (cannot) Play This was a very poor move. It allowed Samual to retain his Triangle of Oreo AND.. By moving his checker from 19 to 24 it guaranteed Samuel a King This questioned how strong a player Nealy really was Computers & Game Playing : A Potted History

12 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

13 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

14 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K then 5-1, Chinook said would be a draw

15 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

16 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

17 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

18 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

19 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

20 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K This checker is lost

21 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K

22 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K This checker could run (to 10) but.. K

23 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K K

24 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K K

25 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K Forced Jump K

26 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K K

27 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K K

28 Games Computers (cannot) Play Computers & Game Playing : A Potted History White (Nealey) Red (Samuels Program) : After Move K Victory

29 Games Computers (cannot) Play Two Mistakes by Nealy Allowing Samuel to get a King Playing a move that led to defeat when there was a draw available Computers & Game Playing : A Potted History

30 Games Computers (cannot) Play The next year a six match rematch was won by Nealy 5-1. Three years later (1966) the two world championship challengers (Walter Hellman and Derek Oldbury) played four games each against Samuels program. They won every game. Computers & Game Playing : A Potted History

31 Games Computers (cannot) Play Checkers Chinook Blondie 24 (aka Anaconda) Computers & Game Playing : A Potted History

32 Games Computers (cannot) Play Perfect Each Player has complete knowledge of the game state Usually only two players, who take alternate turns Examples include Chess, Checkers, Awari, Connect-Four, Go, Othello Types of Games

33 Games Computers (cannot) Play Imperfect Some of the game state is hidden Examples include Poker, Cribbage, Bridge Types of Games

34 Games Computers (cannot) Play Games with an element of chance The game moves have some stochastic element For example, Backgammon Types of Games

35 Games Computers (cannot) Play Types of Games Solved or Cracked Over Champion World Champion Grand- Master Amateur Connect-FourCheckers (8x8) ChessGo (9x9)Go (19x19) QubicOthelloBackgammon Nine Mens Morris Go_moku Awari 6 Jaap van den Herik H., Uiterwijk and van Rijswijck J. Games Solved: Now and in the future. Artificial Intelligence 134 (2002)

36 Games Computers (cannot) Play Case Study 1: Checkers Samuels work, perhaps, restricted the research into Checkers until 1989 when Jonathan Schaeffer began working on Chinook He had two aims To develop the worlds best checkers player To solve the game of checkers

37 Games Computers (cannot) Play Case Study 1: Checkers Chinook, at its heart, had an evaluation function Piece count (+30% for a King) Runaway checker Dog Hole The weights were hand-tuned

38 Games Computers (cannot) Play Case Study 1: Checkers Opening game database from published work (with corrections they found) Initially 4000 openings, leading to an eventual 40,000 Cooks – innovative lines of play that could surprise an opponent The aim was to take opponents into unknown territory

39 Games Computers (cannot) Play Case Study 1: Checkers Endgame database: Started writing in May 1989 The 8-piece endgame database finished on February 20 th 1994

40 Games Computers (cannot) Play Case Study 1: Checkers , ,224 47,092, ,688,232 62,503,611, ,779,531, ,309,208,481

41 Games Computers (cannot) Play Case Study 1: Checkers 94,048,627,642, ,778,882,769, ,669,578,902, ,695,618,078,654, ,726,900,031,328, ,134,911,067,979, ,511,510,918,189, ,888,183,557,922,816

42 Games Computers (cannot) Play Case Study 1: Checkers 172,905,162,728,973,680, ,568,043,414,939,516, ,661,954,506,100,113, ,352,957,062,510,379, ,459,728,874,435,248, ,435,747,136,817,856, ,406,908,049,181,900, ,072,726,844,888,186,880 TOTAL500,995,484,682,338,672,639

43 Games Computers (cannot) Play Case Study 1: Checkers With a 4-piece database Chinook won the 1989 Computer Olympiad In the 1990 US National Checkers Championship Chinook was using a 6-piece database. It came second, to Marion Tinsley, defeating Don Lafferty on the way who was regarded at the worlds second best player.

44 Games Computers (cannot) Play Case Study 1: Checkers Marion Tinsley Held the world championship from 1951 to 1994 Before playing Chinook, Tinsley only lost 4 competitive games (no matches)

45 Games Computers (cannot) Play Case Study 1: Checkers The winner of the US Championship has the right to play for the world championship. Finishing second (with Tinsley first) entitled Chinook to play for the world championship The American Checkers Federation (ACF) and the European Draughts Association (ADF) refused to let a machine compete for the title.

46 Games Computers (cannot) Play Case Study 1: Checkers In protest, Tinsley resigned The ACF and EDF, created a new world championship, man versus machine and named Tinsley as the world champion. At this time Tinsley was rated at 2812, Chinook was rated at 2706

47 Games Computers (cannot) Play Case Study 1: Checkers The match took place Aug The $300,000 computer used in the tournament ran at about half the speed of a 1GHz PC The match finished 4-2 in favour of Tinsley (with 34 draws)

48 Games Computers (cannot) Play Case Study 1: Checkers A 32 game rematch was held in piece end game Processors four times as fast (resulted in a factor of 2 speed up due to more complex evaluation function and the overhead of parallel processing) Opening book of 40,000 moves In preparation Chinook no losses in 94 games against Grandmasters

49 Games Computers (cannot) Play Case Study 1: Checkers Six games in (1-1, with 4 draws) Tinsley resigned for health reasons. His symptoms were later diagnosed as pancreatic cancer. Tinsley died on 3 rd April 1995 (aged 68). Undoubtedly the best player ever to have lived Chinook was crowned the man versus machine champion. The first automated game player to have achieved this. A 20-match with Don Lafferty resulted in a draw (1-1, with 18 draws)

50 Games Computers (cannot) Play Case Study 1: Checkers Opening Game Database (40,000) moves End Game Database (8-pieces) Hand Crafted Evaluation Function ( / search) Won the World (Man Versus Machine) Championship in 1994… …defeating the world who had held the title for 40 years Marion Tinsley lost his 5 th, 6 th and 7 th games to Chinook Schaeffer J. One Jump Ahead: Challenging Human Supremacy in checkers, Springer, 1997

51 Games Computers (cannot) Play Case Study 2: Anaconda Project started in the summer of 1998, following a conversation between David Fogel and Kumar Chellapilla It was greatly influenced by the recent defeat of Kasparov by Deep Blue Chess was seen as too complex so draughts was chosen instead The aim is to evolve a player – rather than build in knowledge

52 Games Computers (cannot) Play Case Study 2: Anaconda Reject inputting into a neural network what humans think might be important Reject inputting any direct knowledge into the program Reject trying to optimise the weights for an evaluation function

53 Games Computers (cannot) Play Case Study 2: Anaconda The Gedanken Experiment I offer to sit down and play a game with you. We sit across an 8x8 board and I tell you the legal moves We play five games, only then do I say You got 7 points.I dont tell you if you win or lost We play another five games and I say You got 5 points You only know higher is better

54 Games Computers (cannot) Play Case Study 2: Anaconda The Gedanken Experiment How long would it take you to become an expert at this game? We cannot conduct this experiment but we can get a computer to do it

55 Games Computers (cannot) Play Case Study 2: Anaconda Samuels Challenge: Can we design a program that would invent its own features in a game of checkers and learn how to play, even up to the level of an expert?

56 Games Computers (cannot) Play Case Study 2: Anaconda Newells Challenge: Could the program learn just by playing games against itself and receiving feedback, not after each game, but only after a series of games, even to the point where the program wouldnt even know which programs had been won or lost? Newell (and Minsky) 7 believed that this was not possible, arguing that the way forward was to solve the credit assignment problem. 7 Minsky M. Steps Towards Artificial Intelligence. Proceedings of the IRE, 1961, 8-30

57 Games Computers (cannot) Play Case Study 2: Anaconda I1I1 I 32 HL1 1 HL1 40 HL2 1 HL2 10 O # weights=1741 Evaluation used for MiniMax Later changed to an explicit piece differential

58 Games Computers (cannot) Play Case Study 2: Anaconda K

59 Games Computers (cannot) Play Case Study 2: Anaconda K All other neurons have an value of zero

60 Games Computers (cannot) Play Case Study 2: Anaconda Algorithm Initialise 30 Networks Each network played 5 games as red against random opponents Games were played to completion or until 100 moves had been made (a draw) +2 for a win, 0 for a draw, -1 for a loss 15 best performing networks were saved for the next generation and copies were mutated

61 Games Computers (cannot) Play Case Study 2: Anaconda Observations The points for a win, lose draw were set such that wins were encouraged. No experimentation with different values were tried Players could play a different number of games. This was, purposefully, not taken into account Mutation was carried out using an evolutionary strategy

62 Games Computers (cannot) Play Case Study 2: Anaconda After 10 Generations After 10 generations the best neural network was able to beat both its creators and a simple (undergraduate project) program which, by the authors admission was weak Note: 400MHz PC

63 Games Computers (cannot) Play Case Study 2: Anaconda ACF Ratings Grand (Senior) Master2400+Class E Master Class F Expert Class G Class A Class H Class B Class I Class C Class J<200 Class D

64 Games Computers (cannot) Play Case Study 2: Anaconda After 100 Generations Playing on zone.com Initial rating = 1600 Beat a player ranked at 1800 but lost against a player in the mid 1900s After 10 games their ranking had improved to After 100 games it had improved to 1750 Typically a 6-ply search but often 8-ply

65 Games Computers (cannot) Play Case Study 2: Anaconda Observations The highest rating it achieved was 1825 The evolved King value was 1.4, which agrees with perceived wisdom that a king is worth about 1.5 of a checker In 100 generations a neural network had been created that was competitive with humans It surpassed Samuels program The challenge set by Newell had been met

66 Games Computers (cannot) Play Case Study 2: Anaconda The Next Development Alpha-Beta Pruning introduced and evolved over 250 generations Over a series of games, Obi_WanThe Jedi defeated a player rated at 2134 (48 out of 40,000 registered) and also beat a player rated 2207 (ranked 18) Final rating was 1902 (taking into account the different orderings of the games)

67 Games Computers (cannot) Play Case Study 2: Anaconda The Next Development Spatial nature of the board was introduced as at the moment it just saw the board as a vector of length 32

68 Games Computers (cannot) Play Case Study 2: Anaconda x3 Overlapping squares

69 Games Computers (cannot) Play Case Study 2: Anaconda x4 Overlapping squares

70 Games Computers (cannot) Play Case Study 2: Anaconda x5 Overlapping squares

71 Games Computers (cannot) Play Case Study 2: Anaconda x6 Overlapping squares

72 Games Computers (cannot) Play Case Study 2: Anaconda x7 Overlapping squares

73 Games Computers (cannot) Play Case Study 2: Anaconda x8 Overlapping squares

74 Games Computers (cannot) Play Case Study 2: Anaconda The Next Development = 91 inputs 5,046 weights

75 Games Computers (cannot) Play Case Study 2: Anaconda I1I1 I 32 O... # weights= x3 25 4x4 16 5x5 9 6x6 4 7x7 1 8x8 HL 1 (91 nodes) HL 2 (40 nodes) HL 3 (10 nodes) Sum of 32 Board Inputs

76 Games Computers (cannot) Play Case Study 2: Anaconda 2 months and 230 generations later!! After 100 games the rating was 1929 A 27 point increase over the previous network. Nice but not decisive Maybe it was due to there being three times more weights but the training period was the same?

77 Games Computers (cannot) Play Case Study 2: Anaconda 6 months and 840 generations later!! After 165 games it was rated at (sd 33.94) Rated in the top 500 at zone.com (of the 120,000 players now registered) That is better than 99.61% of the players

78 Games Computers (cannot) Play Case Study 2: Anaconda Playing Chinook 8 In a ten match series against Chinnok novice level it had two wins, two losses and 4 draws 8 Fogel D. B. and Chellapilla K. Verifying Anacondas expert rating by competing against Chinook: experiments in co-evolving a neural checkers player, Neurocomputing 42 (2002) 69-86

79 Games Computers (cannot) Play Case Study 2: Anaconda Blondie The neural checkers player went through a number of names David0111 Anaconda Blondie24

80 Games Computers (cannot) Play Case Study 2: Anaconda

81 Games Computers (cannot) Play Case Study 2: Anaconda References Fogel D.B. Blondie24: Playing at the Edge of AI, Morgan Kaufmann, 2002 Fogel D. B. and Chellapilla K. Verifying Anacondas expert rating by competing against Chinook: experiments in co-evolving a neural checkers player, Neurocomputing 42 (2002) Chellapilla K and Fogel D. B. Evolving neural networks to play checkers without expert knowledge. IEEE Trans. Neural Networks 10(6): , 1999 Chellapilla K and Fogel D. B.Evolution, neural networks, games, and intelligence, Proc. IEEE 87(9): Chellapilla K and Fogel D. B. Evolving an expert checkers playing program without relying on human expertise. IEEE Trans. Evolutionary Computation, 2001 Chellapilla K and Fogel D. B. Anaconda Defeats Hoyle 6-0: A Case Study Competing an Evolved Checkers Program Against Commercially Available Software. Proc. Of CEC 2000:


Download ppt "Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)"

Similar presentations


Ads by Google