Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

Similar presentations


Presentation on theme: "Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers."— Presentation transcript:

1 Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers Institute for Knowledge and Agent Technology, IKAT Department of Computer Science Universiteit Maastricht The Netherlands SOFSEM 2004 Prague, Hotel VZ Merin Sunday, January h

2 2Sofsem'04 Contents Games in Artificial-Intelligence ResearchGames in Artificial-Intelligence Research Game-tree search principlesGame-tree search principles Using opponent modelsUsing opponent models Opponent-Model searchOpponent-Model search Gains and risksGains and risks Implementing OM search: complexitiesImplementing OM search: complexities Past future of computer chessPast future of computer chess

3 3Sofsem'04 Games, such as Chess, Checkers, and Go are an attractive pastime and scientifically interesting

4 4Sofsem'04 Chess Much research has been performed in Computer ChessMuch research has been performed in Computer Chess Deep Blue (IBM) defeated the world champion Kasparov in 1997Deep Blue (IBM) defeated the world champion Kasparov in 1997 A Micro Computer better than the world champion?A Micro Computer better than the world champion?

5 5Sofsem'04 World Champion Programs KAISSA1974StockholmKAISSA1974Stockholm CHESS1977TorontoCHESS1977Toronto BELLE1980LinzBELLE1980Linz CRAY BLITZ1983New YorkCRAY BLITZ1983New York CRAY BLITZ 1986KeulenCRAY BLITZ 1986Keulen DEEP THOUGHT1989EdmontonDEEP THOUGHT1989Edmonton REBEL1992MadridREBEL1992Madrid FRITZ1995Hong KongFRITZ1995Hong Kong SHREDDER1999PaderbornSHREDDER1999Paderborn JUNIOR2002MaastrichtJUNIOR2002Maastricht SHREDDER2003GrazSHREDDER2003Graz ?2004Ramat-Gan?2004Ramat-Gan

6 6Sofsem'04 International Draughts Buggy best draughts programBuggy best draughts program Human better than computer, but the margin is smallHuman better than computer, but the margin is small Challenge: More knowledge in programChallenge: More knowledge in program

7 7Sofsem'04 Go Computer Go programs are weakComputer Go programs are weak Problem: recognition of patternsProblem: recognition of patterns Top Go programs: Go4++, Many Faces of Go, GnuGo, and HandtalkTop Go programs: Go4++, Many Faces of Go, GnuGo, and Handtalk

8 8Sofsem'04 Awari A Mancala game (pebble-hole game)A Mancala game (pebble-hole game) The game is a drawThe game is a draw Solved by John Romein and Henri Bal (VU Amsterdam) 2002Solved by John Romein and Henri Bal (VU Amsterdam) 2002 All positions (900 billion) with a cluster computer of 144 1GHz processors and 72Gb ram computed in 51 hourAll positions (900 billion) with a cluster computer of 144 1GHz processors and 72Gb ram computed in 51 hour Proven that even the best computer programs still make many errors in their gamesProven that even the best computer programs still make many errors in their games

9 9Sofsem'04 Scrabble Maven beats every human opponentMaven beats every human opponent Author Brian SheppardAuthor Brian Sheppard Ph.D. (UM): Towards Perfect Play of Scrabble (02)Ph.D. (UM): Towards Perfect Play of Scrabble (02) Maven can play scrabble in any languageMaven can play scrabble in any language

10 10Sofsem'04 Overview

11 11Sofsem'04 Computer Olympiad Initiative of David Levy (1989)Initiative of David Levy (1989) Since 1989 there have been 8 olympiads; 4x Londen, 3x Maastricht, 1x GrazSince 1989 there have been 8 olympiads; 4x Londen, 3x Maastricht, 1x Graz Goal:Goal: Finding the best computer program for each gameFinding the best computer program for each game Connecting programmers / researchers of different gamesConnecting programmers / researchers of different games Computers play each other in competitionComputers play each other in competition Demonstrations:Demonstrations: Man versus MachineMan versus Machine Man + Machine versus Man + MachineMan + Machine versus Man + Machine

12 12Sofsem'04 Computer versus Computer

13 13Sofsem'04 Computer Olympiad Last time in Graz 2004Last time in Graz 2004 Also World Championship Computer Chess and World Championship BackgammonAlso World Championship Computer Chess and World Championship Backgammon 80 particpants in several categories80 particpants in several categories Competitions in olympiads history:Competitions in olympiads history: Abalone, Awari, Amazons, Backgammon, Bao, Bridge, Checkers, Chess, Chinese Chess, Dots and Boxes, Draughts, Gipf, Go-Moku, 19x19 Go, 9x9 Go, Hex, Lines of Action, Poker, Renju, Roshambo, Scrabble, and Shogi

14 14Sofsem'04 UM Programs on the Computer Olympiads Bao 1.0 Gipfted - Gipf Anky - Amazons MIA - LOA Magog - Go

15 15Sofsem'04 Computer Game-playing Can computers beat humans in board games like Chess, Checkers, Go, Bao?Can computers beat humans in board games like Chess, Checkers, Go, Bao? This is one of the first tasks of Artificial Intelligence (Shannon 1950)This is one of the first tasks of Artificial Intelligence (Shannon 1950) Successes obtained in Chess (Deep Blue), Checkers, Othello, Draughts, Backgammon, Scrabble...Successes obtained in Chess (Deep Blue), Checkers, Othello, Draughts, Backgammon, Scrabble...

16 16Sofsem'04 Computer Game-Playing Sizes of game trees:Sizes of game trees: Nim-5: 28 nodesNim-5: 28 nodes Tic-Tac-Toe: 10 5 nodesTic-Tac-Toe: 10 5 nodes Checkers: nodesCheckers: nodes Chess: nodesChess: nodes Go: nodesGo: nodes In practice it is intractable to find a solution with minimax: so use heuristic searchIn practice it is intractable to find a solution with minimax: so use heuristic search

17 17Sofsem'04 Three Techniques Minimax SearchMinimax Search α-β Searchα-β Search Opponent ModellingOpponent Modelling

18 18Sofsem'04 Game-playing Domain: two-player zero-sum game with perfect information (Zermelo, 1913)Domain: two-player zero-sum game with perfect information (Zermelo, 1913) Task: find best response to opponents moves, provided that the opponent does the sameTask: find best response to opponents moves, provided that the opponent does the same Solution: Minimax procedure (von Neumann, 1928)Solution: Minimax procedure (von Neumann, 1928)

19 19Sofsem'04 Minimax Search Players remove on turn 1, 2, or 3 matches. The player who takes the last match wins the game. Nim-5

20 20Sofsem'04 Minimax Search +1 –

21 21Sofsem'04 Minimax Search –1 +1–1+1 – –1 +1 –1+1 –1 +1 –1+1 MINIMAXING

22 22Sofsem'04 Pruning You do not need the total solution to play (well): only the move at the root is neededYou do not need the total solution to play (well): only the move at the root is needed Methods exist to find this move without a need for a total solution: pruningMethods exist to find this move without a need for a total solution: pruning Alpha-beta search (Knuth & Moore 75)Alpha-beta search (Knuth & Moore 75) Proof-number search (Allis et al. 94), etc.Proof-number search (Allis et al. 94), etc. Playing is solving a sequence of game treesPlaying is solving a sequence of game trees

23 23Sofsem'04 Heuristic Search Truncate the game tree (depth + selection) Use a (static heuristic) evaluation function at the leaves to replace pay-offs Miminax on the reduced game tree 0.25– – –

24 24Sofsem'04 Heuristic Search The approach works very well in Chess, but...The approach works very well in Chess, but... Is solving a sequence of reduced games the best way to win a game?Is solving a sequence of reduced games the best way to win a game? Heuristic values are used instead of pay-offsHeuristic values are used instead of pay-offs Additional information in the tree is unusedAdditional information in the tree is unused The opponent is not taken into accountThe opponent is not taken into account We aim at the last item: opponentsWe aim at the last item: opponents

25 25Sofsem'04 Minimax [3]

26 26Sofsem'04 α-β algorithm β-pruning

27 27Sofsem'04 The strength of α-β More than thousand prunings

28 28Sofsem'04 The importance of α-β algorithm β-pruning 3 3

29 29Sofsem'04 THE NUMBER OF DIFFERENT, REACHABLE POSITIONS IN CHESS IS (CHINCHALKAR): The Possibilities Of Chess

30 30Sofsem'04 A Clever Algorithm (α-β) SAVES THE SQAURE ROOT OF THE NUMBER OF POSSIBILITIES, N, THIS IS MORE THAN 99, %

31 31Sofsem'04 A Calculation NUMBER OF POSSIBILITIES:10 46 SAVINGS BY α-Β ALGORITHM: PARALLEL PROCESSORS:10 3 POSITIONS PER SECOND:10 9 LEADS TO: = SECONDS A CENTURY IS10 9 SECONDS SOLVING CHESS:10 2 CENTURIES SO 100 CENTURIES OR 10,000 YEAR

32 32Sofsem'04 Using opponents strategy (NIM-5) Player 1 never takes –1 +1 – –1 +1–1 +1 –1

33 33Sofsem'04 Using opponents strategy Well known Tic-Tac-Toe strategy:Well known Tic-Tac-Toe strategy: R1: make 3-in-a-row if possible R2: prevent opponents 3-in-a-row if possible if possible H1: occupy central square if possible H2: occupy corner square if possible knows thatuses this strategy

34 34Sofsem'04 Opponent-Model search Iida, vd Herik, Uiterwijk, Herschberg (1993) Carmel and Markovitch (1993) Opponents evaluation function is known (unilateral: the opponent uses minimax)Opponents evaluation function is known (unilateral: the opponent uses minimax) This is the opponent modelThis is the opponent model It is used to predict opponents movesIt is used to predict opponents moves Best response is determined, using the own evaluation functionBest response is determined, using the own evaluation function

35 35Sofsem'04 OM Search Procedure:Procedure: At opponents nodes: use minimax (alpha-beta) to predict the opponents move. Return the own value of the chosen moveAt opponents nodes: use minimax (alpha-beta) to predict the opponents move. Return the own value of the chosen move At own nodes: Return (the move with) the highest valueAt own nodes: Return (the move with) the highest value At leaves: Return the own evaluationAt leaves: Return the own evaluation Implementation: one-pass or probingImplementation: one-pass or probing

36 36Sofsem'04 OM Search Equations

37 37Sofsem'04 OM Search Example V 0 : 6 V op : 7 V 0 : 8 V op : 6 V 0 : 7 V op : 6 V 0 : 8 V op : 9 v 0 : 7 v op : 6 v 0 : 8 v op : 6 v 0 : 8 v op : 6

38 38Sofsem'04 OM Search Algorithm (probing)

39 39Sofsem'04 Risks and Gains in OM search Gain: difference between the predicted move and the minimax moveGain: difference between the predicted move and the minimax move Risk: difference between the move we expect and the move the opponent really selectsRisk: difference between the move we expect and the move the opponent really selects Prediction of the opponent is important, and depends on the quality of opponent model.Prediction of the opponent is important, and depends on the quality of opponent model.

40 40Sofsem'04 Additional risk Even if the prediction is correct, there are traps for OM searchEven if the prediction is correct, there are traps for OM search Let P be the set of positions that the opponent selects, v 0 be our evaluation function and v op the opponents function.Let P be the set of positions that the opponent selects, v 0 be our evaluation function and v op the opponents function.

41 41Sofsem'04 Four types of error V 0 overestimates a position in P (bad)V 0 overestimates a position in P (bad) V 0 underestimates a position in PV 0 underestimates a position in P V op underestimates a position that enters P (good for us)V op underestimates a position that enters P (good for us) V op overestimates a position in PV op overestimates a position in P

42 42Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

43 43Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

44 44Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

45 45Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

46 46Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

47 47Sofsem'04 V o : our evaluation V op : opponents evaluation Mmx: minimax value Obt: obtained value

48 48Sofsem'04 Pruning in OM Search Pruning at MAX nodes is not possible in OM search, only pruning at MIN nodes can take place (Iida et al, 1993).Pruning at MAX nodes is not possible in OM search, only pruning at MIN nodes can take place (Iida et al, 1993). Analogous to α-β search, the pruning version is called β-pruning OM searchAnalogous to α-β search, the pruning version is called β-pruning OM search

49 49Sofsem'04 Pruning Example a gfed bc on h qp i sr j ut k wv lm yx

50 50Sofsem'04 Two Implementations One-pass:One-pass: visit every node at most oncevisit every node at most once back-up both your own and the opponents valueback-up both your own and the opponents value Probing:Probing: At MIN nodes use α-β search to predict the opponents moveAt MIN nodes use α-β search to predict the opponents move back-up only one valueback-up only one value

51 51Sofsem'04 One-Pass β-pruning OM search

52 52Sofsem'04 Probing β-pruning OM search

53 53Sofsem'04 Best-case time complexity

54 54Sofsem'04 Best-case time complexity C OM = Subtree A C OM = Subtree B

55 55Sofsem'04 Best-case time complexity The time complexity for the one-pass version is equal to that of the theoretical bestThe time complexity for the one-pass version is equal to that of the theoretical best The time complexity for the probing version is different:The time complexity for the probing version is different: first detect the number of leaf evaluations needed:first detect the number of leaf evaluations needed: Then find the number of probes needed:Then find the number of probes needed:

56 56Sofsem'04 Best-case time complexity In the best case, pruning in the probes is not changed by β=v+1 : all probes take the same number of evaluations as with a fully open window.

57 57Sofsem'04 Best-case time complexity

58 58Sofsem'04 Time complexities compared (Branching factors 4, 8, 12, 16, 20)

59 59Sofsem'04 Average-case time complexity (Measured on random game trees)

60 60Sofsem'04 Probabilistic Opponent-model Search Donkers, vd Herik, Uiterwijk (2000) More elaborate opponent model:More elaborate opponent model: Opponent uses a mixed strategy: n different evaluation functions (opponent types) plus a probability distributionOpponent uses a mixed strategy: n different evaluation functions (opponent types) plus a probability distribution It is assumed to approximate the true opponents strategyIt is assumed to approximate the true opponents strategy Own evaluation function is one of the opponent typesOwn evaluation function is one of the opponent types

61 61Sofsem'04 PrOM Search Procedure:Procedure: At opponents nodes: use minimax (alpha- beta) to predict the opponents move for all opponent types separately. Return the expected own value of the chosen movesAt opponents nodes: use minimax (alpha- beta) to predict the opponents move for all opponent types separately. Return the expected own value of the chosen moves At own nodes: Return (the move with) the highest valueAt own nodes: Return (the move with) the highest value At leafs: Return the own evaluationAt leafs: Return the own evaluation Implementation: one-pass or probingImplementation: one-pass or probing

62 62Sofsem'04 PrOM Search Equations

63 63Sofsem'04 PrOM Search Example : 6 1 : 10 0 : 8 1 : 8 0 : 7 1 : 6 0 : 8 1 : 7 v 0 : 8v 0 : 7v 0 : 8v 0 : 6 0 : 6 1 : 8 0 : 7 1 : 6 Pr( 0 ) = 0.3 Pr( 1 ) = : 7 1 : v 0 : 0.3 x x 8 = 7.4 v 0 : v 0 : 7.4

64 64Sofsem'04 PrOM Search Algorithm

65 65Sofsem'04 How did chess players envision the future of non-human chess players? - Euwe - Donner - De Groot - Timman - Sosonko - Böhm Question: Do you think that a computer will be able to play at grandmaster level?

66 66Sofsem'04 Euwe (June 10, 1980) I dont believe so. I am almost convinced of that, too

67 67Sofsem'04 Timman (June 22, 1980): If you would express it in ELO-points, than I believe that a computer will not be able to obtain more than lets say ELO-points.

68 68Sofsem'04 Sosonko (August 26, 1980): I havent thought about that so much, but I believe that it will certainly not be harmful for the human chess community. It will become more interesting. I am convinced that computer chess will play a increasingly important role. It will be very interesting, I thinkI havent thought about that so much, but I believe that it will certainly not be harmful for the human chess community. It will become more interesting. I am convinced that computer chess will play a increasingly important role. It will be very interesting, I think

69 69Sofsem'04 Donner (April 13, 1981): But the computer cannot play chess at all and will never be able to play the game, at least not the first two thousand years, (...) What it does now has nothing to do with chess.

70 70Sofsem'04 De Groot (June 19, 1981): No, certainly not. As a matter of fact, I believe it will not even possible that it can reach a stable master level. I believe it is possible that the computer can reach a master level, but not that it is a stable master. I believe it is possible that the computer can reach a master level, but not that it is a stable master.

71 71Sofsem'04 Contributions from Science COMPUTERS PLAY STRONGER THAN HUMANS.COMPUTERS PLAY STRONGER THAN HUMANS. COMPUTERS CAN NOT SOLVE CHESS.COMPUTERS CAN NOT SOLVE CHESS. COMPUTERS ENABLE AN ALTERNATIVE FORM OF GAME EXPERIENCE.COMPUTERS ENABLE AN ALTERNATIVE FORM OF GAME EXPERIENCE.

72 72Sofsem'04 Conclusions 1. Chess is a frontrunner among the games 2. Kasparovs defeat has become a victory for brute force in combination with knowledge and opponent modelling

73 73Sofsem'04 The Inevitable Final Conclusion TECHNOLOGY, AND IN PARTICULAR OPPONENT MODELLING, MAKES THE BEST BETTER, BUT DOES NOT LEAVE THE WEAKER PLAYERS WITH EMPTY HANDS SINCE THEIR LEVEL WILL INCREASE TOO.


Download ppt "Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers."

Similar presentations


Ads by Google