電腦對局概論 2012/05/02

O UTLINE 電腦對局基本原理 西洋棋 象棋 圍棋

T YPES OF GAMES 3 No chanceChance Perfect information Chess, Chinese chess, Go 大富翁 Imperfect information 麻將，橋牌，撲克

T HE M INIMAX P ROCEDURE 零合遊戲 對手的失敗就是我的快樂

 -  A LGORITHM 5 function A LPHA -B ETA -S EARCH ( state ) returns an action inputs: state, current state in game v  M AX -V ALUE ( state, – ,   ) return the action in S UCCESSORS ( state ) with value v function M AX -V ALUE ( state, ,  ) returns a utility value inputs: state, current state in game , the value of the best alternative for MAX along the path to state , the value of the best alternative for MIN along the path to state if T ERMINAL -T EST ( state ) then return U TILITY ( state ) v  –  for a, s in S UCCESSORS ( state ) do v  M AX ( v, M IN -V ALUE ( s, ,  )) if v   then return v// fail-high   M AX ( , v ) return v

 -  A LGORITHM ( CONT.) 6 function M IN -V ALUE ( state, ,  ) returns a utility value inputs: state, current state in game , the value of the best alternative for MAX along the path to state , the value of the best alternative for MIN along the path to state if T ERMINAL -T EST ( state ) then return U TILITY ( state ) v    for a, s in S UCCESSORS ( state ) do v  M IN ( v, M AX -V ALUE ( s, ,  )) if v   then return v// fail low   M IN ( , v ) return v

N EGAMAX (B. C HEN, 2010) 7

8

O THER S EARCH T ECHNIQUES Null-Move heuristic generate a good lower bound on the value of a position, using a shallow search in which the opponent gets to move twice at the beginning the lower bound allows  -  pruning without the expense of a full-depth search 9

N ULL -M OVE F ORWARD P RUNING Instead of extending the search in hard positions, we reduce the search in easy positions Allow a chess program to experience a dramatic reduction in branching factor with some manageable risk of missing something important If I do nothing here, can the opponent do anything? Assumption Making a move will result in a higher score than not making a move, except in the endgame 10

N ULL -M OVE F ORWARD P RUNING ( CONT.-1) Before you generate any move, you do a reduced- depth search, with the opponent move first, even though it is usually not the best move If that search result in a score   (fail high), then the true best move is also likely to fail high, and you return  right away without searching any moves Idea You give the opponent a free shot at you, if your position is still so good that you exceed , you assume that you’d also exceed  if you went and searched all of your moves 11

N ULL -M OVE F ORWARD P RUNING ( CONT.-2) Reason of saving time The initial search is made with reduced depth R: depth reduction factor Rather than search all of your moves to depth D, you search all your opponent’s moves with a depth of D – R An excellent value of R is 2 Example If you were going to search all your moves to depth 6, you end up searching all of your opponent’s moves to depth 4 A practical speed up of a ply or two 12

N ULL -M OVE A LGORITHM 13 #define R 2 function A LPHA -B ETA -S EARCH ( depth, ,  ) returns an evaluation value if depth == 0 then return E VALUATE () if last move was not null and position is unlikely to be zugzwang then M AKE N ULL M OVE () v  – A LPHA -B ETA -S EARCH ( depth – 1 – R, – , –  + 1 ) U NMAKE N ULL M OVE () if v   then return  G ENERATE L EGAL M OVES () for each possible move do M AKE N EXT M OVE () v  – A LPHA -B ETA -S EARCH ( depth – 1, – , –  ) U NMAKE M OVE () if v   then return    M AX ( , v ) return  minimal window (  – 1,  )

T RICKS Minimal window To see if the score of the null-move search is beta or better If not, I don’t care how much worse than beta it is Use minimal window (  -1,  ) to get more cutoffs faster Note Null-move changes the result of the search, and may cause you to miss some important line of play 14

S HOULD N OT U SE N ULL -M OVE Null-move doesn’t work if the side to move is in check The degree allowed to happen recursively should be considered, since if you let several of them follow in a row, the search degenerates down to just returning the evaluation Not allow two null-move searches to occur without an intervening real move Allow exactly two null-move searches before forcing a real move 15

S HOULD N OT U SE N ULL -M OVE ( CONT.) 16 Violate the assumption Zugzwang: Positions in which all possible moves make the position worse Only use null-move in positions with many pieces left ?

S OLVING GAMES Game-theoretic value of a game The outcome, i.e., win, loss or draw, when all participants play optimally. 17

C LASSICATION OF GAMES ' SOLUTIONS (L.V. A LLIS IN 1994 ) Ultra-weakly solved the game-theoretic value of the initial position has been determined. Weakly solved for the initial position a strategy has been determined to achieve the game-theoretic value against any opponent. The strategy must be efficient and practical in terms of resource usage. Strongly solved: a strategy has been determined for all legal positions. 18

C OMPLEXITY ISSUE State-space complexity of a game the number of all legal positions in a game. Game-tree (or decision) complexity of a game the number of all leaf nodes in a solution search tree. A solution search tree is a tree where the game- theoretic value of the root position can be decided.. 19

C OMPLEXITY FOR T IC -T AC -T OE Upper bound  legal positions  rotation and reflections State space 3 9 (19683)  5478  765 Game tree size 9!  

G AME TREE (2- PLAYER, DETERMINISTIC, TURNS ) 21

F AIR GAME A fair game the game-theoretic value is draw and both players have roughly an equal probability on making a mistake 22

A DOUBLE DICHOTOMY OF THE GAME SPACE 23

24 GamesLog(state-space)Log(game tree size Nine Men's Morris 1012 Connect-four 1421 Checkers 2131 Othello2858 Chess46123 Chinese chess48150 Hex(11x11)5798 Shogi71226 Renju(15x15)10570 Go-Moku(15x15)10570 Go(19x19)172360

G AMES IN PRACTICE Predictions for 2010 at 2000(Herik et.al.) 25 SolvedOver champion World champion Pro or Grand master amateur Go-Moku & Renju(15x15) Othello(8x8) Checkers(8x 8) Chess Backgammon Go(9x9) Chinese chess Hex Amazons Bridge Shogi Go(19x19)

G AMES IN PRACTICE Status 26 SolvedOver champion World champion Pro or Grand master amateur Go-Moku & Renju(15x15) Othello(8x8) Checkers(8x 8) Chess Backgammon Go(9x9) Chinese chess Hex Amazons Bridge Shogi Go(19x19)

C HESS DeepBlue 30

M INIMAX 36

象棋 1. 子力 2. 位置

子力 將 2000 士 40 象 車 200 馬 90 包 卒 10

ELP 陳志昌、鄭武堯、許舜欽 1 st Computer Olympiad 銅牌 (1990) 2 nd Computer Olympiad 金牌 4 th Computer Olympiad 銀牌 6 th Computer Olympiad 金牌 7 th Computer Olympiad 金牌 8 th Computer Olympiad 銅牌 9 th Computer Olympiad 銀牌 2004 年世界電腦象棋爭霸賽第八名 2007 年世界電腦象棋爭霸賽第三名

象棋世家 2004 世界電腦象棋賽第四名 2007 世界電腦象棋賽第二名 2010 世界電腦象棋賽第二名 總會榮譽六段證書 6th 、 7th 、 10th 、 11th 、 14th Olympiad 銀 牌 15th Olympiad 金牌

人機大戰 2006 年 1 月 14 日象棋世家紅先勝台灣象棋八段 陳振國 2008 年第九屆人機大賽，千慮、象棋世家、 ELP 及棋天大聖參賽，邀請三位七段及一位八 段高手參賽，象棋程式三勝四和一負 2009 年第十屆人機賽，千慮、象棋世家、 ELP 及天馬行空參賽，邀請二位七段和兩位八段高 手參賽，人腦以三勝五和的成績獲勝 2010 年 12/27 新竹交大，象棋世家一勝一和劉 國華八段與馬仲威八段。

圍棋 45

棋子與棋盤 46

棋力分級 47 弱 強 Kyu Dan 912 業餘初學者 職業棋士 段

百萬美金的懸賞 應昌期先生曾提供一百萬美金給能夠寫出擊敗職業棋士的圍 棋程式的人 48

R ANK OF C OMPUTER G O 49

電腦棋類的設計方式 西洋棋，象棋 game tree, α-β cut off 成功原因 合法步不多 (30~40) 有一個簡單而合理的審局函數 此方式不適用於圍棋 合法步太多 (>200) 目前沒有好的審局函數 50

M ONTE C ARLO A PPLICATION 個 63 個 Left = 0.37 * the measure of area Right = 0.63 * the measure of area

T ABLE G AMES Perfect information, no chance 圍棋，象棋，西洋棋 Perfect information, with chance 西洋雙陸棋，大富翁 imperfect information, with chance 麻將，橋牌，十三張，接龍 52

電腦圍棋設計方法 電腦圍棋的傳統設計法 模擬人類思維；經驗法則 JIMMY 程式 (since 1994) 2007 年 UCT 出現，奪得奧林匹亞 19 路圍棋的金牌 UCT: 運用蒙地卡羅法為基礎的樹狀搜尋演算法 53

UCT 樹狀搜尋演算法 UCT ：一個樹狀搜尋演算法 特性：往較有可能為好點的分支，展開較深，最後 會長成一棵不平衡樹 如何決定一個點是否為好點 ? 蒙地卡羅法 統計許多模擬棋局之結果，進行局面之判斷 54

蒙地卡羅樹搜尋演算法架構 55 計畫內容簡述 (2) 蒙地卡羅樹搜尋與局部可觀察機遇型遊戲 MCTS & POSG B CD E F H I A J G J Selectio n Expansi on Simulatio n Back propagation B CD E FG H I A B CD E FG H I A Repeated B CD FG H I A J E

G O 19 X : win against a pro (8p) 19x19, H9 MoGo 2009: win against a pro (9p) 19x19, H7 MoGo 2009: win against a pro (1p) 19x19, H6 MoGo

G O 9 X : win against a pro (5p) 9x9 (blitz) MoGo 2008: win against a pro (5p) 9x9 white MoGo 2009: win against a pro (5p) 9x9 black MoGo 2009: win against a pro (9p) 9x9 white Fuego 2009: win against a top pro (9p) 9x9 black MoGoTW

結論 西洋棋，象棋，圍棋比想像中難 棋海無涯 58