Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 1 Chapter 23 Planning in the Game of Bridge Dana S. Nau Dept. of Computer Science, and Institute for Systems Research University of Maryland Lecture slides for Automated Planning: Theory and Practice

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 2 Connect Four:solved Go-Moku:solved Qubic:solved Nine Men’s Morris:solved Othello:better than humans Checkers:better than any living human Backgammon:better than all but about 10 humans Chess:competitive with the best humans Bridge:as of 2004, about as good as mid-level humans Computer Programs for Games of Strategy

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 3 Computer Programs for Games of Strategy l Fundamental technique: the minimax algorithm minimax(u) = max{minimax(v) : v is a child of u} if it’s Max’s move at u = min{minimax(v) : v is a child of u} if it’s Min’s move at u l Largely “brute force” l Can prune off portions of the tree u cutoff depth & static evaluation function u alpha-beta pruning u transposition tables u … l But even then, it still examines thousands of game positions l For bridge, this has some problems …

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 4 West North East South Q Q  J 6  5  9  7  A K 5 3 A  9   How Bridge Works l Four players; 52 playing cards dealt equally among them l Bidding to determine the trump suit u Declarer: whoever makes highest bid u Dummy: declarer’s partner l The basic unit of play is the trick u One player leads; the others must follow suit if possible u Trick won by highest card of the suit led, unless someone plays a trump u Keep playing tricks until all cards have been played l Scoring based on how many tricks were bid and how many were taken

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 5 l Bridge is an imperfect information game u Don’t know what cards the others have (except the dummy) u Many possible card distributions, so many possible moves l If we encode the additional moves as additional branches in the game tree, this increases the branching factor b l Number of nodes is exponential in b u worst case: about 6x10 44 leaf nodes u average case: about leaf nodes u A chess game may take several hours u A bridge game takes about one and 1/2 minutes Game Tree Search in Bridge Not enough time to search the game tree b =2 b =3 b =4

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 6 Reducing the Size of the Game Tree l One approach: HTN planning u Bridge is a game of planning u The declarer plans how to play the hand u The plan combines various strategies (ruffing, finessing, etc.) u If a move doesn’t fit into a sensible strategy, it probably doesn’t need to be considered Write a planning procedure procedure similar to TFD (see Chapter 11) u Generalized to generate game trees instead of just paths u Describe standard bridge strategies as collections of methods u Use HTN decomposition to generate a game tree in which each move corresponds to a different strategy, not a different card HTN-generated trees Worst case Average case Brute-force search ≈ leaf nodes≈ 26,000 leaf nodes ≈ 305,000 leaf nodes≈ 6x10 44 leaf nodes

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 7 …… Methods for Finessing PlayCard(P 3 ; S, R 3 )PlayCard(P 2 ; S, R 2 )PlayCard(P 4 ; S, R 4 ) FinesseFour(P 4 ; S) PlayCard(P 1 ; S, R 1 ) StandardFinesseTwo(P 2 ; S) LeadLow(P 1 ; S) PlayCard(P 4 ; S, R 4 ’) StandardFinesseThree(P 3 ; S) EasyFinesse(P 2 ; S)BustedFinesse(P 2 ; S) FinesseTwo(P 2 ; S) StandardFinesse(P 2 ; S) Finesse(P 1 ; S) task method 1st opponent declarer 2nd opponent dummy

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 8 (North—  Q) …… Instantiating the Methods PlayCard(P 3 ; S, R 3 )PlayCard(P 2 ; S, R 2 )PlayCard(P 4 ; S, R 4 ) FinesseFour(P 4 ; S) PlayCard(P 1 ; S, R 1 ) StandardFinesseTwo(P 2 ; S) LeadLow(P 1 ; S) PlayCard(P 4 ; S, R 4 ’) StandardFinesseThree(P 3 ; S) EasyFinesse(P 2 ; S)BustedFinesse(P 2 ; S) FinesseTwo(P 2 ; S) StandardFinesse(P 2 ; S) Finesse(P 1 ; S) Us:East declarer, West dummy Opponents:defenders, South & North Contract:East – 3NT On lead:West at trick 3 East:  KJ74 West:  A2 Out:  QT98653 (North—  3) East—  J West—  2 North—  3South—  5South—  Q task method 1st opponent declarer 2nd opponent dummy

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 9 (North—  Q) …… How to Generate a Game Tree PlayCard(P 3 ; S, R 3 )PlayCard(P 2 ; S, R 2 )PlayCard(P 4 ; S, R 4 ) FinesseFour(P 4 ; S) PlayCard(P 1 ; S, R 1 ) StandardFinesseTwo(P 2 ; S) LeadLow(P 1 ; S) PlayCard(P 4 ; S, R 4 ’) StandardFinesseThree(P 3 ; S) EasyFinesse(P 2 ; S)BustedFinesse(P 2 ; S) FinesseTwo(P 2 ; S) StandardFinesse(P 2 ; S) Finesse(P 1 ; S) (North—  3) East—  J West—  2 North—  3South—  5South—  Q

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 10 Game Tree Generated using the Methods N— QQ E— KK FINESSE N— 22 E— JJ N— 33 W— 22 E— KK S— 33 QQ 55 33 W— AA 33 E— 44 5 CASH OUT N—S— – later stratagems...

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 11 l Tignum 2: procedure to plan declarer play u Stephen Smith (then a PhD student at U. of Maryland) u Funded in part by Great Game Products (maker of Bridge Baron) l Result: u New version of Bridge Baron with significantly better declarer play u Won the 1997 world championship of computer bridge l Stephen Smith is now Lead Programmer at Great Game Products Implementation

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 12 Other Approaches l Monte Carlo simulation: u Generate many random hypotheses for how the cards might be distributed u Generate and search the game trees »Average the results l This can divide the size of the game tree by as much as 5.2x10 6 u (6x10 44 )/(5.2x10 6 ) = 1.1x10 38 »still quite large u Thus this method by itself is not enough

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: 13 Other Approaches (continued) l AJS hashing - Applegate, Jacobson, and Sleator, 1991 u Modified version of transposition tables »Each hash-table entry represents a set of positions that are considered to be equivalent »Before searching below a node, first look for a hash-table entry u Reduces the branching factor of the game tree »value calculated for one branch will be stored in the table and used as the value for similar branches l Example: suppose we have  AQ532 u We may want to classify this as  Aqxxx u View the three small cards as equivalent l GIB (winner of the 1998 world computer bridge championship) used a combination of Monte Carlo simulation and AJS hashing l Several current bridge programs do something similar

Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: Computer Bridge Championship Round-Robin (5 rounds) Semi-final Final ProgramScore Bridge Baron (USA) 147 WBridge5 (France) 145 Jack (Netherlands) 138 Micro Bridge (Japan) 131 Q-Plus Bridge (Germany) 108 Blue Chip Bridge (UK) 63 Meadowlark Bridge (USA) 36 Sabrina (France) 3 Bridge Baron Jack