The challenge of poker NDHU CSIE AI Lab 羅仲耘

2004/11/04the challenge of poker2 Outline Introduction Texas Hold’em rules Poki’s architecture Betting Strategy –Pre-flop betting strategy –Basic betting strategy Reference

2004/11/04the challenge of poker3 Introduction Bridge and poker involve imperfect information Traditional methods like deep search can’t play these game well

2004/11/04the challenge of poker4 Texas Hold’em rules (1) 1. Opening deal – Each player is dealt two cards face down, which are known as hole cards or pocket cards. 2. First round of betting – Starting with the player to the left of the big blind, each player can call the big blind, raise, or fold. The big blind has the option to raise an otherwise unraised pot.

2004/11/04the challenge of poker5 Texas Hold’em rules (2) 3. The flop – The dealer burns a card, and then deals three community cards face up. The first three cards are referred to as the flop, while all of the community cards are collectively called the board.

2004/11/04the challenge of poker6 Texas Hold’em rules (3) 4. Second round of betting – Starting with the player to the left of the dealer button, each player can check or bet. Once a bet has been made, each player can raise, call, or fold.

2004/11/04the challenge of poker7 Texas Hold’em rules (4) 5. The turn – The dealer burns another card, and then adds a fourth card face-up to the community cards. This fourth card is known as the turn card, or fourth street.

2004/11/04the challenge of poker8 Texas Hold’em rules (5) 6. Third round of betting – It follows the same format as the second round, but the size of the bets have usually doubled in limit games.

2004/11/04the challenge of poker9 Texas Hold’em rules (6) 7. The river – The dealer burns another card, and then adds a fifth and final card to the community cards. This fifth card is known as the river card, or fifth street.

2004/11/04the challenge of poker10 Texas Hold’em rules (7) 8. Final round of betting – It follows the same format as the second and third rounds. 9. The showdown – Using the best five- card combination of their hole cards and the community cards, the remaining players show their hands, with the bettor or last raiser showing first. The highest five-card hand wins the pot. (In case of a tie, the pot is evenly split among the winning hands.)

2004/11/04the challenge of poker11 Poki’s architecture

2004/11/04the challenge of poker12 Pre-flop betting strategy(1) Income rate: base on roll simulation Roll simulation: an off-line computation that consists of playing several million hands (trials) where all players call the first bet, and then all the remaining cards are dealt out without any further betting David Sklansky wrote books on this game and classify the hand cards.

2004/11/04the challenge of poker13 Pre-flop betting strategy(2)

2004/11/04the challenge of poker14 Pre-flop betting strategy(3) Prefer roll-out simulation than David Sklansky’s classification –Roll-out simulation information is quantitative rather than qualitative –Can apply many difference specific situation

2004/11/04the challenge of poker15 Basic betting strategy(1) Hand strength (HS): is the probability that a given hand is better than that of an active opponent For example: hand is A ♦ -Q ♣ and the flop is J ♥ -4 ♣ -3 ♥.47 remaining unknown cards and therefore {47 choose 2} = 1081 possible hands an opponent might hold. 444 cases are better than hand, 9 are equal, and 628 are worse than hand. There is a 58.5% chance that hand is better than a random hand. If there are 5 opponents, the HS 5 is = 6.9%

2004/11/04the challenge of poker16 Basic betting strategy(2) Hand potential –Positive potential (PPot): Chance that a hand which is not currently the best improves to win at the showdown –Negative potential (NPot): Chance that a currently leading hand ends up losing All possible opponent cards is {47 choose 2} = 1081, therefore remains {45 choose 2} = 990 possible turn and river cards to consider for each opponent

2004/11/04the challenge of poker17 Basic betting strategy(3) For example: if the hand A ♦ -Q ♣ is ahead against one opponent after 5 cards, then after 7 cards there is a / = 72% chance of still being ahead

2004/11/04the challenge of poker18 Basic betting strategy(4) Effective hand strength (EHS): combines hand strength and potential to give a single measure of the relative strength of Poki’s hand against an opponent P(win) = P(ahead) x P(opponent does not improve) + P(behind) x P(we improve) = HS x (1 - NPot) + (1 - HS) x PPot

2004/11/04the challenge of poker19 Basic betting strategy(5) Weighting the enumerations –The probability of each hand being played to a particular point in the game will very –For example: the probability that the opponent holds Ace-King is much higher than 7-2 after the flop, because most will fold 7-2 before the flop

2004/11/04the challenge of poker20 Basic betting strategy(6) To account for this, Poki maintains a weight table for each opponent The weights have a value in the range zero to one

2004/11/04the challenge of poker21 Basic betting strategy(7) For example: [ A ♠ K ♣, 0.40 ],…[ Q ♦ 2 ♦, 0.20 ],… the PredictOpponentAction procedure generates probability distributions { P( fold), P( check / call), P( bet / raise)} A ♠ - K ♣ { 0.0, 0.7, 0.3 } Q ♦ - 2 ♦ { 0.0, 0.1, 0.9 } If the observed player action is a bet, after reweighting, the new table entry for A ♠ - K ♣ will be 0.4 x 0.3 = 0.12 Q ♦ - 2 ♦ will be 0.2 x 0.9 = 0.18

2004/11/04the challenge of poker22 Basic betting strategy(8) Probability triple PT = {f, c, r}, such that f + c + r = 1.0, representing the probability distribution that the next betting action in a given context is fold, call, or raise, respectively

2004/11/04the challenge of poker23 Reference The challenge of Poker / Darse Billings, Aaron Davidson, Jonathan Schaeffer, Duane Szafron / Artificial Intelligence 134 (2002)

2004/11/04the challenge of poker24 Thanks for your attention!