Jeff Berkowitz

Tonight’s Talk  Why is poker an interesting problem?  A bit about poker strategy  A simple tournament bot  A more interesting tournament bot  Onward and beerward

Before we begin … a word about “tells”  Somebody always asks about Poker “tells”  “Computers can’t read tells!” (etc.)  Answer: tells are critical to success  … if you’re making a movie.  Seriously: tells are not a big part of poker.  Questions about tells usually come from people who haven’t yet grasped what a deep intellectual challenge the game is.

Characterizing Games Open Information Limited Information Deterministic Nondeterministic Chess Checkers Go … Battleship Minesweeper … Backgammon … Poker Stock market …

Strategy (1) – Equity Basics  You hold: 5  4   Board shows: J  T 6  2   You believe your opponent has JJ or TT  She bets. Should you call?  It depends on the pot odds  4 of the 46 unseen cards give you a likely winner  Pot odds must be 11 : 1 (or so) to call Limit Poker

Strategy (2) – Controlling Equity  You Hold: K  T  Board shows: K 7 5   Pot contains 8 bets [note: small bets]  You bet your pair of kings [9 bets in pot]  Turns out, player on your left holds 9  8   Your bet helped give him the odds to call  You should have checked, hoping to raise  Faced your opponent with calling 2 bets Limit Poker

Deception (1) – Face Up Poker  Imagine Hold’em with hold cards face-up  We’ll say heads-up (2 players) for simplicity  All equity decisions are certain  Your turn: simulate a few million hands  < 1 second on modern CPUs  Model opponents as equally perfect  Bet based on the simulation outcome Limit Poker

Deception (2) – Your Hidden Cards  Now imagine you get to hide hole cards  But the other player still plays face-up ;-)  You play the same “optimal” strategy  And your opponent knows it (important)  Your bet  positive expectation  Negative for them – so they will always fold  So you must “sandbag” some of your hands  Now defining “optimal” is harder Limit Poker

NL Tourney Strategy (1) - Background  In general, for both limit and no-limit …  Pre-flop play is simpler, more formulaic  Because there is less information to go on  Post-flop play is harder  More information available  More time (flop, turn, river)  More room for deception, application of skill, …

NL Tourney Strategy (2)  But in no-limit, post-flop play is optional  All-in on the pre-flop  no post-flop decisions!  Creates worst possible equity position for others  Only possible in no-limit  Especially important [later] in tournaments  “Advance of the blinds”  Consequence: “Move-in” or “All-in” players  Book “Kill Phil” [Rodman & Nelson 2005]

Digression: Poker in Academia  University of Alberta Poker Research Group  Their work started long before the fad – 1995  Numerous papers on the internet   Excellent software (100% pure Java)  Spinout company – Poker Academy  The company is actually called “Biotools”  Product has a plug-in API for bots, “Meerkat”

Demo - KillPhilBot  Implements “Kill Phil Beginner” strategy  From ibid (“Kill Phil”, Rodman & Nelson)  Modified with a slightly smarter postflop

Rules, rules, lotsa rules  A nontrivial rule-based bot gets hairy  If this else if this then that else if the other thing…  Tracing scattered throughout  Must restart Poker Academy to change the rules  Solution: separate behaviors from code  “Rule” (not really) interpretation system  “In the future, every programmer will have their own language” (me, 2008)  Poker Rules Definition Language - PRDL

PRDLBot (“Phil Purdle”)  Express decision tree in PRDL  Parse with JJTree / javacc  Evaluate the parse tree for the action  No code generation (although there could be)  Advantages:  Put tracing in the engine not the rules definition  Change and reload without restarting PA  Not fully implemented yet

PRDL Example (1) stage = PREFLOP // all identifiers refer back to bot core { m > 30 // “when we have a huge stack …” { holeCards in [KPGROUP_1, QQ, KQs] // use of a list of expressions { toCall <= BLIND // or write "numberOfRaises = 0" { bettors = 0 // no limpers { action [BET, (bankroll / 6)] // another use of a list of expressions } : // “otherwise, there are limpers”; push all-in { action [BET, bankroll] } // CONTINUES NEXT SLIDE

PRDL Example (2) : // otherwise, raised pot { numberOfRaises = 1 { action [BET, max[5 * raise, bankroll / 6]] } : // multiple raisers { action [BET, bankroll] } : // otherwise big stack, we don't have ultra-strong cards: we're done { action CHECK_FOLD // use of an expression that is not a list } : // Post-flop { //..... etc..... }

PRDLBot Tour

Bibliography (1 of 2)  U of Alberta (CA) Poker Research Group:   Scroll to Publications. Recommended there:   Great online bot programming articles:   Poker Academy software:  

Bibliography (2 of 2)  Best overall poker web site / book publishers:   Recommended books from them:  See  Best for beginners and for learning basic strategy are Ed Miller’s books, Gettings Started in Hold’em and Small Stakes Hold’em: Winning Big with Expert Play.  Best for learning tournament play are the Harrington and Robertie book series, Harrington on Hold’em (volumes 1 through 3, to be studied in order).

Online Poker  Early years (1990s – 2002)  IRC, academic, first commercial sites  Fad (2003 – 2006)  TV with exposed hole cards, Chris Moneymaker  Reality ( )  UIGEA, bot suspicions  No individual has ever been arrested or charged with any crime for playing poker online in the United States.

Cheating at Online Poker  General approach  Write bot to Meerkat interfaces  Test and debug to high level of skill in PA  Emulate Meerkat on a commercial poker client  Consider Omaha  4 hole cards, must play 2  Like playing 6 Hold’em hands simultaneously  Often played for a split high/low pot (12 hands)  Computer advantage is maximized