1. 2006 AAAI Computer Poker Competition Michael Littman Rutgers University Martin Zinkevich Christian Smith Luke Duguid U of Alberta

2. What is Poker? The OR View –A partial information game with over 10 18 states. The AI View –A huge opponent modeling challenge. The Public View –Really popular! Lots of fun!

3. What is the Game? Play 1000 hands of Heads-up Limit Texas Hold’em Poker against an opponent Reset bots, switch seats, and play again with the same hands. –lower variance –more fair Repeat 6-20 times –Can treat the outcome of the duplicate match as a random variable which we have sampled several times.

4. How do you win? Make money! Bankroll Competition –Against overall, have the maximum total bankroll –Highlights opponent modeling and learning Series Competition –Against individuals, have positive bankroll –Highlights the OR problem of “solving” the game

5. The Teams Hyperborean: University of Alberta Edmonton, Canada Bluffbot: Finland Monash: Monash University, Australia Teddy: USA Gs2: Carnegie Mellon University, Pittsburgh, USA

6. Before I Begin There is a period that is allocated for competitors to contest the results that has yet to expire.

7. Bankroll Results CompetitorWinnings (small bets/hand) Teddy (USA) -0.4616 sb/h

8. Bankroll Results CompetitorWinnings (small bets/hand) Monash (Monash U., Australia) -0.0273 sb/h Teddy (USA) -0.4616 sb/h

9. Bankroll Results CompetitorWinnings (small bets/hand) Bluffbot (Finland) 0.0954 sb/h Monash (Monash U., Australia) -0.0273 sb/h Teddy (USA) -0.4616 sb/h

10. Bankroll Results CompetitorWinnings (small bets/hand) Hyperborean (U. Alberta) 0.3935 sb/h Bluffbot (Finland) 0.0954 sb/h Monash (Monash U., Australia) -0.0273 sb/h Teddy (USA) -0.4616 sb/h

11. Bankroll-Heads Up Hyperborean (U Alberta) Bluffbot (Finland) Monash (Monash U) Teddy (USA) Hyperborean (U Alberta) Hyperborean Wins Hyperborean Wins Hyperborean Wins Bluffbot (Finland) Hyperborean Wins Bluffbot WinsTeddy Wins Monash (Monash U) Hyperborean Wins Bluffbot WinsMonash Wins Teddy (USA) Hyperborean Wins Teddy WinsMonash Wins

12. Bankroll-Heads Up (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Monash (Monash U) Teddy (USA) Hyperborean (U Alberta) 0.0514 ±0.0171 0.7227 ±0.0161 0.4067 ±0.0247 Bluffbot (Finland) -0.0514 ±0.0171 0.5271 ±0.0197 -0.1895 ±0.0289 Monash (Monash U) -0.7227 ±0.0161 -0.5271 ±0.0197 1.1678 ±0.0427 Teddy (USA) -0.4067 ±0.0247 0.1895 ±0.0289 -1.1678 ± 0.0427

13. Bankroll-Overall Significance Difference between Bluffbot and U of Alberta 0.2982 small bets/hand 0.0190 standard deviations

14. The Most Interesting Result In the bankroll competition, in head-to- head, BluffBot beat Monash who beat Teddy who beat BluffBot A practical example of the non-transitivity of poker

15. Results-Series (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Gs2 (CMU) Monash (Monash U) Hyperborean (U Alberta) Winner: Hyperborean Winner: Hyperborean Winner: Hyperborean Bluffbot (Finland) Winner: Hyperborean Winner: Bluffbot Winner: Bluffbot Gs2 (CMU) Winner: Hyperborean Winner: Bluffbot Winner: Gs2 Monash (Monash U) Winner: Hyperborean Winner: Bluffbot Winner: Gs2

16. Results-Series (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Gs2 (CMU) Monash (Monash U) Hyperborean (U Alberta) 0.1145 ±0.0375 0.1843 ±0.0079 0.7344 ±0.029 Bluffbot (Finland) -0.1145 ±0.0375 0.1200 ±0.0561 0.5214 ±0.0408 Gs2 (CMU) -0.1843 ±0.0079 -0.1200 ±0.0561 0.6512 ±0.0397 Monash (Monash U) -0.7344 ±0.029 -0.5214 ±0.0398 -0.6512 ±0.0398

17. Things to Never Assume All bots will work on the competition machines the first time The server code is bug-free Everybody has a common idea of the rules of poker (or even heads-up Texas Hold’em) People can write code instantaneously

18. Exhibitionary Aspects of the Competition Bots submitted late Bots debugged after the deadline The time limit was very large for the series competition

19. Maybe Next Year More advance notice Competitors need access to one of the machines they will use Server code needs to be frozen before the competition begins More variants of poker need to be included, especially: –>2 players –>1000 hands There has to be a 7 sec/hand time limit

20. Is Poker “Solved”? No one has ever solved a four-round abstraction of poker without a partition into the early game and the late game. The game of poker is also about your opponent. For instance, playing rock- paper-scissors against a four-year-old is different than against an adult.

21. Next Year This summer: write the rules This winter: write the code Next summer: next competition

22. Are You Interested? Come talk to us maz@cs.ualberta.ca bowling@cs.ualberta.ca

23. Summary The poker competition brought together five teams in two competitions. The competition was very close, and very interesting, non-transitive (A beats B beats C beats A) performance was observed. A freeware codebase was developed for future competitions.