1. Yikan Chen (yc2r@virginia.edu) Weikeng Qin (wq7yt@virginia.edu) 1

2. 2 Evolutionary Algorithm Poker!

3. 3

4.  Evolution Process 4 Crossover Mutation Natural Selection  Evolutionary Algorithm

5.  Encoding and Crossover 5 11100110 01001011 00110 01011 010 111

6.  Mutation 6 11100110 11000111

7.  Natural Selection 7 Run the roulette-wheel selection based on the fitness value of candidates

8.  Important Parameters  Crossover rate  Mutation rate  Elite rate  Fitness function  Demo http://userweb.elec.gla.ac.uk/y/yunli/ga_demo/ 8

9.  AKQ 2-player game  $1 blinds for each player  Player1 bet or fold  Player2 call or fold 9

10.  Derive the optimal strategy using EA  Chromosomal representations  Fij: fold threshold when Pi got Cardj  Fitness functions 10 Card1Card2Card3 P12/300 P212/30

11.  Fitness functions  Fi: fitness function  Wij: money won by candidate I against candidate j 11

12. 12

13. 13

14. 14  Decreased fluctuation  Further decreased fluctuation 400-500 generations Var(f11) ; Var(f22) Mean(f11); Mean(f22) Count only wins.065;.067.67;.60 Penalize failure.037;.035.67;.70 Penalize Failure heavier.028;.024.67;.74

15.  Real Texas Hold’em  Encoding Strategy (Turn and River)  Hand strength (player confidence)  Fraction of opponent raise (opponent confidence)  Total raise (profit) 15

16.  Fitness Criterion 16

17.  Performance 17

18. 18

19. 19 ∑ ∑ w1w1 w2w2 wnwn b …… a1a1 a2a2 anan 1 f f output

20. 20 Input output Hidden Layer

21.  Simplest Encoding Method 21 a a b b c c d d d d c c b b a a

22.  http://www.cs.utexas.edu/users/nn/ http://www.cs.utexas.edu/users/nn/  Neuro Evolution of Augmenting Topologies  Encoding Strategy: Node-based  Neuron gene table  Link gene table  Innovation number  Global database of innovations  Each innovation has unique ID number 22

23. 23

24.  Mutation  Perturb weights  Add a link gene  Add a neuron gene  Crossover  By innovation number 24

25.  Crossover 25 2 2 3 3 5 5 6 6 4 4 3 3 1 1 2 2 5 5 4 4 1 1 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 8 1->5 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 6 5->6 7 6->4 9 3>5 10 1->6

26.  Crossover 26 2 2 3 3 5 5 6 6 4 4 1 1 8 1->5 1 1->4 2 2->4 3 3->4 4 2->5 5 5->4 6 5->6 7 6->4 9 3>5 10 1->6

27.  Simplified Poker Model  1-10  Initial credit: 10 chips  One chip ante at the beginning  Call, raise (1 chip each time), fold  Tournament 27

28. 28 Two player game

29. 29

30.  Four different types of opponents 30 Tight Aggressive (TA)Tight Passive (TP) Loose Aggressive (LP)Loose Passive (LP)

31.  α: min win probability to call  β: min win probability to raise 31

32. 32 A: player type B: player action

33. 33

34.  Bluffing…… 34

35. 35 Thanks!