Selective Search in Games of Different Complexity Maarten Schadd

Playing Chess

Computer vs. Human Intuition Feelings Only few variations Aggressive Pruning Selective Search Calculator Fast Examines most variations Safe Pruning Brute-Force Search

Problem Statement How can we improve selective-search methods in such a way that programs increase their performance in domains of different complexity ?

Domains of Different Complexity One-Player Game No Chance Perfect Information Two-Player Game No Chance Perfect Information Two-Player Game Chance or Imperfect Information Multi-Player Game No Chance Perfect Information

One-Player Game No Chance – Perfect Information Research question 1 How can we adapt Monte-Carlo Tree Search for a one-player game?

One player games No opponent! No uncertainty! Why not use all time at the beginning? Deviation on the score of moves

SameGame

Single-Player Monte Carlo Tree Search Selection Strategy – Expansion Strategy –Same Simulation Strategy –TabuColourRandom Policy Back-Propagation Strategy –Average Score, Sum of Squared Results and Best Result achieved so far

Experiments – Simulation Strategy 250 random positions 10 million nodes in memory

One search or several?

Parameter tuning

Highscores DBS 72,816 SP-MCTS(1)73,998 SP-MCTS(2)76,352 MC-RWS76,764 Nested MC77,934 SP-MCTS(3)78,012 Spurious AI84,414 HGSTS84,718

Position 1 – Move 0

Position 1 – Move 10

Position 1 – Move 20

Position 1 – Move 30

Position 1 – Move 40

Position 1 – Move 52

Position 1 – Move 53

Position 1 – Move 63

Two-Player Game No Chance – Perfect Information Research Question 2 How can we solve a two-player game by using Proof-Number Search in combination with endgame database?

Two-Player Game No Chance – Perfect Information Proof-Number SearchEndgame Databases

Fanorona

Average Branchin Factor

Average Number of Pieces

Endgame Database Statistics

Two-Player Game No Chance – Perfect Information 130,820,097,938 nodes Fanorona solved – Draw!

Two-Player Game Chance or Imperfect Information Research Question 3 How can we perform forward pruning at chance nodes in the expectimax framework?

Two-Player Game Chance or Imperfect Information ChanceProbCut Predictions based on shallow search

ChanceProbCut

Stratego

Predicting Stratego

Node Reduction

Performance gain

Multi-Player Game No Chance – Perfect Information Research Question 4 How can we improve search for multi-player games?

What games do you play?

Coalitions

Multi-Player Game No Chance – Perfect Information MaxN

Multi-Player Game No Chance – Perfect Information Paranoid

Max^n ,2,6,35,5,1,24,1,6,81,1,3,17,2,9,54,5,6,71,5,0,85,2,1,4 6,2,6,3 4,1,6,8 7,2,9,5 6,2,6,3 5,2,1,4 7,2,9,5

Paranoid 1 7,2,9,5 7-9

Paranoid <=

Multi-Player Game No Chance – Perfect Information Best-Reply Search

Only 1 opponent plays Chose opponent –Strongest counter move Other opponents have to pass Long term planning Less paranoid Pruning possible

Best-Reply Search 1 2,3, <= ,3,

Chinese Checkers

Focus

Rolit

Experiments 3 Players: 6 setups 4 Players: 14 setups 6 Players: 62 setups

Validation

Average Depth

BRS vs. Max^n

BRS vs. Paranoid

BRS vs. Max^n vs. Paranoid

Multi-Player Game No Chance – Perfect Information New Search Algorithm: Best-Reply Search Ignoring Opponents Long-Term Planning Illegal Positions don’t disturb Generally Stronger than Max^n and Paranoid

Conclusions We have investigated four ways to improve selective search methods –Single-Player Monte-Carlo Tree Search –Proof-Number search + endgame databases –ChanceProbCut –Best-Reply Search

Future Research Testing selective search methods in other domains –Other games in the same complexity level –Games of other complexity levels

Thank you for your attention!