Adversarial Environments/ Game Playing Why? Fun  Many real-world applications in game theory Economics Industry/Operations Research

Types of games we’ll consider 2 player Zero-sum: one player’s loss is the other’s gain Fully observable environment Deterministic Exs: chess, checkers, tic-tac-toe Not: dice, black jack, poker

Strategic thinking ?= intelligence Two-player games have been a focus of AI since its inception… Clear success criteria Begs the question: Is strategic thinking the same as intelligence?

How humans play games… An experiment (by Adriann deGroot) was performed where valid chess boards were shown to novice and expert players… - experts could reconstruct these perfectly - novice players did far worse…

How humans play games… An experiment (by Adriann deGroot) was performed where valid chess boards were shown to novice and expert players… - experts could reconstruct these perfectly - novice players did far worse… Random chess positions (non-legal ones) were then shown to the two groups - experts and novices did just as badly at reconstructing them!

What makes a human play well? Implications? What makes a human play well?

Implications? What makes a human play well? Problem solving Visual Memory

Strategic thinking ?= intelligence Humans and computers have different relative strengths in these games: humans computers good at evaluating the strength of a board good at looking ahead in the game to find winning combinations of moves

Tic Tac Toe as search How can we pose this as a search problem?

Baby Nim For winning strategy, should I take (1 or 2 matches)? MAX MIN Action: Take 1 or 2 matches at each turn Goal: Take the last match MAX MIN For winning strategy, should I take (1 or 2 matches)? MAX

Baby Nim MAX MIN MAX MIN MAX 1 2 W L MAX (5) MIN (4) (3) 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (3) (1) MAX MAX (2) MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) W = 1.0 1 1 1 2 1 MAX MAX (1) L = -1.0 MAX loses/ MIN wins ( ) = # of remaining matches

Baby Nim +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 1 2 W MAX (5) MIN (4) (3) MIN 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (1) MAX (3) MAX (2) MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) +1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L L L L L = -1.0 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Baby Nim +1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 1 2 W MAX (5) (4) (3) MIN 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) MAX (3) (1) MAX (2) MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) +1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L L L L L = -1.0 +1 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Baby Nim -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 1 2 W MAX (5) MAX MIN (4) (3) MIN 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (1) MAX (3) MAX (2) MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) -1 -1 -1 +1 -1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L = -1.0 +1 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Baby Nim +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN 2 W MAX (5) MAX MIN (4) (3) MIN 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (1) MAX (3) MAX (2) +1 +1 -1 +1 MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) -1 -1 -1 +1 -1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L = -1.0 +1 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Baby Nim +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX 2 W MAX (5) MAX MIN (4) (3) +1 MIN -1 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (3) (1) MAX MAX (2) +1 +1 -1 +1 MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) -1 -1 -1 +1 -1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) W = -1.0 +1 -1 -1 -1 -1 MIN wins/ MAX loses +1 ( ) = # of remaining matches

Baby Nim +1 +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 2 W MAX (5) MAX +1 MIN (4) (3) +1 MIN -1 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) MAX (3) (1) MAX (2) +1 +1 -1 +1 MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) -1 -1 -1 +1 -1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L = -1.0 +1 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Baby Nim +1 +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX can always win by taking 2 on first turn 1 2 W MAX (5) MAX +1 MIN (4) (3) +1 MIN -1 2 1 2 1 Take 1 or 2 at each turn Goal: take the last match (2) (3) (1) MAX MAX (2) +1 +1 -1 +1 MAX wins 1 2 1 2 1 2 1 MIN (2) (1) MIN (1) (1) -1 -1 -1 +1 -1 +1 +1 W = 1.0 1 1 1 2 1 MAX MAX (1) L = -1.0 +1 -1 -1 -1 -1 MAX loses/ MIN wins +1 ( ) = # of remaining matches

Tree Size Tree size gets very large quickly 0 Ply Branching factor = 7

Tree Size Tree gets very large quickly 0 Ply Branching factor = 7 Branching Factor Estimates for different two-player games 2 Ply Tic-tac-toe 4 Connect Four 7 Checkers 10 Othello 30 Chess 35 Go 300

Search Times On average: Tic Tac Toe: b ≈ 4, d ≈9  solvable Chess, b ≈ 35, d ≈100 for reasonable games  exact solution is infeasible!

Tree Size Tree gets very large quickly “solved” games 0 Ply Branching factor = 7 1 Ply Branching Factor Estimates for different two-player games 2 Ply Tic-tac-toe 4 Connect Four 7 Checkers 10 Othello 30 Chess 35 Go 300 “solved” games computer-dominated Google’s AlphaGo (2015)

Tree Size Tree gets very large quickly How to reduce search time? 0 Ply Branching factor = 7 1 Ply Branching Factor Estimates for different two-player games 2 Ply Tic-tac-toe 4 Connect Four 7 Checkers 10 Othello 30 Chess 35 Go 300 “solved” games How to reduce search time? computer-dominated Google’s AlphaGo (2015)