1. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
7/6/2018
Today’s Topics
Playing Deterministic (no Dice, etc) Games
Mini-max
- pruning
ML and games?
1997: Computer Chess Player (IBM’s Deep Blue) Beat Human Chess Champ (Kasparov)
2015: World-Class Computer GO Player
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

2. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
Mini-Max Algorithm
Generate Search Tree of ALL Possible Games
Score Leaves (Assume Higher Better for ‘You’)
eg, +1 you win, -1 you lose, 0 if a draw (ie, a tie)
Assume Opponent Scores ‘Board Configurations’ the Same Way You Do (more on this later)
Propagate Info from Leaves to Root of Search Tree, then Choose Best Move
Your Turn: choose MAX score
Opponent’s Turn: choose MIN score (which is best for opponent)
Only Choose NEXT Move; When Your Turn Again, Repeat
opponent might not have done what you expected
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

3. Tic-Tac-Toe (https://en.wikipedia.org/wiki/Tic-tac-toe)
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

4. What If Game More Complex?
CS 540 Fall 2015 (Shavlik)
7/6/2018
What If Game More Complex?
Chess: approx 35 moves per turn and 50 moves/player/game, so O(35100) possible board configurations!
Better estimate: en.wikipedia.org/wiki/Shannon_number
Number of atoms in the Earth:
Solution: project ahead as far as possible, then use a ‘static board evaluator’ (SBE) to score in-progress board configurations
(Common) Abuse of big-O notation
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

5. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
SBE’s
No Perfect SBE Exists for Chess (otherwise we’d know if black can always win, etc)
But a lot of Domain K can be Put in the SBE
You Might Have Learned “Piece Values” in Chess SBE =  (#myPawns - #theirPawns) …  (#myQueens - #theirQueens)
+ points for ‘center of board’ control + etc
Can Run Mini-Max with SBE
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

6. Saving Unnecessary Calculations: - Pruning
Can Prune Away Subtrees by Realizing no Matter their Score, they won’t Impact Choice
We Won’t Cover the Full - Pruning Algorithm, Since its Applicability Across AI is Limited
But We’ll Cover some Simple Cases of the Two Types of Pruning
Ask: Would it Matter if this Node had Value  or Value -? If Decision Same in Both Cases, No Need to Compute Node’s SBE Value
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

7. An -Cutoff Example (view in Slide Show mode to see animation)
move
Tip: Always fully compute the left-most subtree of root
 2
Us (max) 2
 1
Them (min) 2 7 1
big subtree ?
If good for Us, Opponent will not go here.
If bad for Us, we will take left branch from root.
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

8. A -Cutoff Example (view in Slide Show mode to see animation)
move
Recall: Always compute the left-most subtree of root
 7
Them (min) 7
 9
Us (max) 2 7 9
big subtree ?
If good for Us, Opponent will take left branch from root.
If bad for Us, we will take left branch.
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

9. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
Note
We project ahead as far as we can in the time we have, but only
choose our next move
Next time it is our turn, we repeat - calc
(possibly repeating some work, but too much to store)
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

10. Performance Improvement
Best case: - can double search depth considered is fixed amount of time (doubling depth exponentially increases boards considered!) Worst case: no savings Note: - and mini-max return SAME answer (- simply avoids wasted work)
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

11. Some Built-in Assumptions
Projecting Ahead k Moves will Lead to Better SBE Estimates than Simply Applying SBE to the Legal Next Boards
seems reasonable, but isn’t guaranteed
Our Opponent Thinks the Same Way We Do
no consideration of ‘traps’ and tricking opponent
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

12. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
Horizon Effect
Defn: Since We Usually Stop before Game Ends, ‘Danger’ might be Lurking just Beyond our Search Depth One (Partial) Soln: Once Best Move Chosen, Look Ahead a Few Moves More if turns out bad, spend more time searching this turn
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

13. Perils of Not-Looking Ahead
10/11/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

14. Chess: The ‘E. coli’ of AI?
Chess Originally Thought to be a Great Testbed for Advancing AI
Has had Moderate Impact on AI Progress
Not Much ‘Technology Transfer’ to AI Tasks
Played Minor Role in ‘ML Revolution’
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6

15. CS 540 - Fall 2016 (Shavlik©), Lecture 11, Week 6
ML and Games
ML Led to World-Class Backgammon Player Decades Ago
ML Produced World-Class Poker Player over Last Few Years
World-class GO Player via ‘Deep [Neural] Networks’ (2015)
Deep Networks Recently Learned to Play Well without being given Games’ Rules
10/13/16
CS Fall 2016 (Shavlik©), Lecture 11, Week 6