1. Memory and Analogy in Game-Playing Agents Jonathan Rubin & Ian Watson University of Auckland Game AI Group http://www.cs.auckland.ac.nz/research/gameai

2. Overview ➲ Introduction ➲ General Game Playing ➲ Lazy Learners ➲ Memory in game-playing agents ➲ Analogical Reasoning ➲ Analogical Knowledge Transfer in GGP ➲ Conclusion

3.  Introduction ➲ Views and ideas about a possible approach to general game playing using memory and analogy ➲ Possible research direction ➲ Suggestions and feedback welcome

4. General Game Playing ➲ Unlike specialized game players such as Deep Blue ➲ Able to play different games Accept the rules of the game Play the game effectively without human intervention

5. Approaches to General Game Playing ➲ Partial game tree search with automated evaluation functions ➲ Approximating the minimax value by computing an exact value via simplifying abstractions of the original game

6. Approaches to General Game Playing ➲ Conditional Planning (One-player games) ➲ Automatic Programming – automatic generation of programs that achieve specified objectives

7. General Game Playing Opportunities ➲ Learning Playing multiple instances of a single game Playing multiple games against a single player

8. General Game Playing Opportunities ➲ Identifying common lessons that can be transferred from one game instance to another

9. Possible Approach to General Game Playing ➲ Lazy learning approach ➲ Record a memory of experiences ➲ Analogical reasoning to generalize beyond game domains

10. Lazy Learners ➲ Lazy Learners Defer processing of their inputs until they receive requests for information (Aha, 1997) Use local approaches Ability to generalize well

11.  Memory in Games ➲ One possible definition: Any persistent knowledge an agent has that it does not need to deduce algorithmically

12. Memory-based Agents ➲ GINA – Othello (De Jong & Schultz, 1988) ➲ CHEBR – Checkers (Powell et. al., 2004) ➲ Chess (Sinclair, 1998) ➲ Casper – Poker (Rubin & Watson, 2007)

13. Benefits of Memory ➲ Memory can be used to augment other approaches Informed pruning of game tree search – Sinclair, GINA ➲ Or, approach can be entirely based on memory alone Casper CHEBR

14. Experience-based, Lazy learners ➲ The use of memory has been shown to be successful in a range of specialized game domains. (Non)-Deterministic, (Im)perfect Information ➲ Lazy Learners are able to adapt well to new situations ➲ How can we extrapolate experience-based, lazy learners to handle multiple game domains?

15. Analogical Knowledge Transfer Our expertise is in Poker Let’s consider how our Poker cases could be used in an unknown game, e.g., “Monopoly” knowledge

16. Analogical Knowledge Transfer Poker cases have only three possible actions - Fold, Call & Raise These actions are useless in Monopoly But they do provide a measure of how good or strong a Poker hand is: Fold = weak Call = OK Raise = strong

17. Analogical Knowledge Transfer A pair (two of a kind) is the most basic Poker hand Three of a kind is stronger Obtaining all the properties of the same colour is good in Monopoly

18. Analogical Knowledge Transfer Higher value cards in Poker are stronger than lower value cards Higher value property is also better in Monopoly

19. Analogical Knowledge Transfer A straight in Poker is a good hand A continuous block of properties in Monopoly increases the chances of an opponent landing on you

20. Analogical Knowledge Transfer In poker you must spend money to win money knowledge

21. Knowledge Transfer Superficially there is nothing in common between Poker & Monopoly Knowledge is (in theory) transferable between the games knowledge ?

22. Conclusion ➲ In the context of General Game playing ➲ A memory-based (case-based) component may sometimes be useful ➲ Games of similar types (card, board,...) share concepts in common ➲ Should be easier to transfer knowledge between them ➲ We believe it’s also possible to transfer knowledge between games of different types

23. Thanks We really want community feedback on this