Memory and Analogy in Game-Playing Agents Jonathan Rubin & Ian Watson University of Auckland Game AI Group

Overview ➲ Introduction ➲ General Game Playing ➲ Lazy Learners ➲ Memory in game-playing agents ➲ Analogical Reasoning ➲ Analogical Knowledge Transfer in GGP ➲ Conclusion

 Introduction ➲ Views and ideas about a possible approach to general game playing using memory and analogy ➲ Possible research direction ➲ Suggestions and feedback welcome

General Game Playing ➲ Unlike specialized game players such as Deep Blue ➲ Able to play different games Accept the rules of the game Play the game effectively without human intervention

Approaches to General Game Playing ➲ Partial game tree search with automated evaluation functions ➲ Approximating the minimax value by computing an exact value via simplifying abstractions of the original game

Approaches to General Game Playing ➲ Conditional Planning (One-player games) ➲ Automatic Programming – automatic generation of programs that achieve specified objectives

General Game Playing Opportunities ➲ Learning Playing multiple instances of a single game Playing multiple games against a single player

General Game Playing Opportunities ➲ Identifying common lessons that can be transferred from one game instance to another

Possible Approach to General Game Playing ➲ Lazy learning approach ➲ Record a memory of experiences ➲ Analogical reasoning to generalize beyond game domains

Lazy Learners ➲ Lazy Learners Defer processing of their inputs until they receive requests for information (Aha, 1997) Use local approaches Ability to generalize well

 Memory in Games ➲ One possible definition: Any persistent knowledge an agent has that it does not need to deduce algorithmically

Memory-based Agents ➲ GINA – Othello (De Jong & Schultz, 1988) ➲ CHEBR – Checkers (Powell et. al., 2004) ➲ Chess (Sinclair, 1998) ➲ Casper – Poker (Rubin & Watson, 2007)

Benefits of Memory ➲ Memory can be used to augment other approaches Informed pruning of game tree search – Sinclair, GINA ➲ Or, approach can be entirely based on memory alone Casper CHEBR

Experience-based, Lazy learners ➲ The use of memory has been shown to be successful in a range of specialized game domains. (Non)-Deterministic, (Im)perfect Information ➲ Lazy Learners are able to adapt well to new situations ➲ How can we extrapolate experience-based, lazy learners to handle multiple game domains?

Analogical Knowledge Transfer Our expertise is in Poker Let’s consider how our Poker cases could be used in an unknown game, e.g., “Monopoly” knowledge

Analogical Knowledge Transfer Poker cases have only three possible actions - Fold, Call & Raise These actions are useless in Monopoly But they do provide a measure of how good or strong a Poker hand is: Fold = weak Call = OK Raise = strong

Analogical Knowledge Transfer A pair (two of a kind) is the most basic Poker hand Three of a kind is stronger Obtaining all the properties of the same colour is good in Monopoly

Analogical Knowledge Transfer Higher value cards in Poker are stronger than lower value cards Higher value property is also better in Monopoly

Analogical Knowledge Transfer A straight in Poker is a good hand A continuous block of properties in Monopoly increases the chances of an opponent landing on you

Analogical Knowledge Transfer In poker you must spend money to win money knowledge

Knowledge Transfer Superficially there is nothing in common between Poker & Monopoly Knowledge is (in theory) transferable between the games knowledge ?

Conclusion ➲ In the context of General Game playing ➲ A memory-based (case-based) component may sometimes be useful ➲ Games of similar types (card, board,...) share concepts in common ➲ Should be easier to transfer knowledge between them ➲ We believe it’s also possible to transfer knowledge between games of different types

Thanks We really want community feedback on this