Memory and Analogy in Game-Playing Agents

Jonathan Rubin & Ian Watson

University of Auckland Game AI Grouphttp://www.cs.auckland.ac.nz/research/gameai

Overview➲ Introduction

➲ General Game Playing

➲ Lazy Learners

➲ Memory in game-playing agents

➲ Analogical Reasoning

➲ Analogical Knowledge Transfer in GGP

➲ Conclusion

Introduction

➲ Views and ideas about a possible approach to general game playing using memory and analogy

➲ Possible research direction

➲ Suggestions and feedback welcome

General Game Playing

➲ Unlike specialized game players such as Deep Blue

➲ Able to play different games Accept the rules of the game

Play the game effectively without human

intervention

Approaches to General Game Playing

➲ Partial game tree search with automated evaluation functions

➲ Approximating the minimax value by computing an exact value via simplifying abstractions of the original game

Approaches to General Game Playing

➲ Conditional Planning (One-player games)

➲ Automatic Programming – automatic generation of programs that achieve specified objectives

General Game PlayingOpportunities

➲ Learning

Playing multiple instances of a single game

Playing multiple games against a single player

General Game PlayingOpportunities

➲ Identifying common lessons that can be transferred from one game instance to another

Possible Approach toGeneral Game Playing

➲ Lazy learning approach

➲ Record a memory of experiences

➲ Analogical reasoning to generalize beyond game domains

Lazy Learners

➲ Lazy Learners Defer processing of their inputs until they

receive requests for information (Aha,

1997)

Use local approaches

Ability to generalize well

Memory in Games

➲ One possible definition:

Any persistent knowledge an agent has that it does not need to deduce algorithmically

Memory-based Agents

➲ GINA – Othello (De Jong & Schultz, 1988)

➲ CHEBR – Checkers (Powell et. al., 2004)

➲ Chess (Sinclair, 1998)

➲ Casper – Poker (Rubin & Watson, 2007)

Benefits of Memory

➲ Memory can be used to augment other approaches

Informed pruning of game tree search –

Sinclair, GINA

➲ Or, approach can be entirely based on memory alone

Casper

CHEBR

Experience-based, Lazy learners

➲ The use of memory has been shown to be successful in a range of specialized game domains.

(Non)-Deterministic, (Im)perfect Information

➲ Lazy Learners are able to adapt well to new situations

➲ How can we extrapolate experience-based, lazy learners to handle multiple game domains?

Analogical Knowledge Transfer

Our expertise is in PokerLet’s consider how our Poker cases could be used in an unknown game, e.g., “Monopoly”

knowledgeknowledge

Analogical Knowledge Transfer

Poker cases have only three possible actions - Fold, Call & RaiseThese actions are useless in MonopolyBut they do provide a measure of how good or strong a Poker hand is: Fold = weak Call = OK Raise = strong

Analogical Knowledge Transfer

A pair (two of a kind) is the most basic Poker hand

Three of a kind is stronger

Obtaining all the properties of the same colour is good in Monopoly

Analogical Knowledge Transfer

Higher value cards in Poker are stronger than lower value cards

Higher value property is also better in Monopoly

Analogical Knowledge Transfer

A straight in Poker is a good hand

A continuous block of properties in Monopoly increases the chances of an opponent landing on you

Analogical Knowledge Transfer

In poker you must spend money to win money

knowledgeknowledge

Knowledge Transfer

Superficially there is nothing in common between Poker & Monopoly

Knowledge is (in theory) transferable between the games

knowledgeknowledge

?

Conclusion

➲ In the context of General Game playing➲ A memory-based (case-based) component

may sometimes be useful➲ Games of similar types (card, board, ...)

share concepts in common➲ Should be easier to transfer knowledge between them

➲ We believe it’s also possible to transfer knowledge between games of different types

ThanksWe really want community feedback on this