influence map

Maurice Bergsma and Peter MeijMaurice Bergsma and Peter Meij

Student Lecture GAI, May 3, 2006Student Lecture GAI, May 3, 2006

Influence MapsInfluence Maps

ContentsContents

Influence MapsInfluence Maps Reasoning with Influence MapsReasoning with Influence Maps Sample ApplicationsSample Applications

Recognizing Strategic DispositionsRecognizing Strategic Dispositions Environmental Awareness in Game AgentsEnvironmental Awareness in Game Agents

Issues with Influence MapsIssues with Influence Maps ConclusionsConclusions

Introduction / Influence Maps / Reasoning / App 1 / App 2 / Issues / Conclusions

Influence MapsInfluence Maps

Method for (runtime) spatial analysisMethod for (runtime) spatial analysis Matrix representing the game worldMatrix representing the game world Each cell stores data for one aspect of that part Each cell stores data for one aspect of that part

of the world. of the world. Values propagate to nearby cellsValues propagate to nearby cells This shows the influence of certain objects on This shows the influence of certain objects on

each part of the worldeach part of the world Useful for many types of gamesUseful for many types of games


Influence Maps (2)Influence Maps (2)

Different types of propagation are possibleDifferent types of propagation are possible LinearLinear ExponentialExponential GaussianGaussian Etc.Etc.

Game-specific, must be tunedGame-specific, must be tuned Influences of similar objects can be added Influences of similar objects can be added

togethertogether



OccupancyOccupancy

0,20,2 0,40,4 0,60,6 0,40,4 0,20,2 0,00,0

0,40,4 0,60,6 0,80,8 0,60,6 0,40,4 0,20,2

0,60,6 0,80,8 1,01,0 0,80,8 0,60,6 0,40,4

0,40,4 0,60,6 0,80,8 0,60,6 0,40,4 0,20,2

0,20,2 0,40,4 0,60,6 0,40,4 0,20,2 0,00,0

0,00,0 0,20,2 0,40,4 0,20,2 0,00,0 0,00,0



OccupancyOccupancy PassabilityPassability

0,500,50 0,750,75 0,750,75 1,001,00 0,750,75 0,500,50

0,750,75 1,001,00 1,001,00 1,001,00 0,750,75 0,500,50

0,750,75 1,001,00 0,750,75 0,750,75 0,500,50 0,250,25

0,750,75 1,001,00 0,750,75 0,500,50 0,250,25 0,000,00

0,750,75 1,001,00 0,750,75 0,500,50 0,250,25 0,000,00

0,750,75 1,001,00 0,750,75 0,500,50 0,250,25 0,000,00



Other possible types of Other possible types of influence maps:influence maps:

Vulnerable assetsVulnerable assets ResourcesResources ExplorationExploration Line of fireLine of fire Light levelLight level

0,20,2 0,40,4 0,60,6 0,40,4 0,20,2 0,00,0

0,40,4 0,60,6 0,80,8 0,60,6 0,40,4 0,20,2

0,60,6 0,80,8 1,01,0 0,80,8 0,60,6 0,40,4

0,40,4 0,60,6 0,80,8 0,60,6 0,40,4 0,20,2

0,20,2 0,40,4 0,60,6 0,40,4 0,20,2 0,00,0

0,00,0 0,20,2 0,40,4 0,20,2 0,00,0 0,00,0


Reasoning with Influence MapsReasoning with Influence Maps

Multiple influence maps are stored in a spatial Multiple influence maps are stored in a spatial databasedatabase

These maps can be combined to create a These maps can be combined to create a desirability layer for a certain taskdesirability layer for a certain task

A desirability layer contains the desirability to A desirability layer contains the desirability to move to each cellmove to each cell


Reasoning with Influence Maps (2)Reasoning with Influence Maps (2)

Methods of computing desirability:Methods of computing desirability: Weighted SumWeighted Sum

Simple and efficientSimple and efficient Must be hand-tunedMust be hand-tuned Opposing influences cancel each other outOpposing influences cancel each other out

Product of normalized valuesProduct of normalized values Neural NetworksNeural Networks

Separate weights for each taskSeparate weights for each task Number of inputs is large for non-trivial tasksNumber of inputs is large for non-trivial tasks

(#layers (#layers xx #cells) #cells)


ApplicationsApplications

2 Example applications:2 Example applications: Recognizing Strategic DispositionsRecognizing Strategic Dispositions Environmental Awareness in Game AgentsEnvironmental Awareness in Game Agents


Recognizing Strategic DispositionsRecognizing Strategic Dispositions

(image shamelessly copied from Nyree & Viktor)


Recognizing Strategic DispositionsRecognizing Strategic Dispositions44 66 88 88 88 88 66 44 22 00 -1-1 -2-2 -4-4 -4-4 -4-4 -2-2

44 66 88 88 88 88 44 22 11 00 -2-2 -4-4 -4-4 -2-2

44 66 88 88 88 66 33 11 00 -2-2 -4-4 -4-4 -4-4 -2-2

44 66 88 88 88 66 66 44 11 00 -2-2 -4-4 -4-4 -2-2

22 44 66 88 66 66 44 44 00 -1-1 -2-2 -4-4 -4-4 -4-4 -2-2

11 22 44 66 66 44 22 22 -4-4 -5-5 -3-3 -3-3 -4-4 -4-4 -2-2 -1-1

33 33 33 33 44 22 22 00 -4-4 -5-5 -5-5 -8-8 -8-8 -6-6 -4-4 -2-2

33 33 22 22 22 00 -2-2 -4-4 -8-8 -10-10 -10-10 -8-8 -4-4 -2-2

33 33 33 22 22 11 00 -4-4 -8-8 -10-10 -10-10 -8-8 -8-8 -4-4 -2-2

22 22 22 22 11 11 00 -3-3 -8-8 -10-10 -10-10 -8-8 -8-8 -4-4 -2-2

11 11 11 11 00 00 -2-2 -4-4 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8

00 00 00 00 00 -1-1 -1-1 -2-2 -5-5 -6-6 -6-6 -6-6 -8-8 -8-8 -8-8

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -4-4 -4-4 -4-4 -6-6 -8-8 -8-8 -8-8 -8-8

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2

00 00 00 00 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1


Recognizing Strategic DispositionsRecognizing Strategic Dispositions


44 66 88 88 88 88 66 44 22 00 -1-1 -2-2 -4-4 -4-4 -4-4 -2-2

44 66 88 88 88 88 44 22 11 00 -2-2 -4-4 -4-4 -2-2

44 66 88 88 88 66 33 11 00 -2-2 -4-4 -4-4 -4-4 -2-2

44 66 88 88 88 66 66 44 11 00 -2-2 -4-4 -4-4 -2-2

22 44 66 88 66 66 44 44 00 -1-1 -2-2 -4-4 -4-4 -4-4 -2-2

11 22 44 66 66 44 22 22 -4-4 -5-5 -3-3 -3-3 -4-4 -4-4 -2-2 -1-1

33 33 33 33 44 22 22 00 -4-4 -5-5 -5-5 -8-8 -8-8 -6-6 -4-4 -2-2

33 33 22 22 22 00 -2-2 -4-4 -8-8 -10-10 -10-10 -8-8 -4-4 -2-2

33 33 33 22 22 11 00 -4-4 -8-8 -10-10 -10-10 -8-8 -8-8 -4-4 -2-2

22 22 22 22 11 11 00 -3-3 -8-8 -10-10 -10-10 -8-8 -8-8 -4-4 -2-2

11 11 11 11 00 00 -2-2 -4-4 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8 -8-8

00 00 00 00 00 -1-1 -1-1 -2-2 -5-5 -6-6 -6-6 -6-6 -8-8 -8-8 -8-8

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -4-4 -4-4 -4-4 -6-6 -8-8 -8-8 -8-8 -8-8

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2

00 00 00 00 -1-1 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2 -2-2

00 00 00 00 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1 -1-1

Environmental Awareness in Game Environmental Awareness in Game AgentsAgents

Agents in games need situational awareness to Agents in games need situational awareness to give them a sense of:give them a sense of: What is happening in the environmentWhat is happening in the environment What could happen nextWhat could happen next What options there are for actionWhat options there are for action Possible outcomes of those actionsPossible outcomes of those actions

Environment in the form of a gridEnvironment in the form of a grid Cells track important (threatening) events and Cells track important (threatening) events and

conditions, such as rain, fire, other agentsconditions, such as rain, fire, other agents This can be done using cellular automataThis can be done using cellular automata



Cellular automata can be used in games to Cellular automata can be used in games to model simplified environmental processes, e.g. model simplified environmental processes, e.g. air, fluid flowair, fluid flow

Set of rules represent allowable physics of the Set of rules represent allowable physics of the model, and are used to update cell valuesmodel, and are used to update cell values

New state of a cell is a function of previous state New state of a cell is a function of previous state and state of neighboring cellsand state of neighboring cells



Example automataExample automata



Cellular automata in games:Cellular automata in games: Cells can include data for all kinds of game variables Cells can include data for all kinds of game variables

and are represented by continuous rather than finite and are represented by continuous rather than finite valuesvalues

The rules and variables the cellular automaton The rules and variables the cellular automaton requires depends on what is being modeled in the requires depends on what is being modeled in the game environmentgame environment



Reacting sensibly to the environment requires:Reacting sensibly to the environment requires: Way to sense the environmentWay to sense the environment Way to choose suitable reaction based on sensed Way to choose suitable reaction based on sensed

datadata



Role of influence maps:Role of influence maps: Influence maps divide game map into a grid of cells, Influence maps divide game map into a grid of cells,

with multiple layers of cells containing information with multiple layers of cells containing information about the game worldabout the game world

To include environmental decision-making one of To include environmental decision-making one of these layers should consist of the data from the these layers should consist of the data from the cellular automatoncellular automaton

Also possible to use influence maps for tactical, low-Also possible to use influence maps for tactical, low-level decision-makinglevel decision-making



The reactive agents using the combination of The reactive agents using the combination of influence maps and cellular automata can do influence maps and cellular automata can do this in three steps:this in three steps: Compute “danger” function to determine each cell’s Compute “danger” function to determine each cell’s

utilityutility Use its return value to asses the level of reactionUse its return value to asses the level of reaction Let the agent choose the destination cell based on Let the agent choose the destination cell based on

thisthis



Danger functionDanger function



Level of reactionLevel of reaction



Choosing a destinationChoosing a destination



Agents are likely to have greater goals or Agents are likely to have greater goals or desires they needdesires they need

This can be done through desirability values, This can be done through desirability values, provided through a desirability layerprovided through a desirability layer



Propagation of Propagation of desirability of a goal desirability of a goal in an influence map:in an influence map:



Choosing a destination, also including desirability Choosing a destination, also including desirability of the goal:of the goal:



Algorithm for the agentsAlgorithm for the agents

1.1. Propagate desirability of the goalPropagate desirability of the goal

2.2. Each cycle, calculate the danger value for Each cycle, calculate the danger value for each cell in its local neighborhoodeach cell in its local neighborhood

3.3. Find the optimal cell in the local neighborhood Find the optimal cell in the local neighborhood of the agentof the agent

4.4. Find the optimal cell in the immediate Find the optimal cell in the immediate neighborhoodneighborhood



Step 1: desirability propagationStep 1: desirability propagation



Step 2: Calculate danger valueStep 2: Calculate danger value



Step 3: Find optimal cell in local neighborhoodStep 3: Find optimal cell in local neighborhood



Step 4: Find optimal cell in immediate neighborhood Step 4: Find optimal cell in immediate neighborhood


Issues with Influence MapsIssues with Influence Maps

Time/space complexityTime/space complexity Resolution of the gridResolution of the grid Update frequencyUpdate frequency

TuningTuning Parameter valuesParameter values Agent DesignAgent Design

Overlapping influencesOverlapping influences 3D environments3D environments Suboptimal solutions Suboptimal solutions


ConclusionsConclusions

Influence maps provide an intuitive approach to Influence maps provide an intuitive approach to (runtime) spatial analysis(runtime) spatial analysis

Influence maps are fairly easy and Influence maps are fairly easy and straightforward to implementstraightforward to implement

Influence maps need quite a lot of tuning to Influence maps need quite a lot of tuning to deliver optimal solutionsdeliver optimal solutions

Influence maps are therefore not always Influence maps are therefore not always guaranteed to be usefulguaranteed to be useful

Influence maps can be applied effectively for Influence maps can be applied effectively for strategic decision-making and reactive path strategic decision-making and reactive path findingfinding


ReferencesReferences

Sweetser, Penny. (2004) “Strategic Decision-Making with Neural Sweetser, Penny. (2004) “Strategic Decision-Making with Neural Networks and Influence Maps”. Networks and Influence Maps”. AI Programming Wisdom 2AI Programming Wisdom 2, Charles , Charles River MediaRiver Media

Sweetser, Penny. (2006) “Environmental Awareness in Game Sweetser, Penny. (2006) “Environmental Awareness in Game Agents”. Agents”. AI Programming Wisdom 3AI Programming Wisdom 3, Charles River Media, Charles River Media

Tozour, Paul. (2001) “Influence Mapping”. Tozour, Paul. (2001) “Influence Mapping”. Game Programming Game Programming Gems 2Gems 2, Charles River Media, Charles River Media

Tozour, Paul. (2004) “Using a Spatial Database for Runtime Spatial Tozour, Paul. (2004) “Using a Spatial Database for Runtime Spatial Analysis”. Analysis”. AI Programming Wisdom 2AI Programming Wisdom 2, Charles River Media, Charles River Media

Woodcock, Steven. (2002) “Recognizing Strategic Dispositions: Woodcock, Steven. (2002) “Recognizing Strategic Dispositions: Engaging the Enemy”. Engaging the Enemy”. AI Programming WisdomAI Programming Wisdom, Charles River , Charles River MediaMedia

Questions?Questions?

influence map

Technology

influence maps method

issues conclusions

influence of certain

game agents issues

strategic dispositions

game world

desirability layer

example applications