finite state machine for games spring 2005 ref: chenney, cs679 lectures ai game programming wisdom 2
TRANSCRIPT
Spring 2005 2
Outline
AI and GameIntroduction/examplesDesign Intuition State-based
ImplementationExtending Stack-based Fuzzy-state machine
Spring 2005 3
What is AI?
AI is the control of every non-human entity in a game
The other cars in a car game The opponents and monsters in a shooter Your units, your enemy’s units and your enemy in a
RTS game
But, typically does not refer to passive things that just react to the player and never initiate action
That’s physics or game logic For example, the blocks in Tetris are not AI, nor is a
flag blowing in the wind
Spring 2005 4
AI in the Game Loop
AI is updated as part of the game loop, after user input, and before renderingThere are issues here: Which AI goes first? Does the AI run on every frame? (LOD
problem) Is the AI synchronized?
Spring 2005 5
AI and Animation
AI determines what to do and the animation does it
AI drives animation, deciding what action the animation system should be animating
Scenario 1: The AI issues orders like “move from A to B”, and it’s up to the animation system to do the rest
Scenario 2: The AI controls everything down to the animation clip to play
Which scenario is best depends on the nature of the AI system and the nature of the animation system
Is the animation system based on move trees (motion capture), or physics, or something else
Does the AI look after collision avoidance? Does it do detailed planning?
Spring 2005 6
AI Module
AI Update Step
The sensing phase determines the state of the world
May be very simple - state changes all come by message
Or complex - figure out what is visible, where your team is, etc
The thinking phase decides what to do given the world
The core of AI
The acting phase tells the animation what to do
Generally not interesting
Game Engine
Sensing
Thinking
Acting
Spring 2005 7
AI by Polling
The AI gets called at a fixed rateSenses: It looks to see what has changed in the world. For instance: Queries what it can see Checks to see if its animation has finished
running
And then acts on itWhy is this generally inefficient? Different characters might require different
polling rate
Spring 2005 8
Event Driven AI
Event driven AI does everything in response to events in the world
Events sent by message (basically, a function gets called when a message arrives, just like a user interface)
Example messages: A certain amount of time has passed, so update
yourself You have heard a sound Someone has entered your field of view
Note that messages can completely replace sensing, but typically do not. Why not?
Real system are a mix - something changes, so you do some sensing
Spring 2005 9
AI Techniques in Games
Basic problem: Given the state of the world, what should I do?A wide range of solutions in games: Finite state machines, Decision trees, Rule
based systems, Neural networks, Fuzzy logic
A wider range of solutions in the academic world: Complex planning systems, logic
programming, genetic algorithms, Bayes-nets Typically, too slow for games
Spring 2005 10
Goals of Game AI
Several goals: Goal driven - the AI decides what it should do, and
then figures out how to do it Reactive - the AI responds immediately to changes in
the world Knowledge intensive - the AI knows a lot about the
world and how it behaves, and embodies knowledge in its own behavior
Characteristic - Embodies a believable, consistent character
Fast and easy development Low CPU and memory usage
These conflict in almost every way
Spring 2005 11
Two Measures of Complexity
Complexity of Execution How fast does it run as more knowledge is
added? How much memory is required as more
knowledge is added? Determines the run-time cost of the AI
Complexity of Specification How hard is it to write the code? As more “knowledge” is added, how much
more code needs to be added? Determines the development cost, and risk
Spring 2005 12
Expressiveness
What behaviors can easily be defined, or defined at all?Propositional logic:
Statements about specific objects in the world – no variables
Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage
Go to room8 if you are in room7 through door14Predicate Logic:
Allows general statement – using variables All rooms have doors All splash damage weapons can be used around corners All rocket launchers do splash damage Go to a room connected to the current room
Spring 2005 13
Finite State Machines (FSMs)A set of states that the agent can be inConnected by transitions that are triggered by a change in the worldNormally represented as a directed graph, with the edges labeled with the transition eventUbiquitous in computer game AIYou might have seen them, a long time ago, in formal language theory (or compilers)
What type of languages can be represented by finite state machines?
How might this impact a character’s AI? How does it impact the size of the machine?
Spring 2005 14
Formal Definitions (N. Philips)
"An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states".The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events.Finite State Automaton: the machine with no output
Spring 2005 17
FSM and Game
Game character behavior can be modeled (in most cases) as a sequence of different “mental state”, where change is driven by the actions of player/other characters, …Natural choice for defining AI in games
Spring 2005 19
Ex: predator vs. prey
Prey (laalaa)
Idle(stand,wave,…)
Flee(run)
Sees predator
No more threat
capturedDead
Spring 2005 20
Predator (Raptor)
Idle(stand)
Hungry(wander)
Pursuit(run)
Tidle > 5
Prey in sight
Tpursuit > 10
Dining
Prey captured
Tdining>5
Spring 2005 21
Idling LaaLaa
Stand
Wave
Wander(set random target)
50%
30%20%
Target arrived
Twave>2
RTstand>4
This page illustrates:hierarchical state,Non-deterministic state transition
This page illustrates:hierarchical state,Non-deterministic state transition
Spring 2005 23
FSM + Cinematography
The current state (and state transition) may have an important effect on how camera should be manipulated Idling: focus on character Movement: zoom out to show spatial info State-transition: camera animation …
More complicated camera behavior may be coded in another FSM
Spring 2005 24
Camera Selection Strategy
It is important to choose a camera that has the front of the character
Z
X
Spring 2005 26
Other Concerns
• Adjust zoom (fovy) so that the character occupies a good (fixed!?) portion of the screen
x
w
r
percentd
r
near
nearnearnear
w
fovyw
xpercent
d
near
r
x
percentdr
percentx
2/
2tan
:
2/
2/
percentd
r
near
nearnearnear
w
fovyw
xpercent
d
near
r
x
percentdr
percentx
2/
2tan
:
2/
2/
near
d
Spring 2005 27
FOVY based on Postures
Once in running mode, zoom outIn stand/wave modes, zoom inSmooth transition between two fovy settings (by indirectly controlling the percent value)
Spring 2005 29
Quake2 ExamplesQuake2 uses 9 different states:
standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching.
Incomplete design
Intuitive thinking: model the events and state changes
Intuitive thinking: model the events and state changes
Spring 2005 32
Intuitive Design
Say, a simple teletube baby has three states: idle, run, and waveScenario: When an idle laalaa sees a butterfly, it wa
ves to it. When the butterfly flies away, it returns to idle
When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle
Spring 2005 33
Laalaa
idle
flee
wave
mouse
~mouse
~butterfly
butterfly
How to make sure the design complete? I.e., all states and transitions are considered
Is there any systematic way of developing an FSM?
Spring 2005 34
Quake Bot Example (first-cut)
Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase e
nemy When die, re-spawn (new a bot from start)
Events: see enemy, hear enemy, dieStates: wander, attack, chase, spawn
Spring 2005 35
Remark
With 3 events, potentially there should be 23 states: (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)
Some doesn’t make sense E.g., ESD = 111
Name and create a state for the ones that we want to consider Wander (ESD=000) Chase (ESD=010) Attack (ESD=1x0), x for dont-care Die (ESD=xx1)
Spring 2005 36
FSM (first-cut)
Events: E: see an enemy S: hear a sound D: die
States: E: enemy in sight S: sound audible D: dead
D
~E
ED
~S
S
D
E S D
Problem: Can’t go directly from attack to chase. Why not?
Spawnxx1
Wander000
Attack1x0
Chase010
start
Spring 2005 37
FSM (first-cut)
Events: E: see an enemy S: hear a sound D: die
States: E: enemy in sight S: sound audible D: dead
D
~E
ED
~S
S
D
E S DSpawn
xx1
Wander000
Attack100
Chase010
start E
Attack+S110
~E
~S
S
D
Extra state needs to be defined
Extra state needs to be defined
Spring 2005 38
Quake Bot Example (refined)
Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase ene
my When die, respawn
Extensions: When health is low and see an enemy, retreat When see power-ups during wandering, collect them
[hierarchical FSM]
Spring 2005 39
Example FSM with Retreat
SpawnD
(-E,-S,-L)
Wander-E,-D,-S,-L
E
-SAttack-EE,-D,-S,-L
E
Chase-E,-D,S,-L
S
D
S
D
D
Retreat-EE,-D,-S,L
L
-E
Retreat-S-E,-D,S,L
Wander-L-E,-D,-S,L
Retreat-ESE,-D,S,L
Attack-ESE,-D,S,-L
E
E-E
-L
S
-S
L
-E E
L-L
-L
-L
L
D
• States:– E: enemy in
sight– S: sound
audible– D: dead– L: Low health
A lot more states got added
Spring 2005 40
Hierarchical FSMsWhat if there is no simple action for a state?Expand a state into its own FSM, which explains what to do if in that stateSome events move you around the same level in the hierarchy, some move you up a levelWhen entering a state, have to choose a state for its child in the hierarchy
Set a default, and always go to that Or, random choice Depends on the nature of the behavior
Spring 2005 41
Hierarchical FSM Example
Note: This is not a complete FSM All links between top level states still
exist Need more states for wander
StartTurn Right
Go-throughDoor
Pick-upPowerup
Wander Attack
Chase
Spawn
~E
E~S
SD
~E
Spring 2005 42
Non-Deterministic HierarchicalFSM (Markov Model)
Adds variety to actionsHave multiple transitions for the same eventLabel each with a probability that it will be takenRandomly choose a transition at run-timeMarkov Model: New state only depends on the previous state
Attack
Start
Approach
Aim & Jump &Shoot
Aim & Slide Left& Shoot
Aim & Slide Right
& Shoot .3.3
.4
.3.3
.4
Spring 2005 45
Efficient ImplementationCompile into an array of state-name, eventstate-namei+1 := array[state-namei, event]Switch on state-name to call execution logicMarkov: Have array of possible transitions for every (state-name,event) pair, and choose one at randomHierarchical
Create array for every FSM Have stack of states
Classify events according to stack Update state which is sensitive to current eve
nt
event
state
Spring 2005 46
FSM Advantages
Very fast – one array accessExpressive enough for simple behaviors or characters that are intended to be “dumb”Can be compiled into compact data structure
Dynamic memory: current state Static memory: state diagram – array implementation
Can create tools so non-programmer can build behaviorNon-deterministic FSM can make behavior unpredictable
Spring 2005 47
FSM Disadvantages
Number of states can grow very fast Exponentially with number of events: s=2e
Number of arcs can grow even faster: a=s2
Propositional representation Difficult to put in “pick up the better powerup”,
“attack the closest enemy” Expensive to count: Wait until the third time I see
enemy, then attack Need extra events: First time seen, second time seen,
and extra states to take care of counting
Spring 2005 54
Stack-based FSM
History stack: Remember previous state; create characters with a memory …
Pushdown automaton(PDA)
Spring 2005 55
Goal-based vs. State-based
There is also a slight derivative to the state-based engine, but it used in more complicated games like flight simulators and games like MechWarrior. They use goal -based engines - each entity within the game is assigned a certain goal, be it 'protect base', 'attack bridge', 'fly in circles'. As the game world changes, so do the goals of the various entities.
Spring 2005 56
Processing Models
Polling FSM update
frequency Easy to implement
and debug Inefficiency (Little
Red example)
Event-driven Publish-subscribe mess
aging system Game engine sends eve
nt messages to individual FSMs
An FSM subscribes only to the events that have the potential to change the current state
Higher efficiency, non-trivial implementation
Spring 2005 57
Interfacing with Game Engine
Query, act on the game worldMove the function calls (to the game engine) as DLL; reduce the amount of recompilation required
Spring 2005 58
Efficiency and Optimization
In AI, FSM is the most efficient technology availableYet, there is always room for improvement Level of Detail: depending on the
condition (e.g., distance with player), use different FSM, or different update frequency
Spring 2005 59
References
Web references: www.gamasutra.com/features/19970601/build_brains_in
to_games.htm csr.uvic.ca/~mmania/machines/intro.htm www.erlang/se/documentation/doc-4.7.3/doc/design_pri
nciples/fsm.html www.microconsultants.com/tips/fsm/fsmartcl.htm http://www.angelfire.com/dragon/letstry/tutorials/dfa/
Game Programming Gems Sections 3.0 & 3.1
It’s very very detailed, but also some cute programming