Character Artificial Character Artificial IntelligenceIntelligenceCE00875-3CE00875-3

Finite State Automata Finite State Automata

Lecture 3Lecture 3

State MachinesState Machines

Why State Machines?Why State Machines?

What is a State Machine?What is a State Machine?

Finite State AutomataFinite State Automata

Finite State MachinesFinite State Machines

Using FSMsUsing FSMs

Use in Game AIUse in Game AI

Advantages and DisadvantagesAdvantages and Disadvantages

Hierarchical State MachinesHierarchical State Machines

Why State Machines? Why State Machines?

• The control code for reactive animats is basically just a large The control code for reactive animats is basically just a large (possibly nested) (possibly nested) if-elseif-else if-elseif-else structurestructure

• These work okay for simple behaviour, but can become very These work okay for simple behaviour, but can become very difficult to write, read and debug when the animat needs to difficult to write, read and debug when the animat needs to be more complexbe more complex

• A potentially better model for reactive control is some kind A potentially better model for reactive control is some kind of of state machinestate machine – though as we will see, it is not perfectly – though as we will see, it is not perfectly suited to reactive controlsuited to reactive control

• State machines are widely used both in games and other State machines are widely used both in games and other kinds of intelligent software, so it is well worth studying kinds of intelligent software, so it is well worth studying

What is a State Machine? What is a State Machine?

• A state machine basically is A state machine basically is a way of recognising or a way of recognising or generating sequences of generating sequences of events in an orderly fashionevents in an orderly fashion

• Some sequences are allowed Some sequences are allowed and some are not eg and some are not eg telephone telephone behaviour behaviour

• Formally, we say that Formally, we say that transitions between the statestransitions between the states are are triggered by a triggered by a sequence of input symbolssequence of input symbols from a defined from a defined alphabetalphabet. .

• If a sequence of symbols is allowed by the If a sequence of symbols is allowed by the mappingmapping (state (state transition diagram) then the sequence is recognised, else it transition diagram) then the sequence is recognised, else it is rejectedis rejected

• Alternatively, we might arrange to drive the system to Alternatively, we might arrange to drive the system to generate generate output symbolsoutput symbols from the states and/or on the from the states and/or on the transitionstransitions

Finite State Automata Finite State Automata • A FSA is a directed graph of nodes connected by arcs called transitionsA FSA is a directed graph of nodes connected by arcs called transitions

• FSA can be used to represent classes of state-sequences by the FSA can be used to represent classes of state-sequences by the connectivity of the transitions. Imagine a sequence of symbols from the connectivity of the transitions. Imagine a sequence of symbols from the alphabet alphabet {a,b,c}{a,b,c} make the transitions make the transitions

• We always begin with the start state (1 – bold circle) and allow the We always begin with the start state (1 – bold circle) and allow the sequence to control the transitions between states. If we end up in final sequence to control the transitions between states. If we end up in final acceptance state (4 - double circle), the sequence is accepted as a acceptance state (4 - double circle), the sequence is accepted as a member of the class, (outputs a Boolean T); else, it is rejectedmember of the class, (outputs a Boolean T); else, it is rejected

• This FSA would accept the sequence This FSA would accept the sequence abcccaabccca, but not the sequence , but not the sequence abcbbaabcbba. So the state machine can enforce a . So the state machine can enforce a grammar – a certain syntax grammar – a certain syntax for expressions in a formal languagefor expressions in a formal language

c

ab

c

a

1

2

3

4 Final acceptance state

Start state

Finite State Automata Finite State Automata • Formally, a FSA may be represented by as the quadrupleFormally, a FSA may be represented by as the quadruple

FSA = {FSA = {ΣΣ, Q, , Q, δδ ,F} ,F}

• where where – ΣΣ is an input alphabet of symbols eg {a,b,c} is an input alphabet of symbols eg {a,b,c} – Q is a set of states eg {1,2,3,4}Q is a set of states eg {1,2,3,4}– δδ is a transition function defined over is a transition function defined over ΣΣ and Q eg state diagram and Q eg state diagram– F is a subset of Q called accepting (terminal) states eg {4}F is a subset of Q called accepting (terminal) states eg {4}

• One state, qOne state, q0 0 , has a special meaning as the start state, has a special meaning as the start state

• δδ is a function mapping the set of states Q and the input is a function mapping the set of states Q and the input alphabet alphabet ΣΣ onto the set of states: onto the set of states: – ff(Q,(Q,ΣΣ) → Q ) → Q – This describes the connections in the transition network, as This describes the connections in the transition network, as

diagrammed by the graph in our example diagrammed by the graph in our example

• δδ can also be represented as a table can also be represented as a table

FSA and Finite State FSA and Finite State Machines Machines • Similarly, the following FSA could classify sequences of Similarly, the following FSA could classify sequences of actions as belonging to a class such as “Rocket jump”actions as belonging to a class such as “Rocket jump”

Fire RocketJumpLook (down)

1

2

3

4 Rocket jump

Start state

• Finite State MachinesFinite State Machines are different. While a FSA may only are different. While a FSA may only produce one output if a sequence of transitions is produce one output if a sequence of transitions is recognised, a FSM may produce output symbols at each recognised, a FSM may produce output symbols at each transition, or each statetransition, or each state

• Therefore, a FSM can be used to Therefore, a FSM can be used to transduce transduce a sequence of a sequence of input symbols into a sequence of output symbols within a input symbols into a sequence of output symbols within a certain class certain class

• One type of FSM called a One type of FSM called a Mealy machineMealy machine outputs symbols at outputs symbols at each transition, and are thus suitable for sequential control each transition, and are thus suitable for sequential control of actionsof actions

Mealy Machine Mealy Machine • So a Mealy machine may be formalised as for FSA, but So a Mealy machine may be formalised as for FSA, but

requires an alphabet of output symbols, and a function requires an alphabet of output symbols, and a function to generate them from transitionsto generate them from transitions

FSM = FSM = {{ΣΣ, Q, Z, , Q, Z, δδ , ,λλ}} where where ΣΣ is an input alphabet of symbols eg {a,b,c} is an input alphabet of symbols eg {a,b,c}

Q is a set of states eg {1,2,3,4}Q is a set of states eg {1,2,3,4}Z is an output alphabet of symbols eg {x,y,z}Z is an output alphabet of symbols eg {x,y,z}δδ is a transition function defined over is a transition function defined over ΣΣ and Q eg state and Q eg state

diagram diagram λλ is a function to determine the output symbol Z based on is a function to determine the output symbol Z based on

transitionstransitions

• There is a special start state but no special terminal There is a special start state but no special terminal statestate

• The function The function λλ computes an output symbol Z for each computes an output symbol Z for each transition given the states Q and the input symbols Z ie transition given the states Q and the input symbols Z ie ff(Q,(Q,ΣΣ) → Z) → Z

Mealy Machine Mealy Machine

b/zb/x

c/z

a/x

1

2

b/y

c/x

c/y

4 5

3

inputsymbol

outputsymbol

Moore Machine Moore Machine • A Moore machine has the same definition as a Mealy A Moore machine has the same definition as a Mealy

machine, but the function machine, but the function λλ is simpler but less powerful is simpler but less powerful

FSM = {FSM = {ΣΣ, Q, Z, , Q, Z, δδ , ,λλ}} where where

ΣΣ is an input alphabet of symbols eg {a,b,c} is an input alphabet of symbols eg {a,b,c}

Q is a set of states eg {1,2,3,4}Q is a set of states eg {1,2,3,4}

Z is an output alphabet of symbols eg {x,y,z}Z is an output alphabet of symbols eg {x,y,z}

δδ is a transition function defined over is a transition function defined over ΣΣ and Q eg state and Q eg state diagram diagram

λλ is a function associating states Q with outputs Z is a function associating states Q with outputs Z

• The function The function λλ maps states Q directly onto output maps states Q directly onto output symbols Z symbols Z ff(Q) → Z(Q) → Z

Moore Machine Moore Machine

bb

c

a

x

b

c

a

y y

z

inputsymbol

outputsymbol

• State: description of the internal State: description of the internal settings of the machine, e.g. how settings of the machine, e.g. how much money has been depositied much money has been depositied and not spentand not spent

• Finite states: 0, 25, 50, 75, 100, Finite states: 0, 25, 50, 75, 100,

• Rules: determine how inputs can Rules: determine how inputs can change statechange state

Example: Vending MachineExample: Vending Machine

Example: Vending MachineExample: Vending Machine

0

25 50

75

100

000

001 010

011

100

retret

ret

ret

sodasoda

100

25

25

25

25Inputs

25 = 00 100 = 01soda = 10 ret = 11

Example: Vending Example: Vending MachineMachine

state input new stateS2 S1 S0 I0 I1 S2 S1 S0

0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 1 0 0 1 0 0 1 0 0 0 0 1 0 0

0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 0 0 1 1 0 1 0 1 1 1 0 0 0 1 1 0 0

state input new stateS2 S1 S0 I0 I1 S2 S1 S0

0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 1 0 0 1 1 1 0 0 0 0 1 0 0 1 0 0 0 1

0 0 0 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 0 1 1 0 0 0 0 1 1 1 1 0 0 0 1 0 0 1 1 0 0 0

Example: Vending Example: Vending MachineMachine

state input new stateS2 S1 S0 I0 I1 S2 S1 S0

0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 1 0 0 1 0 0 1 0 0 0 X 1 0 0

0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 0 0 1 1 0 1 0 1 1

state input new stateS2 S1 S0 I0 I1 S2 S1 S0

0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 1 0 0 1 1 1 0 0 0 0 1 0 0 1 0 0 0 1

X X X 1 1 0 0 0

Using FSMs Using FSMs • There are other types of FSMs too, such as Augmented There are other types of FSMs too, such as Augmented

Transition Networks (ATNs). These are commonly used Transition Networks (ATNs). These are commonly used where efficient parsing of complex patterns of symbols where efficient parsing of complex patterns of symbols must be recognised, such as in the must be recognised, such as in the natural language natural language processingprocessing

• Patterns of symbols such as Patterns of symbols such as abcccaabccca which belong to a which belong to a class acceptable to a FSM are called class acceptable to a FSM are called regular expressions regular expressions oror regular languages regular languages. The FSM enforces the . The FSM enforces the grammargrammar of a of a given expression/languagegiven expression/language

• Such state machines may be used to Such state machines may be used to parseparse regular regular expressions making up an expressions making up an artificial languageartificial language, such as Java , such as Java or C.or C.

• They may also be used to parse sentences in a natural They may also be used to parse sentences in a natural language, such as French. This could produce a language, such as French. This could produce a syntactical parse treesyntactical parse tree, a breakdown of the linguistic , a breakdown of the linguistic components such as noun phrase, verb phrase, articles, components such as noun phrase, verb phrase, articles, etc.etc.

Use in Game AI Use in Game AI • Although State Machines are very widely used in actual game Although State Machines are very widely used in actual game

code, they are rarely as pure as the FSA or FSM of theory.code, they are rarely as pure as the FSA or FSM of theory.

• You can use FSAs to recognise actions You can use FSAs to recognise actions of anotherof anotherplayer eg observing enemy basic player eg observing enemy basic enemy acts, and recognising enemy acts, and recognising sequences as belonging to a strategy, sequences as belonging to a strategy, or other high level description for or other high level description for decision-makingdecision-making

• You can use FSMs to organise You can use FSMs to organise complex sequences of complex sequences of behaviour, by making the behaviour, by making the transitions be triggered by transitions be triggered by perceptual events in the world, perceptual events in the world, while arranging for states to while arranging for states to order procedures calling action order procedures calling action functionsfunctions

Advantages/Disadvantages of Advantages/Disadvantages of FSMsFSMs

• ProsPros– Very simple, easy to visualiseVery simple, easy to visualise– Efficient implementation is possibleEfficient implementation is possible– Theory is well-founded and well understood, algorithms Theory is well-founded and well understood, algorithms

off-the-shelfoff-the-shelf– Proven popular in computer games and for control of Proven popular in computer games and for control of

sequences, parsing of artificial and natural languagessequences, parsing of artificial and natural languages

• ConsCons– Large state machines can become complex to debugLarge state machines can become complex to debug– Computationally limited to certain class of problems Computationally limited to certain class of problems

(regular exp)(regular exp)– The design of a FSM is fixed at design-time, not dynamicThe design of a FSM is fixed at design-time, not dynamic– Possible procedural control problems due to sequential Possible procedural control problems due to sequential

chaos, loss of data, and flow of controlchaos, loss of data, and flow of control

Designing a RCS using State Designing a RCS using State MachineMachine• Begin by listing your Begin by listing your

– 1) inputs (perceptual tests – eg 1) inputs (perceptual tests – eg if(sight < 5.0)...) if(sight < 5.0)...)

– 2) states (eg avoid, wander, 2) states (eg avoid, wander, attack, flee, etc.) attack, flee, etc.)

– 3) outputs (eg hurl grenade, 3) outputs (eg hurl grenade, drop gun, eat food)drop gun, eat food)

– Draw a state transition Draw a state transition diagram diagram representing your machinerepresenting your machine

• Actions are represented by Actions are represented by procedures within the states procedures within the states that call basic output functions that call basic output functions in the game eg run(), turn(), in the game eg run(), turn(), watch() etc.watch() etc.

• Question: Where should state transitions be Question: Where should state transitions be processed? Maybe things change while we’re processed? Maybe things change while we’re inside a state procedure, so we should test for inside a state procedure, so we should test for updates in there as well as ‘outside’? But this updates in there as well as ‘outside’? But this leads to redundancy and loss of performanceleads to redundancy and loss of performance

• So, procedures within a state should be So, procedures within a state should be atomicatomic – – ie once a state has been entered, nothing may ie once a state has been entered, nothing may interrupt the flow of control for that action interrupt the flow of control for that action

Designing A RCS using State Designing A RCS using State MachineMachine

• Ideally, a reactive system has inputs Ideally, a reactive system has inputs processed in parallel. Can’t do this with processed in parallel. Can’t do this with a state machine, whose inputs are a state machine, whose inputs are sequences of symbols - still rely on a sequences of symbols - still rely on a sequence of tests to trigger transitionssequence of tests to trigger transitions

• There is the question of priorities – There is the question of priorities – wewe need to ensure that high priority tests need to ensure that high priority tests get done first, so that their transitions get done first, so that their transitions can get in firstcan get in first

• There is also the problem of loss of There is also the problem of loss of data, since events are happening in data, since events are happening in real time; might get two important real time; might get two important events yet we can only process one at events yet we can only process one at a timea time

• Possible solution – keep tests inside a Possible solution – keep tests inside a simplified if-elseif-else structure. Have simplified if-elseif-else structure. Have it generate symbols which are then it generate symbols which are then enqueued onto a short queueenqueued onto a short queue

• A queue is a First In, First Out (FIF0) A queue is a First In, First Out (FIF0) data structure. At each perceive-act data structure. At each perceive-act cycle, the next item in the queue is cycle, the next item in the queue is taken off for processing the transitions taken off for processing the transitions to arrive at a state. to arrive at a state.

Designing A RCS using State Designing A RCS using State MachineMachine

If (test1) enqueue(o)else if (test2) enqueue(a)else if (test3) enqueue(f)else enqueue (w)

dequeue (next)process state transitions

execute stateprocedure

2. next symbol makes transition in state machine

3. state procedures generate actions

w f o a 1. perceptual test enqueues an input symbol

symbols move in this direction

buffer length of 4

Hierarchical State Machines Hierarchical State Machines • But how can we deal with the problem that complex state But how can we deal with the problem that complex state

machines can be just as difficult to write, debug and modify as machines can be just as difficult to write, debug and modify as other designsother designs

• One approach is the concept of hierarchical state machinesOne approach is the concept of hierarchical state machines

• Consider the machine at more than one level, with the levels Consider the machine at more than one level, with the levels themselves related by transitionsthemselves related by transitions

• Divide the complexity into parts and have a related set of states Divide the complexity into parts and have a related set of states - or even a whole nested state machine – regarded from the top - or even a whole nested state machine – regarded from the top level as a single statelevel as a single statea b

zoom in

zoom out

a b

Abstract Refined

Emotive States; Emotive BotEmotive States; Emotive Bot

• Emotions and ComputersEmotions and Computers

• A Model of EmotionsA Model of Emotions

• Emotional Expression for CommunicationEmotional Expression for Communication

• Affective Model for BotsAffective Model for Bots

• Sensations, Emotions, Moods and FeelingsSensations, Emotions, Moods and Feelings

• Portraying Emotions in GamesPortraying Emotions in Games

• An Extensible Affective SystemAn Extensible Affective System

• Processing Sensations & EmotionsProcessing Sensations & Emotions

• The Affective System Put TogetherThe Affective System Put Together

Emotions and ComputersEmotions and Computers• The idea of computers with emotions may seem strange The idea of computers with emotions may seem strange

because because i) our culture separates emotions and logic and i) our culture separates emotions and logic and ii) associates computers with logic ii) associates computers with logic

• Yet the truth is that models of emotions have been built from Yet the truth is that models of emotions have been built from the earliest days of AI and “affective computing” is by now a the earliest days of AI and “affective computing” is by now a well-established subfieldwell-established subfield

• There could be many advantages in making machines cable of There could be many advantages in making machines cable of having emotions, or at least recognising them in humans and having emotions, or at least recognising them in humans and reacting properlyreacting properly

• In nouvelle game AI, we are interested in making emotional In nouvelle game AI, we are interested in making emotional animats for the same reason as story-tellers – because this will animats for the same reason as story-tellers – because this will enhance the realism of non-player characters, make them more enhance the realism of non-player characters, make them more entertaining and help to more fully engage players with them entertaining and help to more fully engage players with them

• A lot of interesting philosophical questions flow from the idea of A lot of interesting philosophical questions flow from the idea of emotional computersemotional computers

A Model of EmotionsA Model of Emotions• Plutchick (1980) described eight primary emotions organised Plutchick (1980) described eight primary emotions organised

into pairs of states. In this model, it is impossible to experience into pairs of states. In this model, it is impossible to experience the two states of a pair at once. The overall emotional status is the two states of a pair at once. The overall emotional status is registered by registered by degrees of arousaldegrees of arousal of one of each pair of one of each pair

• Each state has a Each state has a degree of intensitydegree of intensity and a and a target target object or state of affairsobject or state of affairs to which it refers to which it refers

• Each state has Each state has behavioural implicationsbehavioural implications which which betray its “purpose” in helping the organism betray its “purpose” in helping the organism survivesurvive

causes rejection of the target object or state of affairs/produces aversive signal for learning

FEAR | ANGER

ACCEPTANCE | DISGUST

JOY | SORROW

ANTICIPATION | SUPRISE

recruits energy and directs destructive force toward an obstacle

flags unexpected event/ directs attention to percepts/ raises heart rate

releases hormones, tensions muscles and prepares for fight or flight

recruits energy and produces reinforcing signal for learning

withholds energy, forces withdrawal to safety and produces aversive signal for learning

tolerates target object or state of affairs/produces reinforcing signal for learningallows preparation for expected event/ tests model of the world

A Model of EmotionsA Model of Emotions• The Plutchik model allows states to combine to produce The Plutchik model allows states to combine to produce

compound “flavours” of emotion. Eg sorrow + suprise = compound “flavours” of emotion. Eg sorrow + suprise = disappointment;disappointment; fear + acceptance = submission, etc. fear + acceptance = submission, etc.

• The model presupposes perceptual, learning and cognitive The model presupposes perceptual, learning and cognitive systems, but says little about themsystems, but says little about them

• Since emotions are closely tied to sensors, effectors and Since emotions are closely tied to sensors, effectors and bodily interaction in the world, it follows that embodied AI bodily interaction in the world, it follows that embodied AI might be useful at providing these as a more complete might be useful at providing these as a more complete system than traditional AI models, which had to work on system than traditional AI models, which had to work on disembodied computersdisembodied computers

• Generally speaking, Generally speaking, affective modelsaffective models of this kind try to of this kind try to mimic the different basic functions of emotions: regulation of mimic the different basic functions of emotions: regulation of behaviour, learning and communicationbehaviour, learning and communication

• Plutchik’s model is an example of how emotional states Plutchik’s model is an example of how emotional states might signal might signal survival-related classes of stimulisurvival-related classes of stimuli and how this and how this could could regulate actionregulate action

Emotional Expression for Emotional Expression for CommunicationCommunication• Emotion has an important role in dealing with othersEmotion has an important role in dealing with others

• Being able to Being able to read the emotional stateread the emotional state of another human would have of another human would have powerful survival implications, and hence be selected for in human powerful survival implications, and hence be selected for in human evolutionevolution

• Being able to Being able to influence the emotional state of another humaninfluence the emotional state of another human would also would also be very useful be very useful

• Humans signal their emotional states by Humans signal their emotional states by universal expressionsuniversal expressions, and we , and we can read these in other humanscan read these in other humans

• It could be useful to have computersIt could be useful to have computersor robots do this:or robots do this:

- - vision systems could read human facial vision systems could read human facial expressions and classify them as emotions expressions and classify them as emotions

- graphical or robot faces could have their features set into universal expressions to signal emotions

Emotional Expression for Emotional Expression for CommunicationCommunication• Emotions can also be expressed in (and read from) Emotions can also be expressed in (and read from)

gesturesgestures

• Over a longer time frame, emotions can be expressed in Over a longer time frame, emotions can be expressed in (and read from) (and read from) behavioursbehaviours

• Emotions may also be communicated via Emotions may also be communicated via languagelanguage:: - choice of words may be influenced by emotional state- choice of words may be influenced by emotional state - pitch, tone, speed and loudness of voice carries - pitch, tone, speed and loudness of voice carries

affective affective information information

• This means that an emotional system needs to This means that an emotional system needs to interact interact with other behavioural control mechanismswith other behavioural control mechanisms and influence and influence them according to the current emotional state of the them according to the current emotional state of the systemsystem

• So research on these areas often overlaps with the study So research on these areas often overlaps with the study of realistic body animation, including facial expression, of realistic body animation, including facial expression, mannerisms, natural language generation and speech mannerisms, natural language generation and speech synthesissynthesis

SensationsSensations• Emotions in embodied creatures begin with sensations. In Emotions in embodied creatures begin with sensations. In

this model, a this model, a sensationsensation will be defined as an immediate will be defined as an immediate reaction to a creature’s current situationreaction to a creature’s current situation

• They might include They might include surprise, anticipation, disgust, attraction, surprise, anticipation, disgust, attraction, pain pain andand pleasure pleasure

• Sensations are visceral “gut reactions” – they happen almost Sensations are visceral “gut reactions” – they happen almost instantaneously, and do not last very long. They may arise instantaneously, and do not last very long. They may arise from two different sources:from two different sources:

• Sensations from Perceptions Sensations from Perceptions – The body detects a – The body detects a certain class of stimuli from the environment, which causes certain class of stimuli from the environment, which causes an immediate short term reaction eg sensation of surprise if an immediate short term reaction eg sensation of surprise if a player or NPC appears suddenlya player or NPC appears suddenly

• Sensations from Cognition Sensations from Cognition – A mental state may also – A mental state may also cause a sensation. For example, the thought of blood might cause a sensation. For example, the thought of blood might cause a sensation of disgustcause a sensation of disgust

EmotionsEmotions• Higher-levelHigher-level and and longer-lastinglonger-lasting affective trends are called affective trends are called

emotionsemotions in this model. An emotion is a lasting characteristic in this model. An emotion is a lasting characteristic of a persons affective stateof a persons affective state

• Emotions might include Emotions might include fear, anger, joyfear, anger, joy and and sorrowsorrow

• Emotions change slowly compared to sensations. Emotions Emotions change slowly compared to sensations. Emotions could be triggered by a single sensation, or by multiple could be triggered by a single sensation, or by multiple sensationssensations

• Champardard calls basic dimensions (such as fear, anger etc) Champardard calls basic dimensions (such as fear, anger etc) primary emotionsprimary emotions. Each may have various degrees of . Each may have various degrees of intensity, which Champandard calls intensity, which Champandard calls secondary emotionssecondary emotions (?) (?)

• Eg the primary emotion of fear might have a low intensity Eg the primary emotion of fear might have a low intensity (apprehension) or a high intensity (terror)(apprehension) or a high intensity (terror)

• As with Plutchik’s model, some might be As with Plutchik’s model, some might be mutually mutually antagonisticantagonistic eg eg

joy and sorrow. These are called joy and sorrow. These are called complementarycomplementary emotions. emotions.

MoodsMoods

• Now a creature’s Now a creature’s moodmood will will be defined as its complete be defined as its complete set of emotions at a set of emotions at a particular timeparticular time

• This will be represented as This will be represented as a set of primary emotions, a set of primary emotions, each with its own intensityeach with its own intensity

• A positive mood might A positive mood might include include acceptance, acceptance, satisfactionsatisfaction (moderate (moderate amount of joy) and amount of joy) and aweawe (high level of surprise)(high level of surprise)

• Moods thus include Moods thus include complex emotionscomplex emotions, which , which are subsets of primary are subsets of primary emotionsemotions

FeelingsFeelings• Affective states can refer to an object or event (even one Affective states can refer to an object or event (even one

remembered in the past or anticipated in the future)remembered in the past or anticipated in the future)

• In Champadard’s model, these are called In Champadard’s model, these are called feelingsfeelings

• Feelings are persistent associations between an emotion and Feelings are persistent associations between an emotion and a a (class of) object(class of) object

• Examples are cruelty (to dogs), love (of a brother), hatred (of pirates)

• Feelings do not depend on the current state of the creature, but may refer to past or present (or even hypothetical) states of affairs

Portraying Emotions in Portraying Emotions in GamesGames

• How could an observer see the effects of emotional states in How could an observer see the effects of emotional states in an animat which had them?an animat which had them?

• Many games (certainly FEAR/Quake2) have little, if any Many games (certainly FEAR/Quake2) have little, if any provision for controlling facial expressions well enough to provision for controlling facial expressions well enough to signal emotions that way (The action is usually too fast to see signal emotions that way (The action is usually too fast to see the faces close up anyway) the faces close up anyway)

• However, recall that emotions can also be communicated by However, recall that emotions can also be communicated by actions and languageactions and language

• Existing Existing actions can be triggered or changedactions can be triggered or changed by affective by affective state. Eg, terror might cause the animat to wail, or state. Eg, terror might cause the animat to wail, or depression might cause it walk more slowly than usualdepression might cause it walk more slowly than usual

• A A lack of actionlack of action might also signify something: eg failing to might also signify something: eg failing to respond to attack might be a sign of submission. Higher level respond to attack might be a sign of submission. Higher level decisions, even at the strategic level, might also reveal decisions, even at the strategic level, might also reveal emotions in some circumstancesemotions in some circumstances

Portraying Emotions in Portraying Emotions in GamesGames• Existing gesturesExisting gestures such as Flipoff() may be used to signify such as Flipoff() may be used to signify (anger)(anger)

• Champandard describes a general-purpose interface to the Champandard describes a general-purpose interface to the enumerated action functions of the Quake 2 engineenumerated action functions of the Quake 2 engine

voidvoid Gesture ( Gesture ( constconst GestureType g ); GestureType g );

• Senses indirectly play a role in the portrayal of emotions, Senses indirectly play a role in the portrayal of emotions, because they are closely linked to action – especially in because they are closely linked to action – especially in reactive modelsreactive models

• A good example is anticipation, which has the effect in real A good example is anticipation, which has the effect in real creatures of heightening sensory awareness. The thresholds at creatures of heightening sensory awareness. The thresholds at which stimuli signifying possible danger are detected are lower which stimuli signifying possible danger are detected are lower than usual in this statethan usual in this state

• Chat communications can also be used to make language Chat communications can also be used to make language signify emotions eg Say() or SayTeam() functionssignify emotions eg Say() or SayTeam() functions

voidvoid Say ( Say ( constconst string& text, const GroupType g = string& text, const GroupType g = everybody );everybody );

Portraying Emotions in Portraying Emotions in GamesGames

• Some affective behaviour scenarios:Some affective behaviour scenarios: - Joy: dance or jump- Joy: dance or jump - Fear: faster but less accurate turns than usual; run faster- Fear: faster but less accurate turns than usual; run faster - Extreme fear: stop thinking and freeze for a while- Extreme fear: stop thinking and freeze for a while - Disgust: turn away from reference stimulus; say “ugh”- Disgust: turn away from reference stimulus; say “ugh” - Anticipation: swing gun wildly at any sound; slower walking - Anticipation: swing gun wildly at any sound; slower walking

than than usual usual

• Extra touches for very emotive charactersExtra touches for very emotive characters - Insult enemies during battle; perform taunting gestures- Insult enemies during battle; perform taunting gestures - Run away screaming when the enemy is powerful- Run away screaming when the enemy is powerful - Keep a history of conflict, and target enemy based on - Keep a history of conflict, and target enemy based on

grudges grudges

An Extensible Affective An Extensible Affective SystemSystem

• Champandard designs an affective system for FEAR animats Champandard designs an affective system for FEAR animats thatthat

- is based on the model described so far (and the Plutchick - is based on the model described so far (and the Plutchick psychoevolutionary model) psychoevolutionary model)

- uses a FEAR finite state automata/machines module- uses a FEAR finite state automata/machines module - works with other behavioural control architectures (since - works with other behavioural control architectures (since

emotions emotions are only part of a functioning cognitive system) and can be are only part of a functioning cognitive system) and can be enhancedenhanced

• Simplified: only models four primary emotions: fear/anger, Simplified: only models four primary emotions: fear/anger, surprise/anticipation. These are complementary states, so surprise/anticipation. These are complementary states, so one should observe no more than one should observe no more than twotwo at any time at any time

• Simplified: each emotion is represented as a binary value: Simplified: each emotion is represented as a binary value: either fully activated or inactive (no secondary emotional either fully activated or inactive (no secondary emotional levels here)levels here)

• Simplified: emotional states merely raise or lower the Simplified: emotional states merely raise or lower the effectiveness of existing reactive behaviourseffectiveness of existing reactive behaviours

FEAR’s FSA/FSM ModuleFEAR’s FSA/FSM Module• The FEAR platform allows both Finite State The FEAR platform allows both Finite State

Automata (recognisers, acceptors, parsers) and Automata (recognisers, acceptors, parsers) and Finite State Machines (transducers) to be easily and Finite State Machines (transducers) to be easily and efficiently implementedefficiently implemented

• FSA – inputs are a sequence of symbols (eg from an FSA – inputs are a sequence of symbols (eg from an vector of characters) which can trigger the state vector of characters) which can trigger the state transitions. The output can be any kind of data, but transitions. The output can be any kind of data, but usually sets a categorical variable (for classification) usually sets a categorical variable (for classification) or a Boolean variable (for acceptance):or a Boolean variable (for acceptance):

boolbool Accept ( Accept ( const const vector<Symbol>& sequence );vector<Symbol>& sequence ); SymbolSymbol Recognise( Recognise( const const vector<Symbol>& sequence );vector<Symbol>& sequence );

• FSM – one output for each input, so you need to FSM – one output for each input, so you need to either step a single symbol in and out, or else run an either step a single symbol in and out, or else run an entire set of symbols and get a matching output set:entire set of symbols and get a matching output set: SymbolSymbol Tick ( Tick ( const const Symbol in );Symbol in );

voidvoid Simulate( Simulate(const const vector<Symbol>& in vector<Symbol>& out );vector<Symbol>& in vector<Symbol>& out );

Processing SensationsProcessing Sensations• Sensations will be simply perceived classes of objects or Sensations will be simply perceived classes of objects or

events in the environment, such as events in the environment, such as unexpected soundsunexpected sounds, , blood splatsblood splats or or explosions explosions (cognitive sensations are not (cognitive sensations are not addressed in this prototype)addressed in this prototype)

• A simple FSA, used as a recogniser, will accept A simple FSA, used as a recogniser, will accept perceptionsperceptions or or sequences of perceptionssequences of perceptions and move into a state representing and move into a state representing the current sensation, at each Think cycle the current sensation, at each Think cycle

Anger/anticipation

Anger/surprise

Fear/surprise

Fear/anticipation

expecteddamage

unexpectedsound

explosion

item picked upfa

fs

as

aa

Processing EmotionsProcessing Emotions• The The mood of the systemmood of the system may be in any of four states, each may be in any of four states, each

consisting of a pair of (non-complementary) emotions. This consisting of a pair of (non-complementary) emotions. This enforces the constraint that complementary emotions may not enforces the constraint that complementary emotions may not be expressed at the same timebe expressed at the same time

• The moods are The moods are fully interconnectedfully interconnected, so sensations from the , so sensations from the sensation FSA may move the system from any mood to any other sensation FSA may move the system from any mood to any other

fear/surprise fear/anticipation

anger/surprise anger/anticipation

faaa

fs

fa

fsas

as

aa

faas

aa

fs

• Each state will output an associated change in the parameters used

to control accuracy and power of actions; and the delay and precision of perceptual functions...

Emotional OutputEmotional Output

• AccuracyAccuracy is the randomness or error of an action: 1.0 means the is the randomness or error of an action: 1.0 means the commanded action is performed perfectly; 0.0 means the action is commanded action is performed perfectly; 0.0 means the action is randomisedrandomised

• PowerPower is used to scale the magnitude of the action: 1.0 means the is used to scale the magnitude of the action: 1.0 means the actions have their maximal value 0.0 means the action is frozenactions have their maximal value 0.0 means the action is frozen

• DelayDelay alters perceptions. A delay of n means n seconds pass before alters perceptions. A delay of n means n seconds pass before a stimuli is recognised by perceptual functionsa stimuli is recognised by perceptual functions

• PrecisionPrecision alters the location of objects in space by perceptual alters the location of objects in space by perceptual functions. A value of 1 means the position is exact; while 0 means functions. A value of 1 means the position is exact; while 0 means that a large random number is added to the distance and anglethat a large random number is added to the distance and angle

Precision = 0.7 Power = Precision = 0.7 Power = 0.20.2

Delay = 0.0 Accuracy = Delay = 0.0 Accuracy = 0.90.9

Precision = 0.9 Power = Precision = 0.9 Power = 0.40.4

Delay = 1.0 Accuracy = Delay = 1.0 Accuracy = 0.20.2

Precision = 0.0 Power = Precision = 0.0 Power = 1.01.0

Delay = 0.3 Accuracy = Delay = 0.3 Accuracy = 0.80.8

Precision = 0.1 Power = Precision = 0.1 Power = 0.90.9

Delay = 0.8 Accuracy = Delay = 0.8 Accuracy = 0.30.3

surprise anticipation

fear

anger

Affective System Put Affective System Put TogetherTogether

affecting allactions andperceptions

sensation

pattern of parameters

emotion FSM sensation FSAcycle ofperceptions

• At each Think cycle a (possibly new) sensation may arise At each Think cycle a (possibly new) sensation may arise in the sensation FSA if it reaches an end statein the sensation FSA if it reaches an end state

• New sensations are passed asynchronously to the New sensations are passed asynchronously to the emotion FSM, which may change state to a new moodemotion FSM, which may change state to a new mood

• Each mood outputs its characteristic parameters which in Each mood outputs its characteristic parameters which in turn affect both actions and perceptions turn affect both actions and perceptions

Evaluation of Affective Evaluation of Affective SystemSystem• This design is (partly) implemented in the Moody animatThis design is (partly) implemented in the Moody animat

• The alterations to action and perceptual parameters do make a The alterations to action and perceptual parameters do make a difference to the animat’s behaviour, but it is difference to the animat’s behaviour, but it is subtlesubtle and you need and you need observe carefullyobserve carefully to see the effects to see the effects

• It would be possible to make the emotional system output more It would be possible to make the emotional system output more dramatic behaviourdramatic behaviour - easier to see, but possibly inappropriate at - easier to see, but possibly inappropriate at timestimes

• Errors in perception and changes in speed of response are realistic Errors in perception and changes in speed of response are realistic emotional effects and do lead to game events which wouldn’t emotional effects and do lead to game events which wouldn’t happen otherwisehappen otherwise

• System is simple to adjust (though not as simple as Ch suggests)System is simple to adjust (though not as simple as Ch suggests)

• Transitions are all boolean – states do not rise and fall in degree, so Transitions are all boolean – states do not rise and fall in degree, so no “secondary emotions” are possible. Also the emotions do not no “secondary emotions” are possible. Also the emotions do not changechange

• Emotional states have no “hysteresis”, meaning they can change as Emotional states have no “hysteresis”, meaning they can change as fast as sensations – not true to the modelfast as sensations – not true to the model

SummarySummary• The recognition and production of emotions in computers is possible, The recognition and production of emotions in computers is possible,

and there are many reasons to wish to do soand there are many reasons to wish to do so

• In nature, emotions regular behaviour, assist learning and allow In nature, emotions regular behaviour, assist learning and allow communication to other organismscommunication to other organisms

• Plutchick’s model of emotion specifies 8 primary states organised Plutchick’s model of emotion specifies 8 primary states organised into mutually antagonistic pairs. Each state has behavioural, learning into mutually antagonistic pairs. Each state has behavioural, learning and communication implicationsand communication implications

• Embodied animats in games are a good test bed for emotions, Embodied animats in games are a good test bed for emotions, because they can play out these implications in the game world, and because they can play out these implications in the game world, and should be more entertaining as a resultshould be more entertaining as a result

• But the portrayal of emotions in games can be problematic, because But the portrayal of emotions in games can be problematic, because the opportunities for facial expression, gestures etc. could be limitedthe opportunities for facial expression, gestures etc. could be limited

• Champandard defines sensations, emotions, moods and feelings Champandard defines sensations, emotions, moods and feelings technically, as part of an application of Plutchick’s model to the technically, as part of an application of Plutchick’s model to the design of a FEAR animat (Moody)design of a FEAR animat (Moody)

• This model uses a FSA to transform perceptions into sensationsThis model uses a FSA to transform perceptions into sensations

• The sensations drive a FSM model of emotional states, each of which The sensations drive a FSM model of emotional states, each of which modifies the efficiency and speed of perceptions and actions in the modifies the efficiency and speed of perceptions and actions in the Moody animat Moody animat