journal of ieee transactions on affective computing,...

JOURNAL OF IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, VOL. X, NO. Y, JANUARY Z 1

GAMYGDALA: an Emotion Engine for GamesAlexandru Popescu, Joost Broekens and Maarten van Someren

Abstract—In this article we present GAMYGDALA, an emotional appraisal engine that enables game developers to easily addemotions to their Non-Player Characters (NPC). Our approach proposes a solution that is positioned between event coding ofaffect, where individual events have predetermined annotated emotional consequences for NPCs, and a full blown cognitiveappraisal model. Instead, for an NPC that needs emotions the game developer defines goals, and annotates game events with arelation to these goals. Based on this input, GAMYGDALA produces an emotion for that NPC according to the well-known OCCmodel. In this article we provide evidence for the following: GAMYGDALA provides black-box Game-AI independent emotionsupport, is efficient for large numbers of NPCs, and is psychologically grounded.

Index Terms—Computer Games, Affective Computing, Psychological Model

F

1 INTRODUCTION

MANY computer games would be more interest-ing and entertaining if the Non-Player Charac-

ters would express emotions and behave emotionally.A Non-Player Character (NPC) in a digital game isa character of which the behavior is controlled bya computer program, not by a player. Often, theplayer can interact with (e.g., talk to, shoot at, coop-erate with) the NPCs in the game. For single playergames in particular, NPCs play an important role.Game developers and designers face an importantchallenge when developing NPC behavior: when aplayer plays a game for some time, NPC behaviorbecomes predictable and boring. Making NPC be-havior less predictable by introducing ”randomness”can be perceived as illogical or inconsistent with thegame’s plot and internal environment or with theplayer’s assumptions or understanding. Even largeproductions in genres that benefit from balanced NPCbehavior, such as Role-Playing Games like Skyrim [1],struggle with this challenge of balancing betweenconsistency and variation.

Emotional expression and emotional behavior canincrease the variation of NPC behavior [2], [3], [4].However, given a particular state of a game, what isan appropriate emotion to express? In other words,how to calculate plausible emotional states for anNPC? This is the question we address in this article.We propose a computational model (a specialized AIsub-component, or, emotion engine) that can recog-nize which emotion fits an NPC at a particular event,in a way that is compatible with current psycholog-ical knowledge and with the game’s storyline. It is

• Alexandru Popescu and Maarten van Someren are at the InformaticsInstitute of the University of Amsterdam, Science Park 904, 1098 XHAmsterdam.

• Joost Broekens is at Delft University of Technology, Mekelweg 4, 2628CD Delft. Contact: [email protected].

important to be compatible with psychology becausethis helps to generate emotions that are plausible andunderstandable. In this article we propose such anemotion engine. Our engine’s workings are groundedin the psychology of emotion but the engine has thesimplicity needed to perform efficiently in a gameand has been developed to be easy to use by a gamedeveloper.

Our emotion engine is called GAMYGDALA, withan eye-wink to the amygdala, the part of the hu-man brain heavily involved in emotion processing.It supports basic and social emotions, enabling gamedevelopers to create a wide range of emotional states.It is based on the model of Ortony, Clore and Collins(OCC) [5], which is tried and tested in the fieldof computational modeling of emotion (e.g. [6], [2],[7]). The novelty of GAMYGDALA is that it is po-sitioned between event coding of affect, where theemotional behavior is coded explicitly for each NPCfor each individual event, and a full blown cognitiveappraisal model that would include reasoning aboutevents. When using GAMYGDALA, the game devel-oper defines goals for each NPC that needs simulatedemotion and annotates game events with how theyaffect these goals. Based on this input, GAMYGDALAproduces in reaction to a game event an emotion thatis specific to the goals of each (type of) NPC. DifferentNPCs can thus react differently to the same situation,and the same situation can elicit different emotionsin the same NPC depending on its active goals. Emo-tional reactions can depend on previous events thatchanged the goals of an NPC but GAMYGDALA itselfdoes not include memory.

In essence, GAMYGDALA is a ”black-box ap-praisal” engine. GAMYGDALA can be used in away analogous to physics engines. With little effortand little knowledge of emotional appraisal on thegame developer’s part, a psychologically plausiblesimulation of emotion can be included in a game.We argue that this black-box approach is easy to use


and generic, like a physics engine. The game can useits own AI and does not need special cognitive pro-cessing to benefit from GAMYGDALA. Defining goalsand annotating events for use by GAMYGDALA isstraightforward, like adding mass and speed proper-ties is also a straightforward process when using aphysics engine. Of course knowledge of emotion isstill needed if the NPC is to graphcially express it’semotional state.

GAMYGDALA’s output, i.e., an NPC’s emotionalstate, can be used in two different ways. First it canbe used to express emotions, for example the charactermay look stressed or joyful. The emotional state isvisualized by a rendering engine or the voice maychange. Second, the emotional state can be used toalter the behavior of the character, for example it mayrun away when scared or start shooting when angry.Now, the emotional state is fed back to the game AIthat generates NPC behavior.

GAMYGDALA is designed to be• psychologically grounded: it is based on a tried

and tested model of emotion, also with respect toemotion dynamics;

• modular: a modular, black-box, game-AI inde-pendent appraisal engine, is easy to use and doesnot require deep knowledge of appraisal theoryon the part of the developer or designer;

• efficient: able to run in real-time for large num-bers of agents.

2 RELATED WORK

2.1 Emotion TheoriesIn psychology, there are three main perspectives onemotion [8]. The discrete view defines a set of ”basic”emotions [9] where each emotion is characterizedby feelings, expression and behavior, e.g. fear is as-sociated with a negative feeling, a particular facialexpression, and fleeing behavior. The dimensional viewdefines an emotion as a point in an n-dimensionalspace (e.g. [10], [11]). There are several differentpossible dimensions, e.g. fear is negative, arousingand submissive. The componential view defines anemotion as the result of several evaluative cognitivecomponents, called cognitive appraisals, [5], [12]. E.g.fear is the anticipation of something that is incongru-ent with our goals. As we are interested in simulationof emotion in NPCs, we need to know when andhow an emotion results from events in a game andtherefore the cognitive appraisal view fits our aimbest. We will describe cognitive appraisal in moredetail in the following section.

2.2 Cognitive Appraisal TheoriesCognitive appraisal theories assume that an emotionis the result of a cognitive process. A person assessesat all times what is happening in the world around

him or her. These events are evaluated in terms of howrelevant and conducive these are to that person’s andother persons’ ”goals”. The emotion emerges from thisprocess.

Frijda [13] puts at the center of his theory the”concern”, which is the disposition of a person toprefer certain states of the world over others. Thisis related to the concept of ”goal” in other theories.This theory does not really deal with if and howopposing emotions cancel each other (for example joyand distress) or with how multiple emotions combineinto an emotional state. Further, it does not attemptto define categories of emotions nor label them usingcommon language.

Scherer [14], [12] formulates a componential theoryof emotion that defines a step-based appraisal model.He argues that cognitive appraisal does not happenall at once and that there are steps that influence eachother. As a first step, the person evaluates relevance,i.e.: does the event deserve attention? how unexpectedis the event? does it result in pleasure or pain? doesit influence the person’s goals? Second, the potentialimplications of the event are evaluated. Third, the per-son assesses the amount of control over the situation.Can he change the situation? Finally, how does theevent relate to the person’s standards, social normsand values? Emotion emerges continuously from thisappraisal of events.

In the theory proposed by Ortony, Clore and Collins(OCC, [5]) the authors define 22 emotions (later 24)that are associated with a number of evaluative com-ponents including desirability for the person self,desirability for others, likelihood or praiseworthiness.They define the listed emotions as being emotiontypes and reject the idea of basic emotions. Accordingto OCC, appraisal of an event consists of three parts,of which only the first is currently used in GAMYG-DALA:

• the desirability and consequences of the event inlight of the agent’s goals (e.g. candy is bad formaintaining health);

• approval/disapproval according to norms, incase the event is an action (e.g. hitting is bad);

• liking/disliking of the event, in case the eventrelates to an object that appears in an event (e.g.I like candy).

We decided to use OCC as basis for our emotionengine for the following reasons: it is a well-knownand accepted theory of emotions, it is a componentialmodel of emotion that fits the needs of a compu-tational framework, components are generic enoughto allow for a wide set of emotions, it accounts forboth internal emotions and social relationships whichin games are quite important, and most importantlymany computational models have been built on it.

A framework that complements the OCC modelis that of Pleasure, Arousal and Dominance (PAD)[10], [11], [15]. Pleasure defines positiveness versus


negativeness, Arousal defines alertness versus calm-ness and Dominance defines the feeling of control.For example, Happiness is defined as an emotionwith high pleasure, arousal and dominance (PAD)while Fear is an emotion with high arousal but lowpleasure and dominance. Based on Mehrabian’s work[10] GAMYGDALA can derive PAD values from OCCemotions. The game developer can thus also useemotion dimensions.

2.3 Computational Models of EmotionComputational models of emotion are usually basedon psychological emotion theories and attempt tomodel the theory such that it is usable by artificialagents [16], [6], [2], [17], [18], [19], [20]. For a recentreview see [21]. In this section we highlight severalmodels that have had significant influence. A secondreason to review these models is because they provideinsight to deal with difficult issues in computationalmodeling of emotion, such as the relation betweenemotion and mood, the integration of different emo-tions, emotion dynamics, and the influence of emotionon behavior. These insights have been used for thedevelopment of GAMYGDALA.

ACRES [16] is a computational model based onFrijda’s theory of appraisal [13]. It has the same com-ponents as Frijda’s theory, is built around ”concerns”and has the same steps for appraisal. ACRES contin-uously checks if received information is relevant toits concerns. Then, for the relevant parts, it computesthe rest of the components: coping variables, urgency,seriousness, difficulty, etc. It also has a powerfulaction plan generator that combines actions from apredefined set to form a variety of possible plans.

The ”Affective Reasoner” [6] is a computationalmodel of emotion based on the OCC appraisal the-ory [5]. It models all of the twenty-four emotion typesdefined in OCC. The appraisal process is based ongoals, standards and preferences and produces oneor more emotions. The system then decides whichactions the agent takes, based on the resulting emo-tions. The system also maintains for each agent a setof representations of other agents, that is, what doesagent X think agent Y is feeling right now. This isa feature used to diversify the response. Emotionselicited at a certain time are not combined in the”Affective Reasoner”, as the system is focused onthe output actions instead. The action generation stepeliminates possible behavior that is incongruent withthe emotional state before deciding on an action.

”Em” [2] is a computational model based mostlyon the OCC theory. Emotion generation is based ona set of predefined rules, such as: ”when an agenthas a goal failure and the goal has importance X,generate an emotion structure of type distress andwith intensity X”. An emotion has a type and anintensity. Emotions are stored in a hierarchical struc-ture based on the specificity of the emotion: a generic

distress emotion will be higher in the hierarchy whilemore specific ones, such as grief will be a subtypeof distress. This makes it possible to infer highlevelemotions from emotions that are subsumed. At thispoint each subtree will result in an emotion type witha certain intensity based on emotions in the subtree.Emotions are combined using a set of ”Emotion Com-bination Functions”. The intensities decay over timeas specified by the ”Emotion Decay Functions”. A”Behavioral Feature Map” maps the resulting emo-tions to behavioral features and sends them back tothe system. Later [22], Reilly specifies how emotionsare stored, how their intensity is determined andhow emotions are combined. He describes severalinteresting features, some of which we borrow inGAMYGDALA:

• Expectedness: emotions are triggered by changesin the likelihoods of events and goals. For exam-ple, if an event has a likelihood of 0.9 and then ithappens, it will trigger a lower intensity emotionthan a totally unexpected event, even if at thelatest step, they both have likelihood 1;

• Bias against failure: since humans have higherintensity emotions for failure than for success, heintroduces desirability and undesirability as twoseparate variables for each goal;

• Different decay per emotion type: this allows for”hope” and ”fear” to be treated differently, slowerthan others when the event is still uncertain andinstantly when the event takes place;

• Combining emotions: Em uses a logarithmicfunction to combine and normalize emotions.This ensures, as opposed to a sigmoidal normal-izer function, that accumulating emotional inten-sity is linear for lower values and less additive asvalues increase.

EMA is a ”computational model of appraisal andcoping” [7] inspired by the ”Affective Reasoner” [6]and as a consequence by the ”OCC” [5] appraisaltheory. Appraisal is based on several variables thatare components of the appraisal process:

• Relevance: how significant is an event for theagent

• Desirability: does the agent want this to happenor not

• Likelihood: is it a surprise or an expected event• Causal attribution: who is responsible for the

event• Controllability: does the agent have any control

over the event• Changeability: does the agent have any power

to change the outcome (together with Controlla-bility these two variables refer to the ability ofcoping)

EMA also introduces an interesting frame concept thatmakes certain emotions shift out of focus with timeand come back into focus if a similar event happens


or the agent just remembers and thinks about theevent that caused it. A ”mood” structure is used toaggregate multiple emotions by adding up all currentemotions by type and then passing the results througha sigmoid function in order to normalize them. Themood also creates a bias towards certain emotionsrather than others. EMA includes a planning mecha-nism that works closely together with the appraisal toidentify the best coping strategy and modify resultingemotions accordingly.

MAMID [23] is a computational model that signifi-cantly differs from the other models. It is focused notonly on modeling emotions but also on differencesbetween agents (traits). It represents individual dif-ferences of three different types: cognitive, affectiveand personality. Also, it models several influences ofaffect on cognition, which is an interesting basis foremotional influences on game AI.

We reviewed several influential computationalmodels of emotions. The ”Affective Reasoner” ap-pears to be the most complete model built on top ofthe OCC theory. It implements all possible emotions,including social interaction. For this it includes insideeach agent a representation of all the other agentsthat supports the social interaction by allowing agentsto speculate about other agents’ feelings and goals.Other models also assume (or use) particular AI tech-niques, reasoning techniques or cognitive architec-tures. As one of our goals is to make GAMYGDALAindependent of game AI, we do not want to makesuch assumptions. We are aiming for a standalonemodel of appraisal that can function independentlyfrom AI reasoning and expression rendering becausethat is the only way we can support pluggability intodifferent games. As such, GAMYGDALA computesemotions from goals and beliefs and includes thedecay of emotions but does not include the expressionof emotion or predetermined behavioral influences.The mechanism used to define goals and beliefs andto compute an emotional state for an NPC is explainedin Section 3.

2.4 Affective Gaming

Simulations of emotions can be incorporated ingames, which can benefit from that in several ways.Games could recognize emotions in the human playerand adapt to this to increase satisfaction [24], similarto how players appreciate human opponents for theirability to socialize [25]. Here we focus on augmentingNPC behavior with emotions. There are several gamesthat simulate a certain level of human-like emotionalbehavior [26], such as ”Creatures” by Cyberlife Tech-nology [27], [28], ”The Sims” series by Electronic Artsor ”Black & White” by Lionhead Studios [29]. Up tothis date there are no emotion engines that can beused in a variety of games as a simple pluggableblack-box. Recently, it has been argued that such an

approach would be welcome to ”support the devel-opment of socially complex and affectively realisticgames” [4], [30], [8]. In the following paragraphswe critically review earlier computational models ofemotion for NPCs created specifically for games.

Baillie-de Byl [26] creates an example game wherethe agents emulate emotions to be more believable.The model implemented in this game is based on thework by Smith and Ellsworth [31], which proposessix appraisal dimensions: pleasantness, responsibility,certainty, attention, effort, control. For the output thegame uses the basic emotions of Ekman [9]: hap-piness, sadness, anger, fear, disgust, surprise. Theimplementation is based on a neural network withthe value of each appraisal dimension as input andsix emotion flags as output where only one is set totrue. The result is an agent that cannot have complexemotions and cannot have two emotions at the sametime. This might work for simple games but is notenough to build a complex NPC.

Carlisle proposed a simple framework [32] used tocreate emotional agents for games. This frameworkimplements personality using the OCEAN model ofpersonality [33], which has the potential to give eachagent a unique personality that will affect the behav-ior later. As far as mood goes, this framework usesa one-dimensional representation along the pleasure-displeasure axis where positive values represent plea-sure. The appraisal part is built on a simplified versionof the OCC model, which does not elicit the specificemotion types defined by OCC. Instead, it focuseson the types of emotions (related to events, agentsor objects) and defines a single component for eachof them: desirability for events, and attraction foragents and objects. This offers advantages in com-putation times but reduces the amount of availableemotional consequences and behavior. The downsideof this approach is that many specific emotions, suchas Gratitude or Pity, cannot be generated at all. Formost of the games this is enough but possibly morecomplex social emotions are needed for some games.

Research at the University of Bath [34] aims ata system to create procedural side-quests for role-playing games. Even if it is not based on any estab-lished psychological model, it is an important steptowards specific socially capable agents. Their modelis based on behavior trees that use information froma set of priorities (goals) and memory (beliefs/events)to decide on the next quest to be proposed to theplayer. The emotional state is not maintained butinferred from goals and beliefs.

Researchers at the University of California at SantaCruz developed a ”social simulation”, called PromWeek [35], where agents emulate emotions and socialinteraction between teenagers during the last weekbefore the Prom. The main focus of this game arerelationships, including making and breaking friend-ships that may or may not include the player. The


scenarios have predefined goals and the player winsby achieving the goal before the week is over. It isan interesting project but strongly focused on socialnetworking, and the large set of rules needed for thesocial simulation makes this approach difficult to usein a generic way for emotion simulation.

All in all there has been great progress in the fieldbut the game industry still lacks an AI independent,easily pluggable emotion engine. A fundamental rea-son for this seems to be that most attempts integratethe emotional appraisal with the AI reasoning. Thislimits generic use of the emotion engine, becauseif the game AI is not compatible with the AI usedin the emotion engine, the engine can not be used.Further, simpler approaches support only basic emo-tions, while complex ones support personality, moodand social reasoning. We think a solution in themiddle is needed for game development: support fora reasonable set of emotions (including, e.g., socialemotions), but a simple straightforward implementa-tion of appraisal. We aim for a stand-alone emotionalappraisal engine that implements most of the OCCemotions with low computational cost so that it isscalable for a large number of agents in real-time.Our approach, analogous to a physics engine, shouldsupport rapid development of emotions for NPCs.Further, developers should be able to use it withoutknowledge of appraisal theory.

3 GAMYGDALA: AN EMOTION ENGINEFOR GAMES

In this section we introduce GAMYGDALA, its as-sumptions, its mechanisms of appraisal, how it com-putes the emotion, and what input it needs to doso. GAMYGDALA is based on the OCC model, forreasons mentioned earlier. OCC (and all other ap-praisal theories) assume that an agent has some formof goals, concerns, or a set of preferred states, anda mechanism to perceive the world and act in thatworld. A fundamental assumption for being able touse GAMYGDALA in a game is therefore that thegame can be viewed as events (see [19] for a discus-sion), that goals can be defined for the NPCs that needemotion simulation and that the relevance of eventsfor the goals is specified.

Here we argue that GAMYGDALA is indeed easyto use, AI independent and easily pluggable. Wewill show that in most games goals exist or can beeasily defined. In general, each NPC in a game hasa purpose, otherwise it would not be in the game orits behavior would be incomprehensible to the player.Each NPC in a game can be viewed as an autonomousagent: within the frame of its purpose and possibilityfor action, it perceives the game state (input) andchooses alternative behaviors (plans, actions) that areintended to serve its purpose. Its goal is a desiredstate. An example is a soldier that wants to kill the

player. Upon killing the player, the end state hasbeen reached. An NPC can have multiple goals. Forexample a soldier that wants to stop the player fromentering a building can have the goals player distanceis large and player is killed. Note that it does not reallymatter if the game itself uses goals or reasoning withthese goals to generate behavior. The goals are definedfor the emotion engine. The only thing that is thenneeded is to annotate (some) game events with goalcongruence (how good or bad is this event for whichgoal). This is the essence of how GAMYGDALAworks and supports game AI independence.

To use GAMYGDALA, a game designer needs toexplicitly define one or more goals for each NPC(or NPC type, when all NPCs of a type have thesame goals). The game designer can now define which(type of) events are relevant to each goal, and specifythis in GAMYGDALA. The only essential modifica-tion to a game is that events delivered to the gameAI controlling the NPC’s behavior are also deliv-ered to GAMYGDALA. GAMYGDALA emotionallyappraises the event and outputs the most plausibleemotion(s) based on the OCC model. This way ofusing an emotion engine is analogous to how physicsengines are used. This approach is AI independent.Nowhere a link to the AI system is needed for emotionsimulation, even though some AI mechanisms mightinterface more naturally with GAMYGDALA includ-ing BDI-based goal-oriented reasoning and planning.

GAMYGDALA is a richer model for generatingemotions than event coding. In event coding, eachevent needs to include an emotional consequencefor each of the NPCs involved. The advantage ofusing GAMYGDALA is that the game designer getsplausible emotions without having to design themand without having to specify them for each event.Yet GAMYGDALA avoids the computational costs ofa full-blown emotional AI engine that also includesreasoning and planning. AI specific functionality suchas decision making and action selection is not in-cluded in GAMYGDALA. Also, emotion expressionfunctionality is not included, as this is related tospecifics of the game format, style, story or graphics.This means that GAMYGDALA keeps track of thegoals of each character as well as their beliefs rep-resenting events happening in the game. Further, itappraises the current situation, manages the affectivestate and associated dynamics. GAMYGDALA usesa componential representation of emotion based onOCC that it can translate to a dimensional represen-tation based on Pleasure-Arousal-Dominance uponrequest. [10], [11], [15]. A game engine can thus obtainfrom GAMYGDALA both emotion intensities (e.g. ”fear level”) and affect (”degree of dominance”).

3.1 Game WorldThe term ”Game World” refers to the virtual worldwhere the game takes place and in which the player


(gamer) is immersed while playing. A game consistsof a sequence of events where a new event is causedby the user or the game engine. The game worldis usually populated with NPCs. For example, inthe classic game ”Pac-Man” [36], the game worldis a labyrinth, the player is represented by PacMan,a character that has to survive while collecting alldots in the labyrinth. The ghosts, running around thelabyrinth and trying to block pacman from finishingthe level are the NPCs of this world. GAMYGDALAfinds emotions for the NPCs, in this case the ghost.The game developer can use this to add expressionor emotional behavior to the ghost. We now specifyGAMYDALA’s input in detail.

3.2 GoalsGoals in GAMYGDALA can be states of the gamethat an NPC wants achieved (active pursuit and pas-sive goals) or states that an NPC wants maintained(maintenance goals). The system does not differentiatebetween active pursuit and passive goals, as it is ablack-box oblivious to the NPC’s plans and OCC doesnot really differentiate between the two goal typesin terms of different emotion outcome. Each goal isrepresented using the same structure:

• name: identifier• owner: the NPC that has this as a goal;• utility: the value the NPC attributes to this goal

becoming true (∈ [−1, 1] where a negative valuemeans the NPC does not want this to happen).

Owner Goal Name Utility Explanationknight kill monster 1 the knight wants to kill

the monsterknight self die −1 the knight wants to pre-

serve his own life (notdie)

knight princess die −1 the knight wants tokeep the princess alive

knight find gold 0.5 the knight wants to findtreasures

TABLE 1Examples of defined goals

Some examples of goals are shown in Table 1. Thevalue of the utility is proportional to the level ofdesire for that specific goal. Utility values of 1 or −1should be used for major goals, e.g., the outcome ofthe game. Everything in between is less important. Forexample, finding gold is not as important as stayingalive, so failure to gather gold will not generate thesame level of distress for the NPC as having his ownlife threatened. A goal is achieved if it is true in thecurrent event of the game.

3.3 BeliefsBeliefs are annotated events. Each emotionally rele-vant event is annotated with:

• name: identifier used to define and update abelief

• likelihood: the likelihood that this informationis true (∈ [0, 1]) where 0 means the belief isdisconfirmed and 1 means it is confirmed

• causal agent : the agent responsible for this event(can be an NPC, the player or empty if it is an”Act of God” or irrelevant)

• affected goals: a table of goal identifiers with con-gruences between goals and beliefs. Congruenceis a number ∈ [−1, 1] where negative values meanthis belief is blocking the goal and positive valuesmean this belief facilitates the goal.

The event likelihood gives the game developer thepossibility to implement concepts such as rumors(event was not witnessed but the NPC has heard ofit) or credibility (the NPC does not entirely believethe information because it was delivered by anotherunknown NPC). When a new event occurs it producesa corresponding belief for the NPC’s involved.

Belief Name princess attacked bymonster

found magic sword

Likelihood 0.5 1Causal Agent monster knightAffected Goals princess die kill monsterCongruence 0.8 0.5

TABLE 2Examples of incoming beliefs

3.4 EmotionsEmotion simulation is based on the OCC model ofappraisal [5]. Currently sixteen of the twenty-fourOCC emotions are supported. GAMYGDALA definestwo types of emotion: internal emotions and socialemotions (emotions that are directed at another NPC).Each NPC p has one internal emotional state and forevery other NPC it knows a social emotional state(e.g., angry at q1, happy for q2). Before we detail theappraisal mechanism, we first explain belief desirabil-ity and goal likelihood.

3.4.1 DesirabilityThe desirability of a belief b for a goal g depends onthe utility of g and the congruence between b and g.

desirability(b, g, p)← congruence(b, g)∗utility(g) (1)

In the example above, the desirability for the ”princessattacked by monster” event, as far as the knight isconcerned, would be 0.8∗(−1) = −0.8 so for this NPCit is quite an undesirable event. If we also define the”kill princess” goal for the monster but with a utility1 then the system would also calculate the desirabilityfor the monster 0.8∗1 = 0.8 for which the event is verydesirable. This shows how the same belief can have avery different desirability for two different NPCs dueto different goal utilities.


3.4.2 Goal LikelihoodThe likelihood of the goal is maintained internallyand updated every time when a belief that affects thatspecific goal is received. The initial value is Unknown.The goal likelihood is defined by:

likelihood(g)← (congruence(b, g)∗ likelihood(b)+1)/2(2)

Here likelihood(g) is the likelihood that the goal g isrealized after observing belief b, with congruence(b, g)and likelihood(b) as defined above. In the previousexample, the likelihood of the ”princess die” goal willbe (0.8∗0.5+1)/2 = 0.7. Every time the goal likelihoodis updated, the old value is saved into an internalvariable so we are able to also know the change ingoal likelihood for later calculations. This differencewill be referred to as ∆likelihood(g).

3.4.3 Internal EmotionsThis contains all emotions that do not relate to otherNPCs. These are:

• hope: a desirable uncertain goal increases in like-lihood of success or an undesirable uncertain goaldecreases in likelihood of success

• fear: an undesirable uncertain goal increases inlikelihood of success or a desirable uncertain goaldecreases in likelihood of success

• joy: a desirable goal succeeds or an undesirablegoal fails

• distress: an undesirable goal succeeds or a desir-able goal fails

• satisfaction: a desirable goal was expected tosucceed and it actually does succeed

• fearsConfirmed: an undesirable goal was ex-pected to succeed and it actually does succeed

• disappointment: a desirable goal was expectedto succeed and it fails

• relief: an undesirable goal was expected to suc-ceed and it fails

For emotions such as fear we need the notion of uncer-tainty [5]. We consider a goal or belief to be uncertainif it has a likelihood between 0 and 1 exclusive. Forexample, if the likelihood of an undesirable eventis 0.99 then fear will have a high intensity. If thelikelihood becomes 1, distress and fearsConfirmed aregenerated. Additionally, for confirmation-based emo-tions like disappointment and relief we need to definewhen a situation is reversed or not. Currently reversalis defined as an arbitrary threshold for ∆likelihood(g).If |∆likelihood(g)| > 0.5 we assume the situation hasreversed.

Table 3 shows the eliciting conditions for each emo-tion, where L is the likelihood and des is desirability.When GAMYGDALA’s appraisal process is invoked,for each emotion meeting it’s eliciting condition theintensity is computed as:

intensity(e)← |des(b, g, self) ∗∆L(g)| (3)

Emotion Eliciting condition(des(b, g, self) > 0, L(g) < 1,∆L(g) > 0)∨hope(des(b, g, self) < 0, L(g) > 0,∆L(g) < 0)(des(b, g, self) < 0, L(g) < 1,∆L(g) > 0)∨fear(des(b, g, self) > 0, L(g) > 0,∆L(g) < 0)(des(b, g, self) > 0, L(g) = 1)∨joy(des(b, g, self) < 0, L(g) = 0)(des(b, g, self) < 0, L(g) = 1)∨distress(des(b, g, self) > 0, L(g) = 0)

satisfaction des(b, g, self) > 0, L(g) = 1, |∆L(g)| < 0.5fearsConfirmed des(b, g, self) < 0, L(g) = 1, |∆L(g)| < 0.5disappointment des(b, g, self) > 0, L(g) = 0, |∆L(g)| > 0.5relief des(b, g, self) < 0, L(b) = 0, |∆L(g)| > 0.5

TABLE 3Internal emotions appraisal mechanism

3.4.4 Social Emotions

Social emotions have another NPC as target, althoughthe target can also be the NPC self, the human playeror characters controlled by the player. They relate toan event in which one NPC influences the other NPC’sgoals. These emotions are:

• anger: an undesirable event is caused by anotherNPC

• guilt: this NPC causes an undesirable event for aliked NPC

• gratitude: a desirable event is caused by anotherNPC

• gratification: this NPC causes a desirable eventfor a liked NPC

• happyFor: a desirable event happens to a likedNPC

• pity: an undesirable event happens to a likedNPC

• gloating: an undesirable event happens to a dis-liked NPC

• resentment: a desirable event happens to a dis-liked NPC

• like: this is calculated based on all events causedby that specific NPC (it is neutral = 0 if unknown)

Importantly, the terms desirable and undesirable al-ways refer to the desirability calculated from the goalowner’s point of view. Table 4 shows the eliciting

Emotion Eliciting conditionanger des(b, g, self) < 0, NPC(b) 6= selfguilt des(b, g, q 6= self) < 0, NPC(b) =

self, like(self, q) > 0gratitude des(b, g, self) > 0, NPC(b) 6= selfgratif. des(b, g, q 6= self) > 0, NPC(b) =

self, like(self, q) > 0happyFor des(b, g, q 6= self) > 0, like(self, q) > 0pity des(b, g, q 6= self) < 0, like(self, q) > 0gloating des(b, g, q 6= self) < 0, like(self, q) < 0resentm. des(b, g, q 6= self) > 0, like(self, q) < 0

TABLE 4Social emotions appraisal mechanism

conditions for each emotion. When this condition is


met, the intensity is computed as:

intensity(e)← |des(b, g, q 6= self)∗∆L(g)∗like(self, q)|(4)

except for anger and gratitude that follow the intensitycalculation as defined for internal emotions.

NPC(b) is the NPC that caused belief b andlike(p, q) is how much NPC p likes NPC q. Of coursep and q can also be Player Controlled characters. Thelike(p, q) relationship is influenced by the desirabilityof events caused by q as seen through the eyes of p. Anevent caused by q that facilitates a desirable goal for p,increases the value of like(p, q). Likewise, if the goalis blocked then the value of like(p, q) decreases. Assuch, relationships grow over time, and do not need tobe symmetrical. Consequently, social emotions (exceptanger and gratitude) exist only when a relationshipexists, i.e., |like(p, q)| > 0. This relation is configured,by setting like(p, q) to a value, or develops over timeas explained above. The benefit of this is that socialemotions are tractable, the downside is that actionsdetrimental to q do not trigger empathy in p, unlessthere exists a relation between p and q, such that|like(p, q)| > 0.

3.4.5 Pleasure-Arousal-DominanceIn addition to the OCC model, GAMYGDALA com-putes the values for Pleasure, Arousal and Dominance(PAD). These can be derived from the OCC emotionsfollowing [10] and are multiplied by the intensity ofthat emotion. All OCC emotions are combined into anoverall PAD value using the formula:

PAD ← 0.1× log2(∑e

210×PAD(e)×intensity(e)) (5)

where PAD(e) is the PAD score for each specificemotion, intensity(e) is the intensity of each emotionand PAD is the PAD representation of the emotionalstate.

Temperament Pleasure Arousal DominanceExuberant + + +Bored - - -Dependent + + -Disdainful - - +Relaxed + - +Anxious - + -Docile + - -Hostile - + +

TABLE 5The octants of temperament space in PAD [15]

We also implemented four temperamental axes asdefined in [15], see table 6.

Each negative extreme is the opposite of its positivecounterpart, e.g., bored is −(P+A+D) = (−P−A−D).The weight that is assigned to each axis is

√3, so

for example exuberance is defined as√

3 ∗ P+A+D3 =

0.577 ∗ (P + A + D). The implementation in our case

Axes Extremes FormulaExuberance Exuberant vs. Bored (P + A + D)Dependency Dependent vs. Disdainful (P + A−D)Relaxation Relaxed vs. Anxious (P −A + D)Docility Docile vs. Hostile (P −A−D)

TABLE 6The axes of temperament space in PAD [15]

is a property of the NPC that returns the PAD repre-sentation of the NPC’s emotional state on demand. Ofcourse one is free to use any combination of P, A andD from this output should not all three be needed.

3.4.6 Emotion Management and Dynamics

Each NPC is associated with:

• a list of goals the NPC knows about, owned byanyone

• an array of real values representing the defaultemotional state initialized when the NPC is cre-ated

• an array of real values representing the currentemotional state

• an array of real values representing the defaultsocial stance initialized when the NPC is created

• an array of real values representing the currentsocial stance to each known NPC

• (optional) an emotional decay function and asocial stance decay function

The default emotional state, the default social stance andthe decay functions are used by the game designerto create individual differences in emotion dynamicsbetween NPCs. They represent an NPC’s personalityby defining that NPC’s default (social) emotion (e.g.,an angry guy) and how the intensity of all emotionsdecay. The NPC’s emotional state is initialized to thedefault emotional state, it changes due to events thataffect it and decays back towards the default emotionalstate. A similar thing happens to the social emotions:when a new NPC is encountered the social stance isset to the default social stance. Events change the socialstance as explained previously, and it decays to theNPC’s default social stance. Appraisals related to inter-nal emotions are combined into the agent’s currentemotional state using Reilly’s logarithmic function.Appraisals related to social emotions are combinedinto the agent’s current social stance towards particu-lar other NPCs. For details we refer to equation (5)from [22]. This has the desirable properties of notbeing strictly additive, using all emotions (and notonly the strongest), and being at least as intense as themost intense component. GAMYGDALA thus closelyfollows the OCC and PAD models in computing theemotional state with as little additional assumptionsas possible.


4 EXAMPLES AND EVALUATION

In this section we illustrate how GAMYGDALA isused in combination with games and game AI. Thegoal is to illustrate how it can be integrated intogames. The examples are a simple arcade game, arole playing game (RPG), a real-time strategy game(RTS), and a first-person shooter (FPS). We selectedthese four cases to represent different game genreswith different types of NPC involvement [37], [26],[38]. For these cases we also explain how emotionsgenerated by GAMYGDALA could be used to modifygame AI that controls the NPCs. To support ourefficiency and scalability claim, we present a large-scale multi-NPC simulation.

4.1 Arcade game: Pac-Man (1980)

Pac-Man enjoyed a huge success when it was released.The game generated endless sequels, spin-offs andclones and is a representative game for action arcadegames. Pac-Man is the NPC handled by the playerwho has to collect all dots in a labyrinth but stay awayfrom the enemies (ghosts) who try to eat him and thushinder achieving the goal. There are also some powerdots that give Pac-Man the ability to ”eat” the enemiesand send them back to their base location. This caseis a basic example of how straightforward it is to in-strument an arcade game with simple emotions usingGAMYGDALA, analogous to the approach shown in[19].

The goals of the ghost are modeled as ”get eaten”with utility −1 and ”catch pacman” with utility 0.6(we assume that being eaten is of higher importancethan catching Pac-Man). If a state occurs in whichPac-Man gets close to the ghost then this creates acorresponding belief that affects the likelihood thatthe goal ”catch pacman” will be achieved. Now, if Pac-Man gets close to the ghost this will increase thelikelihood that this goal will be satisfied and thiscauses the emotion hope in the ghost. Proximity of apowered-up Pac-Man also increases the likelihood ofthe goal ”get eaten”, resulting in a second emotion fear.

These emotions can now be used for expressionor they may affect the behavior of the ghost. Thisexample illustrates that GAMYGDALA can be incor-porated into Pac-Man by associating goals with theNPC ”ghost” and adding information on which stateshave an effect on achieving the goal or increase thelikelihood. GAMYGDALA will then infer the appro-priate emotions.

4.2 Role-Playing Game

Classical RPGs involve a storyline with quests inwhich the player character can interact with variousfriendly characters but also fight or compete againstenemies. RPGs have the potential for an infinite vari-ety of stories and situations, including NPC character

Fig. 1. Screenshot from the RPG implementation

variation and thus the use of GAMYGDALA as a wayto vary the emotional reactions of NPCs due to gameevents. We implemented a simple RPG consisting of”the village” where people are neutral to our charac-ter, ”the hero”, and could grow to like him or not, and”the forest” where the character fights enemies. Thestory is that the village is surrounded by monstersand the people are too scared to go outside. This caseserves to show how complex emotions can evolveover time, and how social emotions can be modeled.

We give two examples of in-game situations thatproduce complex emotions and we will explain themechanics behind them.

Example 1 - Relief: A villager is relieved when adangerous situation disappears

Relief is a two-step emotion. First an event or beliefcauses Fear (in this case that the village is surrounded)and when the cause of this disappears the new eventor belief will bring Relief with an intensity propor-tional to that of Fear. Listing 1 shows the trace of thesystem.

Listing 1. Output for example ”Relief”adding goal: village destroyed, utility = -0.9,

owner = selfadding belief: village surrounded, likelihood =

0.6

Decaying...Updating...Recalculating...recalculating belief: village surroundedaffected goal: village destroyed, valence = 1desirability: -0.9 <- 1 x -0.9goal likelihood: 0.8 <- (1 x 0.6 + 1) / 2delta goal likelihood: 0.8 <- 0.8emotion intensity: 0.72 <- abs(-0.9 x 0.8)adding FEAR: 0.72

adding belief: village surrounded, likelihood = 0Decaying...Updating...Recalculating...recalculating belief: village surroundedaffected goal: village destroyed, valence = 1desirability: -0.9 <- 1 x -0.9goal likelihood: 0.5 <- (1 x 0 + 1) / 2delta goal likelihood: -0.3 <- 0.5 - 0.8emotion intensity: 0.27 <- abs(-0.9 x -0.3)


adding JOY: 0.27adding RELIEF: 0.27

Current Internal State: Joy: 0.27; Fear: 0.36;Relief: 0.27

Example 2 - Pride (Gratification): The blacksmith wasproud of saving the village by providing it with weapons.This example illustrates events that cause a socialemotion.

Listing 2. Code for example ”Pride”EmoBrain brain = new EmoBrain();brain.Goal("to live", 0.7f);brain.Belief("provides house", 1f, "village");brain.AffectsGoal("provides house", "live", 1f);brain.Update();brain.Goal("village destroyed", -1f, "village");brain.Belief("village is unarmed", 0.7f);brain.AffectsGoal("village is unarmed", "village

destroyed", 1f);brain.Update();brain.Belief("provide weapons", 1f, "self");brain.AffectsGoal("provide weapons", "village

destroyed", -1);brain.Update();

We first ”create” an initial relationship between theblacksmith and the village caused by the blacksmith’sneed to live somewhere and the village providingthat place. This makes the blacksmith like the village.Alternatively, one could simply configure this rela-tionship, or let it grow in a more natural manner overtime. After the first step, Joy is generated because theblacksmith is pleased to live in a house and Gratitudetowards the village because it is the responsible agent.The second step brings the belief that the village isin grave danger because it is lacking the weaponsneeded to defend itself. This step triggers Pity assomething bad is happening to an NPC that theblacksmith likes, i.e., the village. As a consequence,when the blacksmith is able to help the village byproviding the necessary weapons, two new emotionsappear: HappyFor as a liked NPC has something goodhappening to it and Gratification as the blacksmithfeels proud of helping an NPC that he likes. As anadditional remark, if we were to model the villagewith its own emotional brain at the same time, thenafter the second step it would generate Fear and at theend Relief, Joy and Gratitude towards the blacksmith.This example illustrates the complex emotional rela-tionships between NPCs that can be achieved withGAMYGDALA, where an NPC could be a characteror a more abstract agent, such as a village.

Listing 3. Output for example ”Pride”adding goal: to live, utility = 0.7, owner = selfadding belief: provides house, likelihood = 1,

agent = village

Decaying...Updating...Recalculating...recalculating belief: provides houseaffected goal: to live, valence = 1desirability: 0.7 <- 1 x 0.7goal likelihood: 1 <- (1 x 1 + 1) / 2

delta goal likelihood: 1 <- 1emotion intensity: 0.7 <- abs(0.7 x 1)adding JOY: 0.7adding GRATITUDE: 0.7 towards villageadding goal: village destroyed, utility = -1,

owner = villageadding belief: village is unarmed, likelihood =

0.7

Decaying...Updating...Recalculating...recalculating belief: village is unarmedaffected goal: village destroyed, valence = 1desirability: -1 <- 1 x -1goal likelihood: 0.85 <- (1 x 0.7 + 1) / 2delta goal likelihood: 0.85 <- 0.85emotion intensity: 0.85 <- abs(-1 x 0.85)adding PITY: 0.85 towards villageadding belief: provide weapons, likelihood = 1,

agent = self

Decaying...Updating...Recalculating...recalculating belief: provide weaponsaffected goal: village destroyed, valence = -1desirability: 1 <- -1 x -1goal likelihood: 0 <- (-1 x 1 + 1) / 2delta goal likelihood: -0.85 <- 0 - 0.85emotion intensity: 0.85 <- abs(1 x -0.85)adding GRATIFICATION: 0.85 towards selfadding HAPPY-FOR: 0.85 towards village

Current Internal State: Joy: 0.18Relationships:

village -> Gratitude: 0.18; Gratification: 0.85; HappyFor: 0.85; Pity: 0.43

To illustrate how such an emotional instrumenta-tion can be integrated in the gameplay of an RPG,consider the following. Let’s say there is a quest inwhich one of the villagers, Clark, tells a story abouta ring he lost outside of the village (NPCs tend toloose there jewels quite often). Clark has an initial goalhave item with utility 1. When the player accepts thequest, belief accept quest (congruency have item=0.2;likelihood=1) is added, giving Clark Hope and Grati-tude. When the player actually returns the item, beliefreceive item (congruency have item=1; likelihood=1) isadded, giving Clark feelings of Joy and Satisfaction.If Clark has a wife, Alice, and assuming Alice likesClark, Alice will now feel Happy for Clark and Grati-tude towards the player. Concerning emotional influ-ences on the storyline, the interaction with NPCs caneasily include the emotion in, e.g., the dialog trees, aspecial case of behavior trees [39]. Emotions can beused to select the path through the dialog tree oreven to alter certain sentences. For example, whenAlice feels Gratitude towards the player, this can bea condition to insert an extra line such as ”I am sograteful you helped my partner”, or even open upnew storyline options such as ”Now that you helpedus, I trust you, would you do me a favor.....”. Ofcourse this is all possible using other mechanisms,but the interesting thing is that the player can nowchoose in what way he/she will try to gain enough


Fig. 2. Screenshot from the RTS implementation

gratitude from Alice, without having to script all thesepossibilities explicitly.

4.3 Real-time Strategy game

A real-time strategy game usually contains high levelmanagement of units and resources. The player is thecommander of an army and gives orders to his troops.We developed a simple RTS in which two factions(Human and Orc) each have three types of buildingsand two types of units that can be trained using one ofthe buildings. When a building is destroyed, it staysas ruins and becomes non-functional. Ruins can berebuilt using a unit but doing so costs gold and takesa few seconds. This case aims to show how emotionsgenerated by a simple GAMYGDALA instrumenta-tion can be used to impact RTS NPC behavior ina psychologically plausible way. In our example, (a)individual fleeing units might trigger mass fleeing ofall AI-controlled units through the generation of fear,and, (b) high arousal produces errors in planning.

The initial setup is easy. All AI-controlled units(NPC) initially have the following two goals: die withutility −1 and win with utility 1, assuming winningand not dying are of equal importance. No initialbeliefs are generated and no differences in the defaultemotional state. While playing, beliefs are added tothe NPCs in the game, such as the belief woundedaffecting goal (die) with likelihood 1 − health andconduciveness 1. This generates Fear when units arewounded. Furthermore, if NPCs start fleeing, the be-lief fleeing is added to all NPCs affecting the goal winnegatively with a likelihood proportional to the num-ber of fleeing units. When a player-controlled buildingis destroyed, the win goal is affected positively bya belief enemy buildings down that has a probabilityequal to the proportion of destroyed player-controlledbuildings. All of these beliefs generate emotions suchas Fear, Hope, and even Anger towards the player andhis/her units if they would also be specified as NPCs.

NPC emotions in the RTS example still need tobe used in the game. To influence the NPC behaviorbased on the emotions we propose two methods. First,we modified the Finite State Machine used to controlthe unit. Finite State Machines [40], [37] are simplestructures used for controlling behavior and are verypopular in games because of their implementationsimplicity, scalability, and ability to generate complexbehaviors. Our initial implementation has only twostates: Idle and Attack. To show the effects of emotionswe added a third one, Flee. The transition was verysimple: when the amount of Fear would go above athreshold, the unit would enter the Flee state.

Listing 4. Output for the RTS game exampleadding goal: die, utility = -1, owner = selfadding goal: win, utility = 1, owner = selfadding belief: wounded, likelihood = 0.7

Decaying...Updating...Recalculating...recalculating belief: woundedaffected goal: die, valence = 1desirability: -1 <- 1 x -1goal likelihood: 0.85 <- (1 x 0.7 + 1) / 2delta goal likelihood: 0.85 <- 0.85emotion intensity: 0.85 <- abs(-1 x 0.85)adding FEAR: 0.85adding belief: enemy buildings down, likelihood =

0.5

Decaying...Updating...Recalculating...recalculating belief: enemy buildings downaffected goal: win, valence = 1desirability: 1 <- 1 x 1goal likelihood: 0.75 <- (1 x 0.5 + 1) / 2delta goal likelihood: 0.5625 <- 0.75 - 0.1875emotion intensity: 0.5625 <- abs(1 x 0.5625)adding HOPE: 0.5625

Current Internal State: Hope: 0.56; Fear: 0.69

The second method is to modify the A* pathfindingalgorithm [41] so that troops would lose precisionwhen the emotions were running high. For this casewe used the Arousal component of the PAD structure.In essence we added a random factor to the weight ofeach node in the path, so that aroused NPCs will havemore erratic behavior (see next case for more detail).

4.4 First-Person Shooter game

First-Person shooters are games where the player ex-periences action with a first-person camera positionedat the level of the eyes and that involve shootingweapons and direct combat. We developed a simpleFPS in which NPCs are controlled in a similar mannerto the RTS example: they have a very simple decisionmaking system akin to a finite state machine and theyuse A* for pathfinding. We have implemented thesame emulation of fear as in the RTS when their lifedecreases. The goal for this case is to show in a differ-ent game genre than RTS how emotions generated by


a simple GAMYGDALA instrumentation can impactNPC behavior.

We give the NPCs the goals to avoid ”die” and toachieve ”win”. By defining the responsibility of theinflicted wound as belonging to the player the NPChas a social emotion: Anger towards the player. Thisadds to the current Arousal, which can be used tomodify the behavior of the NPC.

Listing 5. Output for the FPS game exampleadding goal: die, utility = -1, owner = selfadding goal: kill player, utility = 1, owner =

selfadding belief: wounded, likelihood = 0.7, agent =

player

Decaying...Updating...Recalculating...recalculating belief: woundedaffected goal: die, valence = 1desirability: -1 <- 1 x -1goal likelihood: 0.85 <- (1 x 0.7 + 1) / 2delta goal likelihood: 0.85 <- 0.85emotion intensity: 0.85 <- abs(-1 x 0.85)adding FEAR: 0.85adding ANGER: 0.85 towards playeradding belief: player shot, likelihood = 0.5

Decaying...Updating...Recalculating...recalculating belief: player shotaffected goal: kill player, valence = 1desirability: 1 <- 1 x 1goal likelihood: 0.75 <- (1 x 0.5 + 1) / 2delta goal likelihood: 0.75 <- 0.75emotion intensity: 0.75 <- abs(1 x 0.75)adding HOPE: 0.75

Current Internal State: Hope: 0.75; Fear: 0.77Relationships:

player -> Anger: 0.77

The A* algorithm [41] is a graph search algorithmthat finds the shortest path in a graph. Because ingames, speed is more important than precision, A*uses a heuristic function that estimates the futuresteps approximatively so that there is no need to cal-culate all possible paths. Arousal taken from GAMYG-DALA influences A* pathfinding precision by addinga random factor that is proportional to the Arousallevel of the NPC to the estimation of the distanceto the goal. This will produce erratic behavior by anaroused NPC, analogous to real-life: tense and ner-vous individuals make irrational and unpredictablechoices. In addition to this, arousal causes the NPCs tohave less shooting precision. As a consequence, whenthey get hurt arousal increases and they automaticallystart missing some shots.

5 COMPUTATION SPEED

As GAMYGDALA is targeted at games involving apotentially large amount of NPCs present, it is impor-tant that it scales well. We performed two stress tests:timing populations of emotional NPCs of various

sizes, and, running a 60 FPS (frames per second)simulation of 5000 NPCs with which the user caninteract by injecting beliefs using mouse clicks.

For the first experiment we run tests varying thefollowing parameters: a population of size n, a num-ber of goals g for each NPC in the population, and bbeliefs for each NPC. GAMYGDALA’s emotion cal-culation was applied to all NPCs after adding allbeliefs. The generated goals and beliefs have randomparameters and the beliefs affect a randomly chosengoal for that specific NPC . We ran five tests, threesimple ones: with one goal and one belief per NPC,two goals and two beliefs per NPC and four goalsand four beliefs per NPC, and two stress tests with 5goals per NPC and 20 or 50 beliefs per NPC . Figure3 shows a comparison between the three cases.

Fig. 3. Timing various populations of NPC s

Our tests ran on a 3 GHz processor in a singlethread. The y-axis in Figure 3 represents the totalamount of time needed to calculate the correspondingemotions as well as do the belief updates. GAMYG-DALA allows simulation of up to 35, 000 NPCs, 2goals and 2 beliefs per NPC , with a total computingtime of less than one second. This computing timeincludes 70, 000 belief additions (2 per NPC ), anda final emotion computation (appraisal). Of course asecond of appraisal is not possible in a real game, butone can easily use standard load spreading techniquesacross frames, so that not all NPCs need updatesevery frame. When the number of goals and beliefsincreases to a more realistic amount (5 goals and 20beliefs per NPC), GAMYGDALA can still compute theappraisal for 5, 000 NPCs (a total of 25, 000 goals and100, 000 beliefs) in just 0.816 seconds.

As a second test we built a sandbox environment(using Microsoft’s R©XNA Framework). The world is arectangular area with 5, 000 NPCs present. The userinfluences them using the mouse: left clicking an areawould send positive beliefs (beliefs that facilitate theirgoals) to the NPCs in that area and right clicking anarea would send negative beliefs (beliefs that blocktheir goals). As an outcome, the NPCs become agi-


tated (speed increases) if the Arousal is high, or calmif it is low and their color would change betweenred and green to show the level of Pleasure. In figure4 a screenshot is shown (red is negative, green ispositive). This simulation is representative of a userinteracting with a local set of NPCs, such as in anRPG. In this case each emotional wave only hits a partof the population of 5, 000 NPCs and the simulationeasily keeps up a frame-rate of about 60FPS.

Fig. 4. Performance test on a population of NPCs.Negative pleasure in red, positive pleasure in green

The experiments show that in these applicationsthe computational costs are quite acceptable and scalewell with the number of NPCs in the game. When anew events appears and it is relevant for an emotionin an NPC then approximately the same computationstake place. The speed of processing a new eventtherefore grows linearly with the number of NPCstimes the number of beliefs affected by that event(see also Figure 3). The example above shows theeffect on speed of increasing the number of agents andbeliefs. This suggests that for many current games ourapproach is fast enough.

6 CONCLUSION

We have introduced GAMYGDALA, an emotion en-gine for games based on OCC [5]. Our approachproposes a solution that is positioned between eventcoding of affect, where each individual event has apredetermined annotated emotional consequence foreach NPC, and a full blown cognitive appraisal model.Instead, for each (type of) NPC that needs simulatedemotion the game developer specifies goals, and an-notates game events with a relation to these goals.Based on this input, GAMYGDALA finds an emo-tion for that NPC according to the well-known OCCmodel. We have shown that GAMYGDALA is psycho-logically grounded, provides black-box Game-AI in-dependent emotion support, and that it is efficient forlarge numbers of NPCs. We have done so by (1) spec-ifying in detail how our computational mechanismsclosely follow the OCC model, (2) presenting four

examples that show that GAMYGDALA is indeedgame AI (and game genre) independent, and (3) pre-senting results of a large scale simulation of thousandsof NPCs showing acceptable performance. BecauseGAMYGDALA is a black-box emotional appraisalmodule, the game developer and game programmerdo not need to be emotional appraisal experts to useit. Of course, a game developer needs to instrumentthe game to interface with GAMYGDALA. However,we have explained that this process is analogous tousing a physics engine: provide the essentials andthe engine will do the hard stuff. We conclude thatthe goal-based event annotation approach used inGAMYGDALA is suitable for the development ofemotional NPCs in games. In this article we have notshown that emotions generated with GAMYGDALAactually add to the level of fun of a game. This isa next step. Another direction for further work is torefine and test the mechanisms for modeling indi-vidual differences, for example using more elaboratedynamics of emotion. Questions such as, how shouldpossibly conflicting emotions be combined, and, whatare appropriate decay functions for a gaming setting,should be investigated. A relatively straightforwardextension of GAMYGDALA is to include norms andobject liking (the two other parts of OCC).

REFERENCES[1] Bethesda, Skyrim. Zenimax, 2011.[2] W. S. N. Reilly, “Believable social and emotional agents,” Ph.D.

dissertation, School of Computer Science, Carnegie MellonUniversity, 1996.

[3] C. Bartneck, “Integrating the occ model of emotions in em-bodied characters,” in Proceedings of the Workshop on VirtualConversational Characters: Applications, Methods, and ResearchChallenges., 2002.

[4] E. Hudlicka, “Affective game engines: motivation and require-ments,” in Proceedings of the 4th International Conference onFoundations of Digital Games. ACM, 2009, pp. 299–306.

[5] A. Ortony, G. L. Clore, and A. Collins, The Cognitive Structureof Emotions. Cambridge University Press, 7 1988.

[6] C. D. Elliott, “The affective reasoner: A process model ofemotions in a multi-agent system,” Ph.D. dissertation, North-Western University, 1992.

[7] J. Gratch and S. Marsella, “A domain-independent frameworkfor modeling emotion,” Journal of Cognitive Systems Research,vol. 5, pp. 269–306, 2004.

[8] E. Hudlicka, “Guidelines for designing computational modelsof emotions,” International Journal of Synthetic Emotions, vol. 2,no. 1, pp. 26–79, 2011.

[9] P. Ekman, “An argument for basic emotions,” Cognition &Emotion, vol. 6, no. 3-4, pp. 169–200, May 1992.

[10] A. Mehrabian, Basic Dimensions for a General PsychologicalTheory. OGH Publishers, 1980.

[11] ——, “Framework for a comprehensive description and mea-surement of emotional states.” Genetic, social, and general psy-chology monographs, vol. 121, no. 3, pp. 339–361, Aug. 1995.

[12] K. R. Scherer, “Appraisal considered as a process of multilevelsequential checking,” in Appraisal processes in emotion: Theory,methods, research, K. R. Scherer, A. Schorr, and T. Johnstone,Eds., 2001, pp. 92–120.

[13] N. H. Frijda, The emotions, ser. Studies in emotion and socialinteraction. Cambridge University Press, Editions de laMaison des sciences de l’homme, 1986.

[14] K. R. Scherer, “Toward a dynamic theory of emotion: Thecomponent process model of affective states,” Geneva Studiesin Emotion and Communication, vol. 1, pp. 1–98, 1987.


[15] A. Mehrabian, “Pleasure-arousal-dominance: A general frame-work for describing and measuring individual differences inTemperament,” Current Psychology, vol. 14, no. 4, pp. 261–292,Dec. 1996.

[16] N. H. Frijda and J. Swagerman, “Can computers feel? theoryand design of an emotional system,” Cognition & Emotion,vol. 1, no. 3, pp. 235–257, 1987.

[17] Y. Guoliang, W. Zhiliang, W. Guojiang, and C. Fengjun, “Af-fective computing model based on emotional psychology,”in Advances in Natural Computation, LNCS Volume 4221, 2006.Springer.

[18] K.-L. Liu, Affective Computing for Computer Games: Bots withEmotions. MSc thesis at the National Taiwan University ofScience and Technology, 2007.

[19] J. Broekens and D. DeGroot, “Scalable and flexibel appraisalmodels for virtual agents,” in Proceedings of the Fifth Game-onInternational Conference, Q. Mehdi and N. Gough, Eds., 2004,pp. 208–215.

[20] M. Ochs, D. Sadek, and C. Pelachaud, “A formal model ofemotions for an empathic rational dialog agent,” AutonomousAgents and Multi-Agent Systems, vol. 24, no. 3, pp. 410–440,May 2012.

[21] S. Marsella, J. Gratch, and P. Petta, “Computational models ofemotion,” in A blueprint for affective computing: A sourcebook andmanual., K. Scherer, T. B”anziger, and E. Roesch, Eds. OxfordUniversity Press, 2010.

[22] W. S. N. Reilly, “Modeling what happens between emotionalantecedents and emotional consequents,” R. Trappl, Ed. Aus-trian Society for Cybernetic Studies, 2006.

[23] E. Hudlicka, “This time with feeling: Integrated model of traitand state effects on cognition and behavior,” Applied ArtificialIntelligence, vol. 16, no. 7-8, pp. 611–641, 2002.

[24] G. N. Yannakakis and J. Hallam, “Real-Time Game Adapta-tion for Optimizing Player Satisfaction,” IEEE Transactions onComputational Intelligence and AI in Games, vol. 1, no. 2, pp.121–133, Jun. 2009.

[25] P. Sweetser, D. Johnson, J. Sweetser, and J. Wiles, “Creatingengaging artificial characters for games,” in Proceedings ofthe second international conference on Entertainment computing.Carnegie Mellon University, 2003, pp. 1–8.

[26] P. Baillie de-Byl, Programming Believable Characters for ComputerGames. Charles River Media, Inc., 2004.

[27] S. Grand, D. Cliff, and A. Malhotra, “Creatures: artificiallife autonomous software agents for home entertainment,” inProceedings of the first international conference on Autonomousagents, W. L. Johnson, Ed.

[28] S. Grand, Creation: Life and How to Make It. Harvard UniversityPress, 2000.

[29] J. Wexler, Artificial Intelligence in Games: A look at the smartsbehind Lionhead Studio’s ”Black and White” and where it can goand will go in the future. University of Rochester, 2002.

[30] E. Hudlicka and J. Broekens, “Foundations for modellingemotions in game characters: Modelling emotion effects oncognition,” in Affective Computing and Intelligent Interaction(ACII). IEEE, 2009.

[31] C. A. Smith and P. C. Ellsworth, “Attitudes and social cogni-tion,” Journal of Personality and Social Psychology, vol. 48, no. 4,pp. 813–838, 1985.

[32] P. Carlisle, “A framework for emotional digital actors,” inGame Programming Gems 8, A. Lake, Ed. Cengage Learning,2010, pp. 312–322.

[33] R. R. McCrae and P. T. Costa, “Toward a new generation ofpersonality theories: Theoretical contexts for the five-factormodel.” in The five-factor model of personality: Theoretical per-spectives, J. S. Wiggins, Ed. Guildford, 1996, pp. 51–87.

[34] J. Grey and J. J. Bryson, “Procedural quests: A focus for agentinteraction in role-playing-games,” in Proceedings of the AISB2011 Symposium: AI & Games, D. Romano and D. Moffat, Eds.SSAISB, 2009, pp. 3–10.

[35] J. McCoy, M. Treanor, B. Samuel, M. Mateas, and N. Wardrip-Fruin, “Prom week: social physics as gameplay,” in Proceedingsof the 6th International Conference on Foundations of DigitalGames. ACM, 2011, pp. 319–321.

[36] T. Watani, Pac-Man. Namco, 1980.[37] B. Schwab, Ai Game Engine Programming. Charles River

Media, Inc., 2004.

[38] D. Mark, Behavioral Mathematics for Game AI. Charles RiverMedia, Inc., 2009.

[39] A. Champandard, “Behavior trees for next-gen game ai,” inGame Developers Conference, Audio Lecture, 2007.

[40] D. Fu and R. Houlette, The Ultimate Guide to FSMs in Games.Charles River Media, 2004, vol. 2, pp. 283–302.

[41] P. Hart, N. Nilsson, and B. Raphael, “A Formal Basis forthe Heuristic Determination of Minimum Cost Paths,” IEEETransactions on Systems Science and Cybernetics, vol. 4, no. 2,pp. 100–107, Feb. 1968.

Alexandru Popescu holds a MSc degreein Artificial Intelligence at the University ofAmsterdam and now works as a softwaredeveloper.

Joost Broekens is assistant professor atDelft University and teaches affective com-puting. He holds a MSc degree in computerscience from Delft University and a PhDin computer science from the University ofLeiden on computational modeling of emo-tion in relation to reinforcement learning. Hisinterests include reinforcement learning, af-fective computing, computational models ofaffective processes, human-technology inter-action, and gaming research.

Maarten van Someren teaches Artificial In-telligence at the University of Amsterdam.His research interests are mostly in MachineLearning, in particular applications, semi-supervised and transfer learning and foun-dations of induction. When the opportunityarises, he combines this with models of hu-man cognition.

journal of ieee transactions on affective computing,...

Documents