emotional agents in game playing€¦ · web viewemotional agents in game playing. msc dissertation...

Emotional Agents in Game PlayingMSc Dissertation submitted

By Mehul Shukla

For MSc in Games Programming

University of Hull 16th September 2011

Project Supervisor: Dr Darryl N. Davis

September 16, 2011

EMOTIONAL AGENTS IN GAME PLAYING

This report is dedicated to my parents Mr Y K Shukla and Mrs Veena Shukla. Without their love and support this degree would not have been possible.

1

September 16, 2011


AcknowledgementsFirst and foremost, I would like to thank my girlfriend Karishma for all her love and support and for being the constant nudge that brought me here and kept me going till the end.

I would also like to thank my supervisor, Dr Darryl N. Davis, for all his help and invaluable knowledge provided during the dissertation. I am also thankful for the opportunity to pursue this topic and for all his patience which I didn’t refrain from testing.

Special thanks goes to Roger Pujol Serra for guiding me and helping me through a lot of my programming limitations throughout the year. His persistent and harsh guidance was much appreciated as it provided me with the base to complete this dissertation.

Last but not least a big thanks to all my friends Aritra, Nuno, Mario and Roger for sharing and drowning all the pains and hardships of the degree, in many a round of pints at the pub.

2

September 16, 2011


Table of ContentsTable of Contents......................................................................................3

Table of Figures........................................................................................6

Abstract.....................................................................................................7

Chapter 1 Introduction............................................................................8

Chapter 2 Project Overview..................................................................112.1 Initial Brief...........................................................................................................11

2.2 Context..................................................................................................................11

2.3 Aims and objectives.............................................................................................12

2.4 Project Plan..........................................................................................................12

Chapter 3 Project Background.............................................................133.1 Early Theories of Emotion..................................................................................13

3. 2 Affect and Affective Computation....................................................................14

3.3 Requirement for an Emotion Architecture.......................................................15

3.4 Emotion Redefined: The Emotion Model..........................................................16

Chapter 4 Theoretical Design................................................................184.1 Game Story and Agents......................................................................................18

4.2 Emotion Actor Scenario......................................................................................19

4.3 Agent Architecture..............................................................................................22

Chapter 5 AI Test Bed Design...............................................................265.1 Testbed Theory....................................................................................................26

5.2 Testbed Design Constraints................................................................................27

5.2.1 Choice of a planning or reactive system:.........................................................................27

5.2.2 Complexity of the world..................................................................................................27

5.2.3 Sensing and Effecting......................................................................................................27

5.2.4 Partial Satisfaction of Goal States...................................................................................28

5.2.5 Multiple Agent Coordination and Communication.........................................................28

5.2.6 A Clean Interface.............................................................................................................28

5.2.7 Time Module...................................................................................................................29

3

September 16, 2011


5.2.8 Experimentation and Testing...........................................................................................29

Chapter 6 Software Design....................................................................306.1 Software tools and SFML...................................................................................30

6.2 C++ Architecture and Implementation.............................................................31

6.2.1 Core Game Engine...........................................................................................................32

6.2.2 Map..................................................................................................................................32

6.2.3 Class LevelLoader...........................................................................................................33

6.2.4 Class ImageManager.......................................................................................................34

6.2.5 Class Agent......................................................................................................................34

6.2.6 Class Pathfinder...............................................................................................................34

6.2.7 Class Engine....................................................................................................................36

6.3 Agent Hierarchy..................................................................................................38

6.3.1 Class Agent......................................................................................................................39

6.3.2 Class Agent0....................................................................................................................40

6.3.3 Class Civilian, Robber and Police...................................................................................40

Chapter 7 Gameplay and Design..........................................................427.1 Detailed Storyline................................................................................................42

7.2 Control Factors and rules for the Testbed........................................................43

7.3 Finite State Machine Design...............................................................................45

7.4 Agent Behaviour and Design..............................................................................45

7.4.1 Base Agent.......................................................................................................................45

7.4.2 Agent0..............................................................................................................................46

7.4.3 Police...............................................................................................................................46

7.4.4 Civilian............................................................................................................................49

7.4.5 Robber..............................................................................................................................52

Chapter 8 Emotion Architecture..........................................................568.1 Civilian emotion model.......................................................................................58

8.2 Robber emotion model........................................................................................58

8.3 Police Emotion model..........................................................................................59

Chapter 9 Conclusion.............................................................................609.1 Critical Evaluation..............................................................................................60

7.2 Future Work........................................................................................................60

7.3 Conclusions...........................................................................................................61

4

September 16, 2011


References...............................................................................................62

Bibliography................................................Error! Bookmark not defined.

Appendix A Original Specification......................................................65

Appendix B Updated Specification......................................................66

Appendix C Requirements Analysis Table.........................................67

Appendix D Character Icons................................................................68

Appendix E Project Plan......................................................................69

Appendix F Considerations..................................................................72Legal Considerations.................................................................................................72

Ethical Considerations..............................................................................................72

Social Considerations................................................................................................72

Professional Considerations......................................................................................72

5

September 16, 2011


Table of FiguresFigure 1 Example Game world.................................................................................................18Figure 2. Fear............................................................................................................................19Figure 3. Sadness......................................................................................................................20Figure 4. Disgust.......................................................................................................................20Figure 5. Anger.........................................................................................................................21Figure 6. Happiness..................................................................................................................22Figure 7. Reflex agent architecture...........................................................................................23Figure 8. Reactive agent architecture.......................................................................................23Figure 9. Reactive and deliberate level architecture.................................................................23Figure 10. Reactive behaviour architecture..............................................................................24Figure 11. Goal based reactive behaviour................................................................................24Figure 12. Emotion based reactive behaviour..........................................................................24Figure 13. Emotion and goal based reactive agent architecture...............................................25Figure 14. Example Benchmarks..............................................................................................26Figure 15. Core Game Engine Class Diagram..........................................................................32Figure 16. Example Map File...................................................................................................33 Figure 17. A* Search Algorithm.............................................................................................35Figure 18. Agent Inheritance Hierarchy...................................................................................38Figure 19 Game/Test bed Window...........................................................................................43Figure 20 Police FSM...............................................................................................................48Figure 21 Civilian FSM............................................................................................................51Figure 22 Robber FSM.............................................................................................................55

6

September 16, 2011


AbstractThe focus of this research was in the area of Artificial Intelligence in Computer Games. The

main goal was to investigate how emotions can be modelled in character agents in computer

games. The study was based on the effect of emotions as an arbitration mechanism for a

reactive goal based architecture. The research approach involved simulation of real world

situations in the form of a C++ program. The program consisted of three types of agents

namely police, civilians and robbers. The program puts these agents in different situations

where they can interact with each other i.e. the game includes multi-agent interaction. An

example situation consists of a rescue operation carried out by a group of policemen where

insurgents have broken into a bank and taken some civilians hostage. . Such scenarios

provide a good investigative platform for research as they commonly occur in real life and

are also frequently modelled in computer games. An emotion model has been used to select

the emotions for the project. These are fear, anger, disgust, sadness and happiness. These

emotions are applied on top of the reactive agent architecture. The comparison made between

the emotive and non-emotive agents, trying to accomplish the same goals, has been used to

analyse the effect of emotion in video games. Other essential features include the judicious

use of Finite State Machines as part of the architecture and an efficient approach towards AI

test-bed design.

7

September 16, 2011


Chapter 1 Introduction

This study focuses on the research and implementation of emotions in computer games. The

focus is on how the overall nature of a game can be changed with the addition of human like

emotions in character agents. The comparison and contrast of the resulting emotional

response, along with the default behaviour of an agent can then be further investigated for the

possibility of improving realism in computer games.

Many of the theories and techniques applied in computer games are derived from the field of

Artificial Intelligence (AI). The common goal of majority of these techniques is to model the

most realistic human or human-like behaviour possible in artificial entities or agents (Nilsson

1998). Such behaviour is complex, uncertain and is expected to occur immediately or in real-

time (Agre and Chapman 1987). A game agent can be defined as a computer controlled

human or non-human character or entity. In other words, any object in the game that is not

controlled by the player (Shihab 2009). This form of computer controlled behaviour is what

gives the perception of ‘intelligence’ in agents. Intelligence can be generalised in terms of

“perception, learning, communication, and acting in complex environments” (Nilsson 1998).

This allows us to understand the rationality of the term ‘Artificial Intelligence’ and allows us

to coin this term once again through our own observation of what an agent does when

programmed to behave in a certain way i.e. a rational response.

It is then we realise where the term ‘Artificial’ comes from and for this reason the term

artefacts applies aptly to agents in computer games. Computer games are geared towards

providing a predictable form of intelligence because a game is planned to start and end in a

specific manner. However, with the evolution of the computer games industry, many gamers

now require a more immersive and unconventional gameplay experience.

In order to achieve said experience, the AI has to move away from a rational behaviour

towards a more empirical methodology (Russell and Norvig 2003). Emotional response is one

such area of research. An emotion can be defined as something that occurs spontaneously. It

is a term that is synonymous with human behaviour. This has to be taken into account when

designing suitable software involving such behaviour. If the application and translation of

emotional behaviour in a video game is done accurately, it will by default exhibit

characteristics of unpredictability and differ from the conventional progression of computer

8

September 16, 2011


games from past years. Emotions and their resulting behaviours are complex and have been

deeply studied by theorists. Some of these theories also have biased opinions. This research

chose not to delve on such theories or postulate any new ideas in order to form a

revolutionary theoretical base. In contrast, the idea of the study was to embrace and conform

to existing suitable ideas from the area of human psychology. These ideas were then matched

with computational techniques in order to design an application to demonstrate emotional

behaviour in a computer game.

Adapting emotions in games is a recent idea and is the main reason for the choice of this

topic. As mentioned previously, not only do emotions aim to replicate human behaviour, they

do so with a high level of unpredictability. This does not mean that this will be beneficial; it

can also be detrimental to a game scenario. The extent of application of emotions in computer

games has to be in accordance with the requirements of a game scenario. For this reason, such

applications have to be tackled in an empirical manner. Only through accurate observation

and experimentation, suitable game AI can be designed that offers not only unique gameplay

but also holds entertainment value for the player.

Some people may argue that this is what learning agents provide in a game. They adapt to the

gameplay and offer a different experience each time. Emotional agents differ in the manner

that they exhibit behaviour that allows the player to connect very easily on a physiological

level. The emotions and actions displayed are familiar to the player in some past scenario.

The study involved implementation of human emotions in the form of a software test bed.

The testbed consists of a mini game involving 3 types of agents namely Civilians, Robbers

and Police. The game puts these agents in a shared game world where they can interact

amongst each other and with the environment. The behaviour of the agents is according to a

game storyline. The storyline puts these agents in a real-time bank robbery scenario.

Systematic and hierarchical Agent design was one of the biggest challenges of this

dissertation. A goal based reactive architecture was used for agent design. The project was

implemented using C++ and SFML1. The agent decisions were implemented using Finite

state Machines (FSM). Detailed explanation of the FSM implementation has been provided in

the report but an attempt will not be made to explain the theory behind finite state machines

as it was a tool that was required for the project. Briefly,

“A finite state machine is a device, or a model of a device, which has a finite number of states

it can be in at any given time and can operate on input to either make transitions from one

1 Simple and Fast Media Library www.sfml-dev.org

9

September 16, 2011


state to another or to cause an output or action to take place. A finite state machine can only

be in one state at any moment in time.”(Buckland2005).

Further work involved the selection of a suitable mechanism that would employ emotions as

an arbitrator for the selection of actions.

The report has been written in conjunction with the implementation of the project. It will

therefore be noticeable that each subsequent section in the report is dependent on the previous

one and the report progression is in a chronological format.

Other emphasis has been placed on good AI testbed design and good software practices for AI

programming. The programming techniques have been evaluated against suitable testbed

requirements to ensure that the end product was of the highest standard and could be extended

further in the future.

10

September 16, 2011


Chapter 2 Project Overview

2.1 Initial Brief

The initial brief of the project was open to negotiation as many different approaches can be

followed. The original project specification can be seen in Appendix A.

This specification was updated and can be found in Appendix B .

2.2 Context

The main goal of the research is to study a method of Artificial Intelligence that can make

games more interesting and immersive. The idea is to involve human emotions in character

agents in computer games. There have been many games in the past where emotions have

been used. Some examples are Black and White and the Fable series from Lionhead Studios.

These games present emotion in such a way that the outcome of a decision taken by the player

causes an emotion or affect in the character agents which depends on their current emotive or

non-emotive state. Another example of emotion can be seen in football games. Most of the

emotion in football games is displayed out of the game by the players. This is triggered by the

competitive gameplay which football games provide and shows that emotions also play a big

part when it comes to competition or rivalry. Another popular genre of games is first person

shooters. In these games emotions are more likely triggered in players through story or

aesthetic elements and visual feedback of critical events in the game. This leads to heightened

emotions in the player which then affects the overall outcome of the player’s decision and

game events. These are all examples where emotions are displayed and provide evidence of

the outcome of emotions in terms of game playing. It is with this in mind that this project has

been chosen and aims to investigate how an ever changing gameplay experience can be

achieved through human like emotions using multi-agent coordination.

11

September 16, 2011


2.3 Aims and objectives

For the purpose of clarity the aims have been listed in a chronological order

i. Research on emotion and affective computation

ii. Research and selection of one model of emotion for application

iii. Scenario and story design of computer game test-bed

iv. Selection of characters and events in storyline

v. Graphical interface design of software test-bed in C++

vi. Design of Finite state machines including input, change and output functions for characters

vii. Design of a character agent termed level0 agent with wandering movement attributes using

finite state machines in C++

viii. Use the level0 agent as the base agent and derive all required characters from this agent

ix. Implement required AI features to enable inter-agent coordination i.e.

functionality/playability of the game.

x. Extend the level 1 agent with the emotion model.

xi. Experimentation and testing/comparison of emotive and non emotive agents and their actions

xii. Further scope of the project by extending the chosen emotion model.

2.4 Project Plan

The project plan can be seen in Appendix E

12

September 16, 2011


Chapter 3 Project BackgroundThis chapter provides an insight into the theoretical framework of the study. It discusses the

need for an emotion model that has been implemented in the design and provides a detailed

overview of the essential background research for this project. As human emotion is a topic

that is more concurrent with the field of psychology, the implementation in video games

requires a broader understanding of the topic. It is for this reason an emotion model has to be

adapted into the design.

The emotion model provides an in depth view of the main theme of this study, emotion and

affect, thereby solidifying the view on affective computation. It shows through example

scenarios how and which kind of emotions can be elicited in character agents.

These ideas are extended further and embedded in a storyline that consists of events where

emotions are induced an affect is experienced. A C++ based programme is used to provide a

visual outcome of the interactions between emotive agents.

Me: needs a bit of editing a slightly broader introduction to the chapter more emphasis on affective computation.

3.1 Early Theories of Emotion

The oxford dictionary defines emotion as “a strong feeling deriving from one's circumstances,

mood, or relationships with others”. This provides us with a generic definition that many

people can relate to but likewise there are many that disagree.

The earliest theories are related to the theories of evolution. Charles Darwin proposed that

emotions evolved in order for humans to adapt. It was a process that was part of Natural

Selection (Darwin 1872). This was his reason behind all humans all around the world sharing

the same emotional response (Gaulin and Donald 2003).

William James, a pioneer psychologist and physiologist, defined emotion as “the perception

of bodily changes as they occur is defined as emotion” (James 1884). This was supported by

Carl Lange and collectively became known as the “James-Lange Theory”. In contrast to this

theory the “Cannon-Bard” Theory says that bodily changes are caused by emotion

completely opposing James and Lange (Britannica Encyclopaedia 2011). A third theory that

13

September 16, 2011


attempted to mediate between these two conflicting theories was the “two factor theory of

emotion”. This theory stated that emotions were caused by both “physical arousal” but was

(were?) not the sole cause of emotion (Schater and Singer 1962).

All these theories provided conflicting ideas as to the origin of human emotion when grouped

together. At this stage, none of these theories could be applied in any way in the

implementation phase of the project. However, these theories provided a benchmark of ideas

to be compared with theorists in the later years. It is the theories of this newer era that would

fulfil the requirement for a suitable model of emotion for the project. Prior to this discovery,

some additional gaps have to be filled in for a better understanding.

3. 2 Affect and Affective Computation

In order to understand affect and its relevance to the study, it was necessary to delve further

into different branches of psychology. Before understanding affect, it is necessary to define

another term relevant to the thought process. ‘Cognition’ refers to a state of mental activity.

Cognition refers to the minds abilities which include memory, perception, thought processes,

speech and problem solving (Feist and Rosenberg 2009). Affect can now be defined as the

feeling of emotion that can be differentiated from all aspects of cognition (Huitt 2003). Since

this is a very abstract definition, affect can also be said to represent a reaction to external or

internal stimuli, manifesting itself in the form of feelings and or emotions. Therefore, affect

can be thought to occur in the time period between a stimulus and the reaction to a stimulus.

This knowledge was sufficient to move on to the computational domain. The transition of a

theoretical model of emotion into a software design depends on previous work done in this

area. As the topic of emotion is a much specialised topic, it falls under a niche topic of AI

known as Affective Computation.

It has been a challenge for people to understand the meaning of affect due to the abstract

nature of the term. For this reason, affective computation was not used in computing for a

very long time and has undergone development in fairly recent years. The origins of affective

computing date back to as recently as 1995 where it has been defined as “computing that

relates to, arises from, or deliberately influences emotions” (Picard 1995). Picard has been

part of MIT University’s research group on affective computation, which is one of the leading

authorities in this field. At this point this was enough understanding of the problem domain in

order to progress to a computational model. Any proposed computational model would have

to conform to the rule that emotions were being influenced deliberately. However, the

reaction of the agents will have following the emotional response will be very intriguing. The

reactions may or may not follow trends of human unpredictability and complexity but will

14

September 16, 2011


however follow a very intelligent and unique response. It is these outcomes and responses that

will allow the evaluation of what new ideas can make a computer game more interesting.

3.3 Requirement for an Emotion Architecture

This section takes the definitions of emotion and affect and puts them into a more practical

perspective. The task at hand involves creating a game that consists of core entities in a

world, which involves real time interaction in human based scenarios. In order to do so the

architecture has to consist of a core driving mechanism. In this case the game will consist

consists of goal based reactive agents. A reactive agent can be defined as an agent which upon

receiving an input, will process it and return or carry out a suitable output. The agent will

continue to follow these protocols until it has achieved its assigned goal or it ceases to exist.

The input for the reactive agent will come from interaction with other agents and external

stimulus from the game world. Once this has been enforced, it can be said that the closed

system now consists of a game world and human-like agents carrying out assigned tasks.

In spite of this, the agents are only human-like in the nature of tasks that will be performed

and not in the way that these tasks should be performed. For the agent to be human, it will

have to display characteristic of a similar grade of unpredictability and complexity. In

retrospect to the agent being goal based and reactive, it still follows a sequence of carefully

selected inputs. In other words the agent ‘does what it is told to do’.

On the other hand, in a multi-agent environment, unpredictability is factored in

automatically. The probability of a specific output being executed reduces after interaction

with another agent. The precedence of execution of the output of one reactive agent may

suppress the input or even the output of another agent. For e.g. if an agent is killed in its input

processing stage, then it will cease to exist even before its output can be executed.

In this way the interactions of multiple agents provide an admissible level of unpredictability

and complexity in a game world.

In many games, this architecture will suffice. When coupled with all the other aesthetic

aspects of a video game, such goal based reactive agents provide a reasonable entertainment

experience. The only problem is that this experience is short lived because it is completely

exhaustible. What this means is, there will be a time when the game will behave in a way that

has been seen many times before. This is when a game becomes “boring”. It ceases to provide

15

September 16, 2011


any kind of challenge to the player. It has now become completely exhaustible . The player

will now abandon the game and move onto the next one.

A popular method of making a game more interesting is a learning agent. A learning agent is

an agent that will constantly improve itself in different situations in order to find a beneficial

outcome (Russell and Norvig 2003). The detailed theory behind a learning agent is out of the

scope of this project. The learning agent offers a comparison against an emotive agent. The

learning agent differs in the fact that it will continue to improve itself until it ceases to exist.

In this manner a learning agent will be different every time. The disadvantage of the learning

agent is that if it gets too proficient at its task, the game will run the risk of abandonment by

the player. The whole idea of a computer game is to offer the players an achievable challenge

to make them think that they have won, while it can so easily be said that the AI has been

tweaked to lose. Thus a learning agent cannot be forever learning in a commercial computer

game. It will have to be limited at some point to ensure the success of the game.

It is imperative at this point that a new class of AI agents is required that which are not only

unpredictable but engaging at the same time. This is where the emotive agents differ from

learning agents. The first and foremost quality of an emotional agent is that it is akin to

humans because it is “emotional”. It exhibits one of the biggest qualities in human beings. We

have already established that a reactive agent can process tasks. Adding emotion in this

equation will add a new stage in the input processing and output of a reactive agent. It is this

new stage which will allow for a more realistic

Need something more here but for now a mental block on this topic,. Will review later needs a sort of link to the next topic

and engaging gameplay.

3.4 Emotion Redefined: The Emotion Model

With the basic theory in place, this section focuses on the research of emotion by theorists in

the late 20th century following the incomplete ending of section 2.1.

The emotions that will be modelled in the project are based on the definition of Oatley and

Jenkins(1996) :

“A state usually caused by an event of importance to the subject. It typically includes

(a) a conscious mental state with a recognizable quality of feeling and directed towards

16

September 16, 2011


some object

(b) a bodily perturbation of some kind

(c) recognisable expressions of the face, tone of voice, and gesture

(d) a readiness for certain kinds of action”.

Other theorists and psychologists have postulated similar theories. These include Frijda

(1986),

Power and Dalgeish (1997). Their agreement on the definitions of the 5 basic emotions will

be used to model the agent behaviour in this project.

These are quoted as follows:

Fear defined as the physical or social threat to self, or a valued role or goal

Anger defined as the blocking or frustrations of a role or goal through the perceived actions

of another agent

Disgust defined as the elimination or distancing from person, object, or idea repulsive to self

and valued roles and goals

Sadness defined as the loss or failure (actual or possible) of a valued role or goal

Happiness defined as the successful move towards or completion of a valued role or goal

17

September 16, 2011


Chapter 4 Theoretical Design

4.1 Game Story and Agents

Figure 1 shows an example of the game world. The game world consists of a top down view

of a bank. The world is tile based and each tile represents a movement node. The physical

properties of the world include obstacles that are represented by different pictorial forms. In

addition to this, the game world consists of entities and agents as can be seen in the

screenshot below. A detailed description of individual agents has been provided in the

following sections.

Figure 1 Example Game world

The game story consists of 3 types of agents namely, policemen, civilians and bank robbers.

The scenario involves a bank in which some robbers have broken into. The robbers will

attempt to get to the vault holding the money and the police will try and stop them from doing

so. It is the job of the soldiers to evacuate the civilians and arrest the robbers.

18

September 16, 2011


In simple terms each agent has its own designated goal. A more specific breakdown of the

agents’ tasks has been shown in chapter 7, which discusses the overall gameplay. These goals

eventually lead to default actions of each agent. Adding emotions in the game will lead to a

change inaction selection by the agents in accordance with the current emotion actor scenario.

Some examples of these scenarios are shown in the next section.

4.2 Emotion Actor Scenario

A further understanding of how the agents in the game will act out emotions can be shown

with the help of some examples. The following examples outline the 5 basic emotions in a

game scenario. The agents in question are part of 3 different sets. The police(green squares),

bank robbers (red triangles) and the civilians (blue circles). The following figures show how

these emotions can be elicited. These scenarios are only a few of the limitless possibilities

that can be modelled. At this point they are purely to detail understanding of the 5 basic

emotions and how they can be applied in the game.

Figure 2. Fear

Figure 2 A civilian get attacked and the first emotion they feel is fear.

19

September 16, 2011


Figure 3. Sadness

Figure 3 A team member of the Police dies, or was killed by an enemy. The rest of the team

feel sadness.

Figure 4. Disgust

Figure 4 One of the members of the police is a traitor and in the middle of the game defects to

the other side. The other team members are now in disgust.

20

September 16, 2011


Figure 5. Anger

Figure 5 Robbers are trying to take civilians hostage. This is their goal. The police are

preventing the robbers from completing their goal. The robbers feel angered.

21

September 16, 2011


Figure 6. Happiness

Figure 6 A Robber has taken a civilian hostage. The tactical team’s goal is to rescue the

civilian. They complete their goal and are now happy

4.3 Agent Architecture

The design of the agents, their behaviour and action was to be done incrementally starting

from the lowest level. The overall architecture can be explained better with the help of

diagrams. These diagrams will be modelled sequentially to design an overall evolved

emotional agent.

Figure 7 shows the most basic level of agent design. This level comprises of an agent’s

sensory level or the means by which an agent experiences external stimuli. The action based

on this stimulus will be totally reflexive, akin to the most natural of responses by humans.

There are virtually no goals present in the architecture at this time. The agent can therefore

22

September 16, 2011


only offer an involuntary reaction. Example predator coming towards prey, prey screams in

fear.

Figure 7. Reflex agent architecture

The next step in this design is to give the agents some thought. This level is known as the

reactive level because it allows the agents to perform an action or a reaction. This is slower

than a reflex and where it differs is that the reaction is voluntary. This takes into account the

reflex level. This is represented in Figure 8. Example prey now runs in fear. This gives the

agent some reactive behaviour.

Figure 8. Reactive agent architecture

In order to make the agent more intelligent, the agent requires decision making capabilities. In

order to accomplish this agent needs some goals. The actions and decisions of the agents will

now be based on their goals i.e. the agents are completely goal driven. This is shown in

Figure 9. For example, a robber trying to take a civilian hostage; it will not deviate from this

goal until achieved.

Figure 9. Reactive and deliberate level architecture

Figure 9 can be simplified by combing the reactive and deliberate level into one level called

the Reactive behaviour level. Figure 10 now defines how an agent behaves in an environment.

23

September 16, 2011


Figure 10. Reactive behaviour architecture

This will be the basis for a Class 0 agent, the most simplified agent in the architecture.

This architecture is now extended by adding more goals to its existing reactive behaviour.

These goals will differ across the 3 classes of agents i.e. the soldiers, robbers and the

civilians. This is shown in figure 11

Figure 11. Goal based reactive behaviour

The next agent architecture is the core of the project. This is the point where emotion will be

added to the reactive behaviour. As shown in Figure 12, the agents will now be driven by a

particular emotion. There will be 5 possibilities of this architecture, each modelled with one

of the 5 basic emotions. Not all agents may require each of the 5 emotions. The civilians for

example will be modelled with a subset of the 5 emotions. This is termer the Class1 Agent

Figure 12. Emotion based reactive behaviour

24

September 16, 2011


Class 1a agent will be the combination of both the class 0 & class 1 agent. The behaviour

will be interdependent with the current goals and resulting emotions of the agent. Both the

goals and emotions will control how the agent behaves. This is shown in figure 13

Figure 13. Emotion and goal based reactive agent architecture

If time permits this architecture will be extended further, to incorporate the secondary goal of

agents with multiple emotions and goals i.e. agents with a personality. This will be the most

complete architecture that can be used in a commercial game because agents will then be able

to behave differently depending on how a player has progressed through the game and what

kind of agent behaviour his/her actions have resulted in.

25

September 16, 2011


Chapter 5 AI Test Bed Design This chapter involves the implementation of the AI testbed in C++.

5.1 Testbed Theory

It was mentioned earlier that the project requires an empirical approach towards

experimentation. There are many theories that have been mentioned that need to be

investigated through careful and controlled testing. From here onwards the game will be

referred to as the Emotion AI testbed.

Before we can understand what testbed is, the term Benchmark needs to be introduced. In

computing, a benchmark is usually used to compare performance against. A specification is

the minimum requirements a product must complete in order to be acceptable. A benchmark

differs in the respect that it allows the scientist to deduce of the behaviour of the product

(Hanks 1993). In the case of this project the minimum benchmark requirements refer to a

series of tasks that must be accomplished. Figure 14 shows a table in which some of the

benchmarks have been listed for each of the 3 characters.

Agent Benchmarks

Civilian Ability to Move in the world environment

Ability to stay isolated in idle tasks

Soldier Ability to Move in the world environment

Ability to differentiate between civilian and robber

Robber Ability to Move in the world environment

Ability to target civilian or soldier

Figure 14. Example Benchmarks

26

September 16, 2011


Usually a benchmark is a measure of performance. In the case of this project, a benchmark is

a reflection of the basic mandatory tasks that a controlled AI test bed should be able to

perform. These tasks are invaluable as they form the basis of the implementation of the AI

test bed.

Taking the context of this project into account, an AI test bed can be defined as a platform for

evaluation and experimentation of proposed emotion theories and the relevant computational

model.

5.2 Testbed Design Constraints

Test bed design is one of the most critical aspects of agent design. There are some

characteristics that needed to be taken into consideration for the AI test bed. These are issues

that are prevalent in testbed design across different AI research applications and need to be

addressed for any significant progress. The following characteristics define an ideal test bed

(Hanks 1993).

5.2.1 Choice of a planning or reactive system:

A planning system requires that the input state of a system is known beforehand. The task of

the agent is to achieve the state through a series of pre-defined actions. This however was not

the requirement of the project. The whole idea behind the project was to design unpredictable

behaviour in the agents. A reactive system was employed which consisted of a set of pre-

defined actions and the selection of these actions was dependent on the current state of the

agents. In other words the choice of the actions was in reaction to the current state .i.e. a

reactive system.

5.2.2 Complexity of the world

In order to model realistic human behaviour the world properties also have to be taken into

account. Ina computer game, the world defines the domain which an agent will interact with.

In order for this interaction to be realistic the physical properties of the world also had to be

modelled in the test bed. Depending on the application some characteristics may be irrelevant.

In this case the only characteristic modelled was the ability of the walls and objects to act as

physical obstacles. This was the sole interaction of the agents in the game world.

5.2.3 Sensing and Effecting

Once the world properties have been incorporated, the test bed requires an agent to interact

with its surroundings and other agents in the environment. This requires the agent to be

perceptive in its environment. This ability is termed as sensing. The actions followed from

27

September 16, 2011


sensing and the interpretation of external or internal stimuli is known as effecting. In order to

preserve the causality of an AI application, the agents need to be aware of all events around

them. Appropriate sense modules need to be included in the test bed to provide this

functionality. The sense modules then need to be linked with the action execution mechanism

of the test bed. The action selection can be done using many of the preferred AI design

schemes. This project used finite state machines as the technique of choice. This linked with

the sense module allows the agent to interact with its surroundings.

5.2.4 Partial Satisfaction of Goal States

For an agent to be believable, it has to display some imperfections which are akin to human

behaviour. It is for this reason that the sensing and effecting cannot always be accurate. An

agent is usually required to move from one state to another or towards a required goal. An

exogenous event is likely to interrupt this transition from one goal state to another. These are

imperfections in the sensing and in turn the effecting because as is evident in human

behaviour, not all goals are not achieved all the time. In certain scenarios, this interruption

may even be beneficial.. For example in a reactive system sometimes it may be beneficial for

the prey to interrupt a goal transition to flee if a predator suddenly approaches. An ideal test

bed should be able to provide support for partial fulfilment of goal states.

5.2.5 Multiple Agent Coordination and Communication

The selection of the bank robbery scenario implied first and foremost that the test bed would

consist of multiple agents of different types. In due course it was inevitable in the

implementation that the agents were required to interact with each other. This was essential

for the success of the game and the progression of the story line. There were certain factors in

the game that required one agent to interact with another. For example, if a robber wanted to

take a civilian hostage, it had to instruct the civilian to follow it. To support the sense modules

in a test-bed, support for multiple agent interaction is not only advantageous but essential. It

is advantageous because it exposes the unknown behaviours that are presented when agents

interact with each other. It is essential because even though individual agents are predictable,

multiple interactions are not and need to be controlled.

5.2.6 A Clean Interface

A big aspect of software design is software maintenance. With this in mind, the test bed was

designed in a modular fashion. The bottom up approach consisted of adding different sections

and modules as the implementation progressed. This meant that the sensing and effecting

modules were designed in distinct and sequential stages. Also all the data was managed

28

September 16, 2011


elsewhere in the software and passed onto the agents as was required. The clean designed

offered a clear understanding of the data flow in the test bed, the evaluation of the data by the

agents, and the actions in reaction to the data. The chapter on software implementation

discusses this in more detail.

5.2.7 Time Module

The occurrence of events and actions in a testbed must be measured through the passage of

time. All inputs processing and output has to be invoked at specific time intervals for a

feasible operation of the program. For the simulation to be acceptable each agent requires a

global time measurement strategy that it can use in its action selection. The design scheme for

this project consisted of Finite-State-Machines. At each instant of time, an agent was

considered to be in one of many finite states applicable to its design. The timing strategy

chosen was a turn based clock. Each turn represented a unit of time, the culmination of

which, led to the next or final state. In addition to this another clock was implemented that

saved the system time for the entire simulation. This was also used by various other parts of

the software, such as the graphics modules. The entire passage of simulation has to be

available as it is a vital tool used in experimentation and testing.

5.2.8 Experimentation and Testing

The method of experimentation depends on the technique of AI. An ideal testbed should be

able to gather data from the simulation and present it for analysis at the end. Again the criteria

for this data can vary. For example, when testing two different path finding techniques, a

measure of time and the number of movement nodes selected offers a brief evaluation of the

performance of each technique. This can be stored in a text format over multiple simulations.

On a larger scale, a survival game can store success and failure rates of a protagonist over

time or multiple simulations. These results will reflect how the AI affects the behaviour in a

testbed.

For rigorous evaluation, a testbed should be flexible enough to alter different behaviours and

observe the results. This is of utmost importance because it allows the designer to compare

and contrast plausible scenarios in order to deduce the most proficient one. The overall

quality in terms of experimentation and testing of a testbed depends on the number of

available attributes that can be altered as required.

29

September 16, 2011


Chapter 6 Software Design The chapter covers design methodologies used for the software. It covers the design tools

used and focuses on the overall implementation in C++. Initial topics cover the tools used

followed by a class diagram of the testbed. The roles of individual classes are also explained.

The idea is to give a clear understanding of how the gameplay elements function in terms of

programming.

6.1 Software tools and SFML

The software has been designed using Microsoft Visual Studio 2010 using the C++

programming language. There are no basic hardware requirements as the testbed is not

graphically intensive. The graphics have been implemented using SFML.

The Simple and Fast Media Layer (SFML) is an object oriented API (Application

programming Interface) that has been written in C++. It provides high and low level access to

features such as graphics, input, audio and sound among others. It has been designed by

Laurent Gomila and is an open source free to use API.

It was used to fulfil the requirement of a quick OpenGL interface with C++ in visual studio.

This was a better alternative than glut or any other OpenGL library because it provided a

clearer and easier to manager interface with many other media components including

graphics. The commands and level of abstraction provided by SFML made graphics

programming easy.

The version of SFML used for this project was version 1.6. All the required files have been

included with the source code. The key things to note that for using SFML the libraries need

to be compiled before they are included in a project or this will result in linker errors. The

libraries also need to be specified in the linker settings in visual studio. The exact procedure

of setting up visual studio with SFML can be found on the website. http://www.sfml-dev.org/

Please note that the website mentions the libraries for use with Microsoft Visual studio 2008.

In order to use the libraries with visual studio 2010, these steps need to be followed.

i. Download the required version of SFML from the website

ii. If the version contains a visual studio solution open it with visual studio 2010

30

http://www.sfml-dev.org/

September 16, 2011


iii. Compile the solution in both release and debug mode to build the static(.lib) and dynamic

libraries(dll) for visual studio 2010.

iv. Use these newly generated library files for visual studio 2010.

If the suitable include and lib folders have been included in the project and the required lib

files have been specified for the linker in visual studio, the project will compile.

6.2 C++ Architecture and Implementation

It was initially intended to use a pre built game and modify it into a desired test bed. The idea

was to use a game called Raven (Buckland 2008). This idea was abandoned and the complete

software design was done from the beginning. The idea behind this was to personalise all

aspects of the dissertation. Another reason was to implement all the basic requirements of a

good testbed that have been mentioned in section 5.2. This resulted in a small 2D tile game

engine. The key features of the engine have been highlighted in this chapter.

The complete software architecture has been explained with the help of diagrams. The

diagram shows the agent hierarchy as well as the other main and utility classes. A brief

description of each class is specified after each diagram.

31

September 16, 2011


6.2.1 Core Game Engine

Figure 15. Core Game Engine Class Diagram

Figure 15 shows the Engine Architecture in its most basic form. The helper and utility classes

have been removed to give a clearer understanding of the software architecture.

6.2.2 Map

Before the explanation of all the classes it will be useful to explain how some of the data is

stored. The project contains a text file which contains the game map. The map contains an

array of numbers which represents the map grid. The rows and columns are specified at the

beginning of the file. The numbers in the map vary from 0 to 5. This can be extended as

required up to 9. The 0s on the map represent walk able grid nodes and the 1s on the map

were used to represent obstacles. The numbers 2, 3, 4 and 5 have been used to represent the

start/spawn positions of the agents in the game. One of the tasks of the policemen was to

patrol from room to room. This has been achieved by representing room doors as number 8s

in the map. The police then patrol between these specified doors. This design was one of the

32

September 16, 2011


main reasons for creating an all new test bed. The advantage of this design is that it is

easily extendable and adaptable. It is also not testbed specific. This approach can be used in

many other tile based games. As will be seen later, it is perfect for path finding algorithms as

the numbers can be used to represent different terrain along with obstacles. Another

advantage is that the size and complexity of the map can be increased or decreased by

just editing the map file. The software remains unaffected by this and will continue to

function as normal assuming all the data required has been provided in the map.

The only disadvantage of this design is that one cell can only represent one unit of

information. An improvement on this design will be to use the bits of a number in each cell.

An example map can be seen in figure 16.

Figure 16. Example Map File

6.2.3 Class LevelLoader

LevelLoader acts as a parser for the map data. It specifies storage containers in the form of

STL2 vectors for storing the data from the map file. Separate vectors were created for

individual data types. These can be seen declared in the header file. The most important thing

to understand is that the LevelLoader provides the entire 2 dimensional x and y coordinates

for every piece of information in the game. These coordinates are essential for the movement

2 Standard Template Library

33

September 16, 2011


logic including path finding. The coordinates are also used for the graphics in the Engine

class. The Load() method parses the file data. GetMapGrid() gets the number of cells in the

map along with the number of rows and columns. GetSpawnPoints() gets the position of all

the spawn points for every agent along with the door positions.

6.2.4 Class ImageManager

The biggest problem in 2D games is image management. When an image is loaded from a

file, it is stored as an array of pixels in memory. This image is now a resource used by the

program (Gomila 2007-2011). The problem with this is that the image itself is an expensive

resource. It is slow to load every time and large in size. This is where the term sprite comes

in. The sprite is a copy or a visual representation of the image that points to the original image

in memory. With sprites it is possible to have multiple sprites of the same image. It is also

faster to load and smaller in size. This was one of the main reasons for using SFML as it has

built in support for image to sprite conversion.

The problem faced in the testbed was loading and allocating these images in a clear and

concise manner. This task was abstracted to the ImageManager class. All the images were

stored in an STL vector for dynamic allocation. The methods AddImage() and GetImage()

were responsible for adding and retrieving images from memory respectively. In this way the

ImageManager is responsible for the Memory management of the images. All the user has to

do is define a path to the image and specify a sprite tag to use with the image.

6.2.5 Class Agent

This class is the parent or base class for all the agents in the system. All the other agents

inherit from this class. It interacts with the Engine class to gather all the required data which

was subsequently used by the derived or children classes. Section 6.2.2 details the

implementation of the agent hierarchy.

6.2.6 Class Pathfinder

There were various elements in the game that required specific character movements. For

example, movement of the robbers to the bank vault as well as chasing and evading agents.

Two approaches were chosen at this point. The first was to implement some logic for the

different types of movement. The second was to use a search space algorithm for path finding.

The latter approach was thought to be beneficial because it could fulfil all the requirements. A

path finding algorithm could be used in many different ways without editing the actual

algorithm. It could also be used in combination with a Finite State Machine (FSM) design for

reactive agents. The FSM design has been discussed in the next chapter.

34

September 16, 2011


The algorithm used for this project was the A* (A-Star) Search Algorithm. Even though the

implementation of this was not an easy task, it has not been fully explained in the report. This

was a tool that was required and its explanation is not in the scope of this project. A clearer

explanation of how the search algorithm works can be understood from the source code in the

Pathfinder class. A brief explanation has been given with the help of figure 17.

In the AI test bed, the A* algorithm is applied on the game map, which consists of a grid of

squares. A* works by finding the shortest distance over the map grid from a start node to a

target node. The nodes are the x and y coordinates in 2 dimensional space on the map. The

advantage of A* is that it is guaranteed to always find the optimum path.

Figure 17. A* Search Algorithm

In figure 16, consider Yellow circle at the start node. We can estimate how many cells it will

take to Enemy B, which is our end node represented by one of the blue circles. This

estimation of how much we think will cost to get to the goal is known as the heuristic or h.

The next movement for the Yellow circle is downward at a cost of 1 cell. This is known as

our cost or g. The best cell/node for movement is therefore determined by f which is a sum of

g+h. In addition we can predefine wall avoidance by identifying cells with 0 movement.

This allows for built in collision detection if the pathfinder is used. The addition of this

technique gave the programmer two kinds of wall avoidance mechanisms in the testbed.

35

Enemy A Enemy B

Worst Path

Optimum Path

September 16, 2011


The first one was through the map file and the second one was through the pathfinder. Either

of these could be used. The idea behind this was to make the testbed more flexible for

future projects and work.

One of the biggest disadvantages of A* is that it is computationally more expensive. The

reason A* was chosen was because of the familiarity with its implementation in C++. An

alternate approach would be to use the Best First Heuristic Search. It is a more simplified and

computationally less expensive than A*. It differs to A* in such a way that instead of

calculating a global path, it checks the next available nodes and expands them further to find

the best path (Davis 2011).

In a game where low performance is a limiting factor, Best First is a better option because it

offers a similar outcome to A* but at a lower computation overhead.

The map file provided agents with a copy of the world indicating obstacles. A second copy

was provided to the pathfinder class which was used by the A* implementation code.

6.2.7 Class Engine

This section focuses on the relation between the previously mentioned classes and the Engine

class in order to provide a stronger understanding of how the complete software works

together. The Engine aptly named, is the biggest and most important class of the entire engine

or testbed. It is responsible for all the inputs, processing and execution in the system. Its

functionality is better explained with the help of the methods used.

Init(): This method is used to initialize the engine. The initialization includes setting up a

rendering window in conjunction with SFML. Init() was also used to create a new object of

the LevelLoader class, required to load the level from the map file. The final method called in

Init() was the LoadContent() method.

LoadContent(): The beginning of this method includes data for the loading of sprite images

and passing them onto the ImageManager class. In the future if sprite animation is required, it

would be more beneficial to include these commands in the respective agent class.

The LevelLoader object created in the init() method is used here to specify which text file to

load for the map data. Note: If multiple levels are required different text files can be used and

a selection can be made between them.

The next content information is the map grid. This and other map information is stored in

respective STL vectors. At this point the engine has all the data from the map file in memory.

The map file is no longer required because continuously reading the map file is extremely

36

September 16, 2011


slow. In fact is the slowest process because it is read from the hard drive. This is where the

significance of data management can be seen. Since the engine has all the data now it can

be passed to whichever class requires it.

For example, a copy of the map is provided to both the pathfinder and the Agent class.

The final task of the LoadContent() method is to create all the agents in the game with the

help of the spawn positions gathered from the map file in one of the earlier steps.

ProcessInput(): The first method in the sequence of the 3 methods which are included in the

Main game loop. This method is used to detect all system events and all user input events. An

example is the movement of agents through keyboard or joypad buttons.

Update(): The second method in the main game loop. This is the global update for all the

agents in the game. It carries out all the individual updates of the agents in conjunction with

the system clock. One thing to highlight here is the distinct sensing and effecting

mechanisms in this method. The sense() method invoked by all the agents is the continuous

sensing for exogenous events and interactions. The call of the update () method calls each of

the individual updates of the agents. As mentioned in chapter 5, this is a good example of a

clean design interface.

RenderFrame(): The final method in the game loop. The graphics of the game are drawn in

this method. Initially, this method starts with the spawn positions of the agents. This position

changes at runtime in accordance with the game logic. This position is passed to the display

command which renders the graphics on screen.

MainLoop(): Calls ProcessInput() , Update() and RenderFrame() Sequentially per unit

execution time.

To finalise this section, it can be seen that the game Engine incorporates a clear flow of data.

It can be clearly seen that the data comes from the file is stored in the engine and forwarded to

respective classes. The design of the engine is also modular. Each new class or section was

added in the course of the implementation as per the requirement. The engine also takes into

account the requirements for a good testbed as stated in Chapter 5. All the characteristics

mentioned were implemented successfully. A lot of time was spent in the design of the

software. This proved to be a very good decision because the expansion of the game was

fairly quick afterwards because of the clean design.

As can be seen this engine is not project specific. Many other 2D based AI projects can be

implemented using the same architecture. There are also enough tools present to implement

37

September 16, 2011


simple 2D games with this architecture. This was what was meant by personalising the

architecture to suit the requirements.

Other classes that have not been mentioned were helper or utility classes and can be seen in

the source code.

6.3 Agent Hierarchy

Figure 15 was incomplete because it did not show the inheritance hierarchy of the Agent

class. This can be seen in figure 18.

Figure 18. Agent Inheritance Hierarchy

Class Agent is the parent class and all other classes have been derived from Agent0 which in

turn is a derived class of Agent. As per the previous section each class has been explained

with the help of its software implementation.

6.3.1 Class Agent

The first thing to note is that the class Agent does not represent an AI agent as per the

definition in the Introduction. To recap this definition, a game agent can be defined as a

38

September 16, 2011


computer controlled human or non-human character or entity. The Agent class can be seen as

more of as an entity rather than an agent. This class represents an agent that is human

controlled i.e. via the keyboard.

The reason for implementing this class was to distinguish between an entity and an agent. The

Agent has been represented by a small car in the game. Considering a remote controlled car, it

complies with given commands of movement. If however, it reaches a wall it cannot move in

that direction because the world does not permit it. The same analogy has been applied to the

Agent. It can be controlled by arrow keys and can move in 4 directions; up, left, down and

right. If however it reaches a wall it cannot move in that direction. This is to show that a

realistic knowledge of the world has been implemented in the testbed. The Agent has this

knowledge of the world but it does not have a thought process to control where to go in the

world. It does what it is told to without a mind of its own. This is the reason that it is an entity

rather than an agent.

In terms of implementation, the Agent class requests a copy of the map grid from the Engine

class. This agent also has default enumerated data types which are DEFAULT, AGENT0,

CIVILIAN, POLICE and ROBBER. DEFAULT refers to the entity or Agent class. This is to

distinguish the name from the Agent0 class. These types are used to represent the type of

agents during interaction with each other. Note: Another characteristic from chapter 5.

Agent also includes methods for movement in 4 directions. In addition to this it also includes

methods to check if a certain direction is free or blocked. This check is only done when the

agent is instructed to move in a certain direction. If there is an obstacle and it is instructed to

move, it will still try to move irrespective of the fact that it knows there is an obstacle there. It

cannot move because it has realistic knowledge of the world.

An important aspect of the design of this class is the 2D coordinates. Since this class was

derived further it was essential to separate the two kinds of coordinates in each agent or

entity. The first x and y coordinates of the agent refer to the logical nodes of the agent. These

logical nodes refer to the map and are values of x and y from the map grid. For example, (5,

5) is a position from the map grid. The second type of coordinates refers to the x and y

coordinates in terms of pixel size. These are used for the rendering of the graphics on screen

in the rendering window. These coordinates are the logical coordinates multiplied by the pixel

size of the window. For example, (5x1024, 5x768) are the render positions of the agents in the

game. These two data types have been separated clearly in the design such that the agent

only deals with its logical coordinates and the game engine deals with the conversion of

the logical coordinates to pixel coordinates. Note: Another example of a clean interface

39

September 16, 2011


The agent also includes virtual methods called sense() and update(). These are the sensory and

effect modules. In this class they are empty but have been overridden in the derived classes as

explained further.

6.3.2 Class Agent0

Agent0 represents the default behaviour of each character agent in the game and derives all

characteristics from the parent Agent class.

Agent0 has been represented in the game by a skeleton sprite. The sense and update methods

of Agent0 were overridden to give the agent some thought. Agent0 exhibits random

movement behaviour across the grid. What is actually happening is that at each node the

agent checks which of the 4 directions are free to move. It then stores all the available

directions and randomly selects between them to move onto the next node. This differs from

the parent class Agent because the Agent0 does not move if there is an obstacle whereas the

parent Agent realises there is an obstacle after the move command has been given by the user.

This shows that Agent0 is now able to sense and effect. It is able to first sense if a

direction is free, make a choice out of all the free directions and then effectively move in

one direction. This is one of the key aspects of agent design. This is a clear

representation of sensing, processing and action. At this point the hierarchy can be

extended further because Agent0 is the first unit of Artificial Intelligence in the testbed. From

here onwards this agent could have been extended to any required behaviour. This is another

example of the ability of the testbed to build many different projects using this architecture.

The drawback here is that Agent0 only interacts with the game world. In this respect the test

bed is said to be incomplete because multi-agent interaction is absent. The next level in the

hierarchy accommodates this requirement.

6.3.3 Class Civilian, Robber and Police

These three classes represent the more sophisticated agents in the game. Each class contains

its own unique methods for sensing and effecting. These methods are overrides of the Agent0

class. If in any instance these methods failed to activate, the default behaviour will be

that of the Agent0 class.

The sense modules of the these classes are continuously active i.e. they are always sensing. If

an agent senses another agent, it is able to distinguish which kind of agent by the enumerated

TYPE that was specified earlier in the inheritance hierarchy. Once an agent has sensed

something it activates specified Boolean flags. These flags are the control variables for the

effect methods.

40

September 16, 2011


The effect methods consist of Finite State Machines (FSM) for the Civilian, Robber and

Police classes. The FSMs have been explained in detail in the next chapter. The transition of

the states in the FSM is dependent on the Boolean flags which are set in the sense method . As

can be seen from this the software has created a link between sensing processing and

effecting for a Multi-Agent Environment . Other characteristics that have been

implemented include the ability of agents to send instructions of messages to each other. For

example, a robber tells a civilian that it is a hostage and the civilian is instructed to follow the

robber.

This chapter can be concluded on the grounds that at this stage in the dissertation all

functional tools of the testbed had been implemented. The testbed was complete but was

missing the gameplay logic. The missing piece of the testbed was the FSM design of the

civilian, robber and police along with the Emotion Architecture. The next chapter focuses on

the gameplay design and includes detailed information about the inner workings of each

agent. This understanding is necessary as it forms the basis for the implementation of the

emotion architecture.

41

September 16, 2011


Chapter 7 Gameplay and DesignChapter 6 focused on the low level design of the testbed. This chapter contains information

about the behavioural design of the game and the agents. It explains the rules of the game and

provides a clear understanding of the scope and limitations of the agents. In addition to this it

focuses on the biggest aspect of the testbed, the FSM design of the character agents. The FSM

design is closely linked to the gameplay of the system as this is the area where most of the

game logic was used.

7.1 Detailed Storyline

A brief description of the story was given in chapter 4. There are various details that were

omitted and are highlighted in this section. To put the gameplay into perspective, the game

window consists of a top down view of the bank. At any given time the interactions of all the

agents and their movement can be viewed by the user. Other things included in the scene

included various furniture and devices found in a bank. These are pictorial representations of

obstacles in the game. An easily distinguishable vault has also been shown in the scene.

These are labelled in figure 19.

This view represents the action scenario of the game. Action scenario represents the current

time in which an event has occurred i.e. bank robbery. There are three agents involved in the

scenario. The first are robbers which have invaded the bank and want to steal money from the

vault. Second are the civilians which are present in the bank. Finally, the task of the

policemen is to warn civilians and arrest the bank robbers. This completes the game scenario.

Once the simulation occurs, these actions are executed and continue until all the agents reach

their respective final goals. These are explained in more detail in the upcoming FSM design

section.

It is useful to note that there is a broader overview of the game scenario that had to be taken

into consideration. This is explained as follows.

It was mentioned that a bank robbery had occurred but not what event initiated the robbery.

One such event would be the triggering of the alarm in the bank. Another would be robbers

posing as civilians which suddenly show their presence.

42

September 16, 2011


Another situation to consider is the presence of the police. In a broader perspective, the police

would have to be alerted by the alarm or someone in the bank.

Finally the presence of civilians can be explained by dividing the civilians into further

categories such as employees and customers.

These were some of the many possible precursors of in scenario that needed to be taken into

consideration while designing the test bed. For this reason they have been mentioned in this

section but have not been shown in the implementation. This was to place emphasis on the

functionality of the agents in the test bed and not on the origin of actions and precursors.

However, in terms of research, these are important factors that need to be taken into account

on order to present a realistic and robust game scenario.

Figure 19 Game/Test bed Window

7.2 Control Factors and rules for the Testbed

The test bed was designed to act as a controlled and closed simulation environment. For the

agents to comply with the scenario of a bank robbery, certain rules and limitations had to be

43

VAULT

POLICE CIVILIAN

ROBBER

September 16, 2011


imposed. These can be seen as guidelines that form a controlled environment for specific test

bed experimentation.

A different scenario or test bed will have its own set of rules. As defined in chapter 6, the base

architecture of the software can remain the same. It is the scenario and the rules imposed in

the scenario that distinguishes one implementation from another. The following set of rules

highlight this project and its limitations.

i. The scope of each agent is limited in the area of the world that can be seen on screen. It is a

bounded region and the agent cannot exist outside it. The map can be extended and changed

as required but should continue to impose a boundary around it to prevent agents from leaving

the scope of the game.

ii. The start positions of all the agents need to be specified in the map file beforehand. The

agents will always start from this position.

iii. When designing the map, it needs to be insured that critical notes are not blocked by walls on

all sides. This is to insure that the required path is available to the Pathfinder. Critical nodes

should not require diagonal movement because the path finder can only move in 90 degree

directions.

iv. An agent can be interrupted during state transitions but once it has achieved the final state in

its cycle, the game is over for that agent. The final state was decided as part of the FSM

design. At this point the Agent must be reset to its initialised state and start position.

It should be noted that a testbed should be as flexible as possible. Imposing too many

limitations on a testbed defeats the purpose of experimentation. The rules on the individual

agents were enforced via the FSM design. These rules are not strict because their limitations

can be broken through multi-agent interactions. This may lead to behaviour that was not

accounted for in the design and may prove beneficial in analysis during experimentation. If

the output of the interaction with another agent is unsuitable, control parameters can be used

to adjust this. This was used to control behaviour when one agent communicated with

another. An example is the case of the robber taking the civilian hostage. This practice is

evident in the next section which focuses on the FSM design.

44

September 16, 2011


7.3 Finite State Machine Design

Before an AI technique can be implemented, its suitability needs to be analysed. This has

been done with the help of a requirements analysis table (Davis 2011) shown in Appendix C.

The table shows suitability against different criteria shown in the first column. It can be seen

that State Machines are the most effective technique.

In relation to the testbed,

Speed and efficiency is a very important as it is a real-time application.

The design needs to be robust because each agent has specific goals and multi agent interaction added randomness to the test bed.

In terms of believability or effectiveness, this testbed requires effectiveness.

The design of the agents needs to be clear. As stated in chapter 5 a clean interface is one of the most important qualities in a good testbed.

Variety was not a very important requirement in the test bed and was ignored.

All the agents needed to consistently achieve their goals in order to conform to the storyline.

Scalability was very important to allow the addition of more characteristics to the agents as the design progressed.

State machines met all these requirements and therefore were chosen for the design.

Finite state machines are state machine albeit with a fixed (‘finite’) number of states,

7.4 Agent Behaviour and Design

The main focus of this section is on the individual Agent FSM designs. Some of the

behaviour of the base agents has also been mentioned briefly. A brief recap of some of the

basic qualities of the agents has also been mentioned.

7.4.1 Base Agent

The base agent is shown in Appendix D. It has been represented analogous to a remote

controlled car because it has no mind of its own. This is the manual agent or entity that has

been mentioned in section 6.3.1 and belongs to the Agent class. It is controlled by the arrow

keys and has no relevance in the overall gameplay. For this reason it does not have its own

state machine.

45

September 16, 2011


7.4.2 Agent0

The Agent0 has been represented by a skeleton shown in Appendix D. This is to signify that it

is a kind of ‘bare bone’ agent. The on screen behaviour of this agent is, checking for free

spaces in four directions. The next step is to select randomly between any available spaces

and move in that direction. This is the default behaviour for the civilians, police and robbers

which are in turn derived from the class. Agent0 contains sense and effect methods which are

in turn overridden by the derived agents. Agent0 also has no involvement in the game. It is

shown on the game screen to demonstrate the agent hierarchy. It also does not employ the use

of a state machine as it is not part of the game scenario.

7.4.3 Police

The architecture of the police was the simplest out of the 3 agents.

The police were divided into 5 states. These were {IDLE, SEEK, WARN, ARREST AND FOLLOW}.

The overall behaviour of the police is explained through the sensory, processing and effecting modules.

Police icon is shown in Appendix D

Police Goals:

As mentioned previously the police were already present in the game at the time of

initialization. It was assumed that they were informed of the robbery prior to the simulation.

The main goals of the police were

i. Enter the bank and search for insurgents

ii. Prevent the robbery by confronting and arresting the robbers

iii. Ensure the safety of civilians

The implementation of these goals is again, a part of the FSM.

Sensory Module:

There were 3 things that the police were required to sense in the game. The first was the

environment and the location of obstacles. This was derived from the parent classes. The

other two things were the civilians and robbers. Note: Concurrent with human behaviour,

the sense module is constantly active and checking for any changes in the environment.

46

September 16, 2011


The sensing can be both voluntary and involuntary. In the case of the police, the sensing

was voluntary because the goals required a search for other agents in the environment. The

main purpose of the sense module was to gather data from the environment. An example of

data is the type of agent sensed.

Pseudo Code for Sensory Module

Processing module:

In terms of C++, the processing code is included in the sense () method. This was done for

ease of programming. In terms of implementation the processing is carried out after sensing.

Where sensing is the gathering of data, processing is the interpretation of data. Processing

decides what the overall state of the world is depending on the sense data. An example of

processing is evaluating whether a civilian is hostage or not. Another example is evaluating

whether a robber has a hostage or not.

Pseudo Code for Processing Module

Effect Module:

47

Police::sense() {

Switch(Get Agent type)

case CIVILIAN:

{

sense civilian;

init pointer to civilian;

case CIVILIAN:

if(a civilian is in range and is not warned)

{

alertcivilian = true //alert civilian is a boolean flag which is used for the next stage

}

Case ROBBER:

{

if(robber has a hostage ){

do nothing

}else if (robber does not have a hostage){

The robber can be arrested;}

}

case ROBBER:

{

sense robber;

init pointer to robber ;

}

September 16, 2011


In terms of C++ code the effecting module has been represented by the update loop of each

agent.

The FSM is part of the effect module which in turn was responsible for the final actions of the

agents. The states are selected according to the output of the processing stage.

The police FSM is shown in Figure 20.

Figure 20 Police FSM

This has been explained in conjunction with some of the implementation in code.

The initial state of the Police is IDLE. It then checks if any other agent is in its line of sight. If

not It processes that no agent is present and the state is changed to SEEK.

In this state the police patrol from door to door of each room searching for civilians or

robbers. If a civilian is detected in sight distance, it processes if it is a hostage or not. If not, it

changes its state to WARN and instructs the civilian to run out of the bank.

Note: The interaction here between the Civilian and Police. The information gathering

at this stage has been extended from the world to other agents.

48

September 16, 2011


If the civilian is a hostage it cannot do anything and continues in the SEEK state. If a robber

is detected in sight distance, it checks whether it has a hostage or not. If not then the state

changes to FOLLOW in which case the policemen chases after the robber. Once it reaches the

ROBBER the state changes to ARREST.

If the ROBBER escapes out of SIGHT distance, the state changes back to SEEK. In ARREST

a robber is taken to jail, which is also indicated on one corner of the map. After this the Police

revert to the IDLE state and the whole process starts all over again.

7.4.4 Civilian

The civilian is shown in Appendix D.

The states for the civilian were {IDLE, WANDER, FLEE, FOLLOW, SAFE}

Civilian Goals:

The civilian goals are as follows

i. Move from one point to another, to simulate daily activities in a bankii. Flee to safety iii. Involuntary goal of following a robber when taken hostage

Sensory Module:

The sensory module for the civilian differs from the police because it is involuntary when

checking for agents. The sense code for the civilian senses only the robber because one of its

goals is to run when a robber is in sight. Instead of sensing the police the civilian

acknowledges a message of warning to run out from the bank.

The civilian was not required to sense the police because there was no requirement to initiate

an interaction by the civilian.

For this reason, the civilian was designed to receive messages from both agents. These

messages were designed to interrupt a fulfilment of a desired final state by the civilian. For

example if a civilian is fleeing, a message can be sent to follow as a hostage. The Pseudo

code for the sense module is therefore fairly simple for the civilian as compared to the effect

module.

49

September 16, 2011



Processing Module:

The processing stage of the civilian indicates to the next stage that a robber has been sensed.

Along with this it initiates a pointer to the robber which says that ‘the civilian can now

acknowledge messages from the robber’. How it interprets the messages to change states is

shown in the next section.


50

Civilian::sense() {


case Police:

{

sense civilian;

init pointer to robber ;

Case ROBBER:

if(a robber is in sight range)

{

_senseRobber = true //alert civilian is a boolean flag which is used for the next stage

_robber = rob //Initiate a pointer to the robber class so that a robber can send messages.

}

September 16, 2011


Effect Module:

The civilian behaviour is an example of partial fulfilment of goal states. Another

characteristic derived from chapter 5. The game engine or testbed allows for easy interaction

between the agents. There is support to allow for one agent to manipulate the workings of

another agent. The civilian was an entity that required such behaviour. A clean design

allowed this without changing any previous code.

The civilian FSM can be seen in figure 21.

Figure 21 Civilian FSM

As with the police, the initial state is IDLE. From this state the civilian can move to three

states FLEE, WANDER and FOLLOW. If the civilian does not detect any robbers in its line

of sight, it will go to a wander state during which is just moves randomly around the

environment.

51

September 16, 2011


From the IDLE and WANDER state, if a civilian sees a robber in its line of sight, the state

will change to FLEE.

If the civilian reaches the safe point, the state becomes SAFE and the game ends for the

civilian.

The most important state in the civilian design is the FOLLOW state. It can be seen that a

civilian can go to this state from all the available states. This is the state in which the civilian

receives a message from the robber that instructs it to FOLLOW. So while it is a fulfilment of

the FOLLOW state, it occurs as a partial fulfilment of all the other states.

For this reason, the warning message from the police to the civilian has not been implemented

in this way because the police cannot warn the civilians whenever they want whereas the

robbers can take a civilian hostage while it is in any state.

For a broader scope once a robber escapes, the civilian is released and it goes to FLEE and

finally to safe when the game ends for the CIVILIAN.

Note that the hostage command is given to the civilian only when the robber reaches the

position of the civilian. This is to demonstrate that the robber has to chase the civilian to take

it hostage.

7.4.5 Robber

The robber was the most complicated design of all the agents. The robber can be seen as a combination of both the civilian and the police classes. Like the policeman it continues on its goal path but at some instant it can be interrupted causing partial satisfaction of goal states.

Where it differs from the civilian is that it cannot be interrupted from all its states. The robber icon is also shown in Appendix D.

The robber states were {IDLE, FLEE, HEIST, ESCAPE, CATCH, SAFE, ARRESTED}

It can be seen that having an FSM as large as this is very hard to manager. For this reason the behaviour of the robber was broken down into two parts. The states can be represented as {IDLE, FLEE, HEIST} leading to {ESCAPE, CATCH, SAFE, ARRESTED}.

The robber goals were

i. Reach the bank vaultii. When bank vault reached, take a hostage to ensure the escapeiii. Avoid all police encounter unless bank vault reached, in which case it tries to escape as

quickly as possible.

52

September 16, 2011


Before explaining the robber’s sensory module, it needs to be made clear that not all aspects of agent behaviour could be made flexible. The reason for this is that certain behaviours created dominance of one agent over the other. This point has been elaborated in the FSM section for the robber.

Sensory Module:

The robbers had previous knowledge of the bank vault location from the start. This was because of the pathfinder class. In terms of sensing the robber is the same as the police.


Processing Module:

Due to its complexity, the robber also has the most data to process.

The first thing the robber has in common with all agents is that it senses the agents on the basis of sight distance. If in the processing stage, the sensed agent is still in sight it executes the next command.

The robber had interactions with both civilians and the police. For this reason pointers are initiated to both those agents in order to pass or receive messages. The robber passes a message to the civilian and receives a message from the police.

The robber also has to check whether a civilian has been taken hostage by another robber. In this case it will not attempt to take that civilian hostage.

53

Robber::sense() {


case CIVILIAN:

{

sense civilian;

init pointer to civilian;

}

case POLICE:

{

sense police;

init pointer to civ ;

}

September 16, 2011



Effect Module:

The robber FSM can be seen in figure 22.

The robber proceeds from the IDLE state to the HEIST state during which it moves towards the bank vault. If it detects a policeman at this time it moves to the FLEE state and tries to run back to the start point. This is repeated until the robber reaches the bank vault. As mentioned earlier it can be seen that these 3 states are sectioned off from the other states in the FSM.

Once the robber reaches the vault it moves into the second section of the FSM. At this point it tries to ESCAPE the bank. During this process if it senses a civilian in sight distance, it proceeds to the CATCH state, takes the civilian hostage and returns to the ESCAPE state and continuous to move towards the exit. If it has a hostage it is guaranteed an escape because the police cannot arrest the robbers in that condition. If it does not have a hostage the robber can be interrupted through a partial satisfaction of its goal states into an ARRESTED state which ultimately leads to the game ending for the robber. The FSM was divided so that the taking of a hostage is only possible in the second half of the FSM. This was to give the police a fair chance. The police can intercept the robber from any state and force it into an ARRESTED state. The police was designed to arrest the robber on reaching the same cell position as the robber.

54

Case:POLICE

If(police is in sight){

_sensePolice = true ;

_police = pol ; // pointer to police

}

Case: CIVILIAN

if(civilian is in sight and not a hostage)

_senseCivilian = true ;

_civilian = civ ;

}else if (civilian is hostage) {

_caughtHostage = true ; //output flag which leads to robber ignoring a hostage in FSM

}

September 16, 2011


Figure 22 Robber FSM

To fill in any gaps this chapter has left out it would be beneficial to recap what has been said.

The core behavioural design of the agents works in 3 sections or modules. The first module senses the environment and gathers all the data. The second processing module sets conditional Boolean flags on the basis of this data. These Boolean flags are then used to arbitrate over the FSM in the third effect module. For a clearer understanding inspection of the source code included with this report should shed further light on the subject.

55

September 16, 2011


Chapter 8 Emotion ArchitectureOnce the FSM had been implemented, the next step was emotions. At this point a lot had been achieved by the FSM architecture alone.

It is worth noting here that the FSM architecture is somewhat analogous to what is known as a Belief-Desire-Intention Architecture (BDI). Due to its similarity to a reactive goal-based architecture, the BDI architecture can be used to contextualize, some of the considered emotion architectures.

The research on this topic has been on the work of Haddadi and SunderMeyer (O’Hare and Jennings 1996).

Belief can be described as the knowledge the agent has of its environment. In the case of this project, the sense module is somewhat similar to gathering knowledge of the environment.

Desires can refer to the possible set of actions an agent might want to carry out. For example, the list of goals of the agents.

Intention refers to the selection of one guaranteed action out of the desires which will result in an action. For example, State selection and action.

One way of implementing emotions would be for the emotions to affect the desires of an agent. Once an emotion is activated it will only allow a possible set of desires, which in turn will result in specific actions.

In the context of this project this would mean that emotions would be implemented in the processing module and depending on the kind of emotion only certain states would be available in the FSM.

An alternate approach to this implementation model is to allow emotions to govern the

decision making process of the agent (Shihab 2009). Again in the context of this project, this

model will allow emotions to affect the state changes of the FSM. Where the previous BDI

approach would be to put emotions one layer above the Desires, this approach would put it

one layer above a set of intentions available for every desire. This approach assumes that an

agent starts with an emotion. Depending on the emotion, the agent makes a decision, which

leads to an action. The emotion also varies according to the environment and other agents and

their interactions. It can also be affected by a choice of a previous action/or decision.

Even though this approach seems ideal, there was one problem with it. It assumes that an

agent has to start with an emotion and is constantly influenced by this. The initial state of an

agent is dependent on an initialised value of emotion. In the context of this game this is

56

September 16, 2011


unwanted in many situations. It would be more interesting if the effects of emotion can be

shown in comparison to the reactive architecture rather than taking full control of the design.

An approach had to be found that was more suitable for reactive goal based architectures. One

such approach is known as Affective Action Selection and Behaviour Arbitration (AASBA)

(Scheutz 2002). The author’s original implementation is based on behaviour arbitration in

autonomous robots.

This scheme is similar to the first approach mentioned in relation to BDI architectures. The

scheme involves creating a new layer of affective states above the current reactive

architecture. These states are used to arbitrate the behaviours of the agents. The affective

states are designed with their own activation level. If one affective state is active, it controls

the corresponding behaviours in the system.

The means of activation of can vary over different systems and test beds. Scheutz proposed

that affective states be modelled with numerical values and also defined a formula for the

activation of each state. The formula is overly complex and irrelevant for this project but

Scheutz’s research paper can be looked at for more knowledge. The affective states then

compete among themselves for superiority in the system. They form what is called a

“competitive cluster”. One arbitration scheme can be that the affective state with the highest

value controls the behaviour of the system .Events in the system increase the value of one

state while decreasing another. Certain events can also reset the values of both states to 0 so

that the default architecture can behave without the effect of these states. This was beneficial

and also solved the problem that Shihab’s model posed. It allowed the possibility of turning

off emotions in the system.

One problem it had was that since it was a complete external layer of arbitration, it could not

be modified by the agent. This was a serious problem because the extended specification of

the project required agents with control over their emotions and agents with personalities.

It was decided that all agents would start with an emotion of a numerical value 0. This means

that the emotion is off. Emotions would be elicited according to the model in section 3.4.

Numerical threshold levels would be specified for each emotion. Events in the system would

cause an increase in the numerical value. Once the increase reached the threshold value, the

emotion would be active and the FSM in the update method would be controlled according to

the emotion in the processing stage. If the arbitration is part of the processing module, over

time the agent can alter the emotions in order to form a personality.

57

September 16, 2011


This scheme is a combination of the first BDI technique and Scheutz’s technique. While it

puts emotions in the processing module, it still allows for internal control of the arbitration

mechanism. In a testbed, these are good characteristics because the arbitration mechanism

requires complete control. Another added advantage is that emotions can be turned off in the

system during run time allowing the default reactive architecture to take over.

8.1 Civilian emotion model From section 4.3 the most obvious emotion for the civilian was fear. The behaviour of the

civilian running away from the robber was already a representation of fear. This had to be

extended in some way to show a different emotion effect for fear.

It was decided to use a numerical value of 0 for the fear. If a civilian is warned by the police

the fear increases to 0.5 and the civilian runs out of the bank. If the civilian senses a robber

the fear goes to 1 which is the activation level. When the emotion is activated a Boolean flag

in the process module is set to true which then affects the FSM in the effect/update stage.

If an event occurs for example, a policeman then senses a civilian it will lower the level of

fear back to 0.5 allowing the civilian to run away.

Since the civilian does not have many available actions the activation level is basically on or

off. In a different situation the emotion can be incremented gradually with multiple activation

levels and actions.

8.2 Robber emotion modelThe emotion chosen for the robber was anger. Deriving from section 4.3, when a robber is

trying to reach the vault and is constantly blocked by the police, it experiences anger. This is

because the police are blocking the robbers from achieving their goals. If the threshold for

anger is activated a robber chooses to ignore the threat of the police in order to achieve its

goal.

This is an example of what is known as the Nash equilibrium (Nash 1950). What this means

is that without anger the robber and the police are in equilibrium. None of the two can make a

move against each other that will result in an advantage. When influenced by anger the robber

chooses to momentarily gain a disadvantage by risking arrest in the hope of gaining an

advantage later i.e. getting to the vault quicker.

58

September 16, 2011


8.3 Police Emotion model The ideal emotion for the police was disgust. If a policeman was part of the robbery and

allowed the robbers to escape the other policeman would feel disgust. On the activation of

disgust, the police would be less involved in their duties. Instead of SEEK, they would

prominently stay IDLE allowing the robbers to escape easily.

59

September 16, 2011


Chapter 9 Conclusion 9.1 Critical Evaluation

Overall, the project provided a wide range of knowledge on many different topics in the field

of games programming and AI design. Invaluable experience was gained which will be

helpful for future projects. I was very happy with my software architecture because even

though it has room for improvement, it does not require complete redesign. A lot of time was

spent on the game engine and for good reason. The game engine took around 4 weeks to

implement. This was a gamble which paid off as the rest of the software implementation was

quicker because of the clean design of the engine.

I felt that the research of the project was sufficient but pinpointing the exact course of action

was the toughest task during the implementation. The multiple topics involved required strict

planning and time management. This was one of the major areas where I had problems. I

believe that I could have utilized my time better for a more efficient outcome.

The biggest drawback was that I was not able to do sufficient testing of the emotion models.

For this reason it has not been mentioned in the report. All the implementation and theory that

has been mentioned was successfully implemented and the explanations were given on

account of the success of the implementation. I cannot however, successfully provided

exhaustive evidence of an implementation for a commercial video game.

7.2 Future Work

Future work involves the refining of the emotion model. The model needs to be extended by implementing more than one emotion successfully.

Other work includes the implementation of multiple emotions at the same time. This was part of the extended specification of the project.

Finally majority of the future work involves refining the current project. If there was more time available I would spend it in improving the standards of the testbed and if time permitted implementing the extended specification.

60

September 16, 2011


7.3 Conclusions

The required outcome of the dissertation was to find out how emotions can make games more

interesting. In terms of theoretical design and research this requirement was achieved. I am

satisfied with the documentation of the project.

I am also satisfied with the design of the software but the fact that I could not refine it to

completion is disappointing.

If I were to start again, I would pay more attention to planning and time management.

In terms of learning outcomes I learnt to successfully implement a hierarchical and systematic

Agent design. I was able to improve on my programming experience in C++. I was also able

to build a small game engine that I intend to use for my next 2D game project.

I learnt to implement some of the basic techniques of AI design. FSMs being one of these

techniques, was the most logical aspect of the design.

The most important thing I learnt was top think in terms of an AI programmer. The

knowledge gained in this report will provide a valuable base for work in the games industry.

61

September 16, 2011


62

September 16, 2011


References and Bibliography

1. "Affective Computing" MIT Technical Report #321 (Abstract), 1995

2. .Cannon-Bard theory." Encyclopædia Britannica. Encyclopædia Britannica Online.

Encyclopædia Britannica, 2011. Web. 03 Aug. 2011

3. .Hadadi, A and SunderMeyer,K, 1996 , Foundations of Distributed Artificial

Intelligence ,Wiley-IEEE

4. Agre , PE and Chapman, D, 1987, Pengi:An implementation of a theory of activity, MIT

Intelligence laboratory.

5. Andronache, V. and Scheutz, M. ContentionScheduling: A Viable Action-Selection

6. Bourg, D M, 2004, AI for Game Developers, USA: O’Reilly Publishers

7. Boyle B, Connoly M, Hainey T, 2011, The role of psychology in understanding the impact

of computer games, University of West Scotland

8. Brams J. Steven,1995 Game Theory and Emotions, New York University

9. Buckland, M, 2004, Programming Game AI by Examples, USA: Wordware Publishing

Inc.

10. Champandard , A J, 2003, AI Game Development,USA, New Riders

11. Darwin, Charles (1872). The Expression of Emotions in Man and Animals.

12. Davis, D N, 2000, Minds have personalities-Emotion is the core, AISB2000-AIand

Society, Univrsity of Birmingham.

13. Davis, DN , 2011 , Lecture Notes : Advanced Rendering and AI , University of Hull

14. de Sousa, Ronald. (2010). Emotion. Available:

http://plato.stanford.edu/archives/spr2010/entries/emotion/. Last accessed 15th September

2011.

15. Encyclopaedia Britannica. (2011). Cannon-Bard theory. Available:

http://www.britannica.com/EBchecked/topic/92792/Cannon-Bard-theory. Last accessed

15th September 2011

16. Feist G and Rosenberg, E (2009). Psychology: Making Connections. England: McGraw-

Hill Humanities/Social Sciences/Languages.

17. Feist, Gregory and Rosenberg, Erika, , 2009, Psychology: Making Connections ,

McGraw-Hill Higher Education

63

http://vismod.media.mit.edu/pub/tech-reports/TR-321-ABSTRACT.html

http://affect.media.mit.edu/pdfs/95.picard.pdf

September 16, 2011


18. Gaulin, Steven J. C. and Donald H. McBurney (2003). Evolutionary Psychology. 2nd ed.

US: Prentice Hall. p 121-142.

19. Hanks S , Pollack E. M, Cohen R.P, 1993, Benchmarks, Test Beds, Controlled

Experimentation and the Design of Agent Architectures, AI magazine volume 14

20. Huitt, W. (2003). THE AFFECTIVE SYSTEM.

Available:http://www.edpsycinteractive.org/topics/affect/affsys.html. Last accessed 15th

September 2011

21. James, William. (1884). “What is an emotion?” Mind, 9, 188-205.

22. Lees, Ma , Logana, B,Theodoropoulosb 2005, Agents, Games and HLA, a University of

Nottingham, b University of Birmingham

23. Lewis, Suzanne C, Computational Models of Emotion and Affect, PhD dissertation,

University of Hull

24. Mechanism for Robotics? In Proceedings of MAICS 2002, AAAI Press, 2002.

25. Millington, I and Funge, J,2004, Artificial Intelligence for Games 2nd Ed.,USA , Morgam

Kauffman Publishers

26. Nash, John (1950) "Equilibrium points in n-person games" Proceedings of the National

Academy of Sciences 36(1):48-49.

27. Nilsson, Nils J, 2000, Artificial Intelligence: A New Synthesis, San Francisco: Morgan

Kauffman Publishers

28. Russell, Stuart J.; Norvig, Peter, 2003. Artificial Intelligence: A Modern Approach. 2nd

ed. New Jersey: Prentice Hall.

29. Schachter, S. and Singer, J. E. , 1962 Cognitive, social and physiological

determinants of emotional states, Psychological Review, 69, 379-399

30. Scheutz, M, 2002, Affective action selection and behaviour arbitration for Autonomous

Robots, University of Notre Dame

31. Shihab K.,2009, Emotional Agents in Computer Games,International Journal of

Computers

64

September 16, 2011


65

September 16, 2011


Appendix A Original Specification

Affective computation is starting to attract considerable interest in a number of computational

domains. One such area is the use of affect in computational agents that allow more involving

interactive game playing. This project will address how a computational model of emotion

(affect) can be used to make better computer games. A relatively simple scenario (e.g.

predator-prey) can be used to investigate the effect of using different emotions or affect and

perhaps different computational models of emotion or affect.

66

September 16, 2011


Appendix B Updated Specification

The updated specification required the selection of a valid computational model of emotion

from the field of psychology. This model of emotion was to be translated into a software test

bed. The software could be designed in any programming language. A scenario had to be

implemented and a storyline had to be selected that could demonstrate emotions in video

games. There was a choice on the architecture of the agent behaviour. Reactive goal based

agents were suggested. Finite State machine were to be used to implement a game that

consisted of reactive goal based agents. Once the default behaviour of the game was achieved,

a model of emotion was to be implemented as a secondary layer on top of the game FSM.

Overall this was to be used to research how Emotions can make games more interesting.

The extended specification included representing agents with personalities and multiple

emotions.

67

September 16, 2011


Appendix C Requirements Analysis Table

68

September 16, 2011


Appendix D Character Icons

Base Agent

Agent0

Police

Civilian

Robber

69

September 16, 2011


Appendix E Project Plan

WEEK SCHEDULED TASK UNSCHEDULED TASK

Week 106/07/11

- Concrete storyline for integration with test bed- Software GUI design using SFML library and C++- Agent(class0) state machine design- Define critical scenarios in storyline-Define quantitative and qualitative evaluation factors for the project

- Refining of software test bed- Document work in draft form

Week 213/07/11

- software implementation of class0 agent FSM- -Test bed complete with class0 agent

- Further refining of test bed-Document work in draft form

Week 320/07/201

1

- Accommodate previous delays- Ensure basic FSM functionality- Extend Class to different roles - Finalise documentation for test bed and background theory

- Basic FSM functionality is mandatory .Extensibility of the design needs to be ensured before progressing further

Week 427/07/201

1

- Implement the 3 types of game charactersCivilians, terrorists and 3 tactical team unitsImplement basic character movementDefine individual character type goalsExtend basic FSM to goal based reactive-Implement testing methodologies-Start draft final report structure

- Integrate goals with storyline- Identify individual FSM design for each agent

Week 503/08/201

1

- Accommodate previous delays-Class1a agent FSM design and state diagram-Class1a class diagram-extend class0 diagramCode implementation clas1a agent

- Finalise list of goals to be implemented example attack rescue, kill, take hostage etc.

70

September 16, 2011


-test class0, class1 and class 1a agents-finish final report structure

Week610/08/201

1

-Continue code implementation-Implement different goal classes for agents-Testing and debugging-Single goal Finite state per agent- Finalise documentation and integrate into draft report

- Complete documentation of all work done until this point- Testing and debugging

Week717/08/201

1

WEEK SCHEDULED TASK UNSCHEDULED TASK

Week 824/08/201

1

- Test and analysis emotions in applicable agent-Document test results-Ensure whether emotions are being demonstratedDocument check-Finalise documentation Draft report complete

- Edit test bed to ensure emotions are being demonstrated

Week 91/09/2011

Ensure all applicable emotion and goal scenarios are functioningComplete software for demonstration

Week 108/09/2011

Complete remaining documentationFinalise report submission

71

September 16, 2011


72

September 16, 2011


Appendix F Considerations

Legal Considerations

All source code used in the project has been designed from the ground up. If any excerpts of source code have been unknowingly used they have been used from a source that is open and free for all. No copyright media or content has been used in this dissertation.

Ethical Considerations

There are no ethical issues involved in this project.

Social Considerations

There are no people involved in the evaluation of the project

Professional Considerations

An attempt has been made in the best respect to cite all information sources to the best of my ability. The SFML software library used for this project is open source and free for all. All the source code used was written either throughout the course of the degree or the duration of the dissertation period.

73

emotional agents in game playing€¦ · web viewemotional agents in game playing. msc dissertation...

Documents