8.3. Agent and Decision Making AIAgent driven AI and associated decision making techniques

1Question Clinic: FAQIn lecture exploration of answers to frequently asked student questions

2AI AgentsUsing an agent driven approach to control game character AI

3An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.Agents may act as an Opponent Ally Neutral character

Game agentsLoops through the following cycle: Sense Think ActOptional learning or remembering step

4Within a game, agents can have access to perfect information about the game world (e.g. complete terrain layout, location and state of the player, etc.). Often a sensing model is used to avoid agent cheating and ensure agents cannot see through walls, know about unexplored areas, etc.

Game agents:SenseThinkAct: Sensing - Sight

Example sensing model: For each game object:Is it within the viewing distance of the agent?Is it within the viewing angle of the agent?Is it unobscured by the environment?

5Do agents respond to sound? If so, how is sound propagation modelled?An event based model is typically used:When sound is emitted, it alerts interested agentsUse distance and zones to determine how far sound can travelTravel distance may also depend upon type of incident surface and movement speed of the player, etc.

Game agents:

SenseThinkAct: Sensing - Sound

6Agents should not normally see, hear and communicate instantaneously (or rather immediately commence actions following the sensing stage)Normally sufficient to introduce artificial reaction times, e.g.:Vision: to secondHearing: to secondCommunication: > 2 seconds

Game agents:SenseThinkAct: Sensing - Reacting

7Approaches towards agent AI decision making (not mutually exclusive) include:Use pre-coded expert knowledgeAlgorithmically search for a solutionMany different techniques exist (we will explore some later)

Game agents:SenseThinkAct: Thinking

Aside: Encoding expert knowledge is appealing as it is relatively easy to obtain and use, but may not be scalable or adaptable. Whilst often scalable and adaptable, algorithmic approaches may not match a human expert in the quality of decision making or be computationally expensive.

8Dumbing down agentsSometimes it may be necessary to dumb down agents, for example:Make shooting less accurateIntroduce longer reaction timesChange locations to make self more vulnerable, etc.

Game agents:SenseThinkAct: Think

Letting agents cheatThis is sometimes necessary for:Highest difficultly levelsCPU computation reasonsDevelopment time reasons

9Sensing and thinking steps are invisible to player, i.e. acting is how player witnesses intelligence. Example of actions include:Move locationPick up objectPlay animationPlay sound effectFire weapon

Game agents:SenseThinkAct: Acting

Agents might also use event-driven communication when within the vicinity of each other to:Alter other agents to some situation (i.e. agent hurt)Share agent knowledge (i.e. player last seen at location x)

10Decision Making AIIntroduction to decision making techniques within game AI

11The input to a decision making process is the knowledge possesses by a game object and the output is a requested action.The input knowledge can consist of internal knowledge (i.e. internal state) and external knowledge (i.e. game world).Likewise, actions can be directed towards changing internal state or the external (world) state.

Decision making techniquesAside: Most games need only use simple decision making techniques such as decision trees and state machines. Rule-based approaches may be needed for more complex needs.Internal knowledgeExternal knowledgeDecision Making ProcessRequested action(s)Internal change (s)External change (s)

12Decision TreesSimple decision making using decision trees

13Decision trees offer a simple, but fast form of decision making.A decision tree consists of a starting decision point, which is connected to more refined decision points. Each leaf contains an action that is executed once reached.The tree can be grown to encapsulate complex behaviour but then often become hard to manage.

Decision trees

14Finite State MachinesThe use of finite state machines to encapsulate a decision process

15A finite state machine occupies one of a finite number of states at any point in time. Actions may be undertaken based on the current state. Inputs to the system can cause a transition from one state to another.Finite State Machines (FSMs)T1T2T3T4AS1AS2AS3In general:Each FSM has a number of possible states {S1 ... SN}Transition functions {T1 ... TM} define the conditions under which a state transition will occur. Every time a state transition occurs and a new state is entered, one or more state actions may be fired {AS1 ... ASN }

16A finite state machine works by decomposing an objects behaviour into defined chunks (states). So long as a character remains in a state it will use the same behaviour.State machines are very widely used, including:Controlling ghosts in Pac-manControlling bots in QuakeSports simulations such as FIFA 2002 NPCs in RTSs such as Warcraft

Finite State Machines (FSMs)

17Each ghost can be in a wander, chase or evade state (each ghost can have a different chase/wander behaviour). Once a powerpill is eaten, all ghosts transition to the evade state, which is exited once the timer expires.Finite State Machines (Pacman example)Pacman in rangePacman out of rangePowerpill eatenPowerpill eatenPowerpill expiredAside: The wander state could be entirely removed in this FSM, i.e. Chase and Evade form the minimum behavioural set.

18State machines along with scripting represent the most common forms of decision making in games, as:They are relatively quick and simple to code and debugThey have little computational overhead (depending on the complexity of transition tests).They are flexible and can often be easily extended or modified.State based behaviour is good for modelling many game-world objects.

Finite State Machines (Examples)

19Simple FSMs cannot easily model all forms of behaviour. One example is alarm behaviour, an action that can be triggered from any state.Consider a robot whose alarm behaviour is to recharge when power levels become low.Using a hierarchical FSM, the states can transition between cleaning up and getting power (at the top level). When in the cleaning up state a lower-level FSMs controls behaviour.

Hierarchical FSMs

20Goal Oriented BehaviourUsing goals to drive behaviour

21GOB is used widely in games such as The Sims.Characters have a range of emotional or physical goals (or motives). Depending on the actions executed by the character (possibly player controlled) the goals (i.e. needs, desires, fears, etc.) will either increase or decrease.GOB algorithms try to fulfil the characters goals by selecting between available actions that influence the goal parameters.Goal Oriented Behaviour (GOB)

22Characters can have a number of currently active goals. Goals might include: eat, seek health, defeat opponent.Each goal has an associated numeric insistence value representing the current importance of that goal.Some goals may be fully achievable (e.g. seek health) others may be only reducible by always remain (e.g. satiate hunger).A set of (possibly situational) actions is presented to the character. The character will select the action that best satisfies their current goal insistence values.Goal Oriented Behaviour (Goals and Actions)

23Consider the shown goals and actions. Which action should be selected?The notion of overall discontentment offers a useful means of selecting the best action.A good discontentment metric is to sum the squares of insistence values and select the action that results in lowest discontentment.More advanced approaches can consider the time to start/complete each activity, or more complex insistence contributions.

Goal Oriented Behaviour (Selecting actions)Goals (Insistence low = 0, high = 5):Eat (4), Sleep(1), Bathroom (3)

Actions:Eat-Food (Eat 3, Bathroom +1)Eat-Snack (Eat 2)Sleep-Bed (Sleep 4)Sleep-Sofa (Sleep 2)Drink-Cola (Eat 1; Bathroom + 3)Visit-Bathroom (Bathroom 4)

Discontentment:

Eat-Food: (4-3)2 + 12 + (3+1)2 = 18Eat-Snack: (4-2)2 + 12 + 32 = 14Sleep Bed: 42 + (1-1)2 + 32 = 25Sleep Sofa: 42 + (1-1)2 + 32 = 25Drink-Cola: (4-1)2 + 12 + (3+2)2 = 35Visit-Bathroom: 42 + 12 + (3-3)2 = 17

24

Summary

To do:If applicable to your game, explore finite-state machines, agents and goal driven behaviour.Word towards your alpha hand-in goals.Today we explored:

The notion of a game AI agentDecision making processes including finite state machines and goal driven behaviour

25